CatalogManagerConfigurations/MasterStressTest.Test/0: ==3947==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190000bf988 at pc 0x000000967ea4 bp 0x7fc79187f570 sp 0x7fc79187f568
READ of size 4 at 0x6190000bf988 thread T54 (rpc worker-4005)
I20260811 18:57:05.681866 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:05.690676 4162 tablet_bootstrap.cc:492] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.690936 4162 ts_tablet_manager.cc:1397] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.003s
I20260811 18:57:05.692020 4162 raft_consensus.cc:348] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.692552 4162 raft_consensus.cc:374] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.692665 4162 raft_consensus.cc:729] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.693018 4162 consensus_queue.cc:260] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.693789 4162 ts_tablet_manager.cc:1428] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260811 18:57:05.694322 4162 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.697446 4167 ts_tablet_manager.cc:1918] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:05.697849 4167 log.cc:1196] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:05.699116 4168 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:05.700865 4168 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:05.707552 4168 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:05.708313 4167 ts_tablet_manager.cc:1939] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.709568 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 103) took 1624 ms. Trace:
I20260811 18:57:05.709802 4167 rpcz_store.cc:276] 0811 18:57:04.084645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.084700 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:05.659586 (+1574886us) ts_tablet_manager.cc:1145] Deleting tablet 3085d74b0bf7447d9c247b7e8872b215
0811 18:57:05.659604 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.682058 (+ 22454us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.697272 (+ 15214us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.709489 (+ 12217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1501326}
I20260811 18:57:05.697554 4162 tablet_bootstrap.cc:654] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.712311 4167 tablet_replica.cc:330] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.712730 4167 raft_consensus.cc:2229] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.713092 4167 raft_consensus.cc:2258] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.714063 4167 ts_tablet_manager.cc:1905] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.690984 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:05.716815 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 6d1d58835ad24749a00ce27b4bc73c59 deleted and replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
W20260811 18:57:05.692198 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6381a951a966473c80670b0dd1aff062 consensus state has no leader
I20260811 18:57:05.693181 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc0e1659d31645b28fe322618e22cb6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.693949 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61c8443df21949ff9bdc63fa6aa34108 consensus state has no leader
W20260811 18:57:05.694329 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 consensus state has no leader
I20260811 18:57:05.695228 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a945f1a824043b6b99704ee0994c6c6" } new_table_name: "default.Table_44e9cbdec47545899467895fd869b389" modify_external_catalogs: true
W20260811 18:57:05.720510 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc0e1659d31645b28fe322618e22cb6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.713681 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5345e1fe57284913bcf778715a827515"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.714715 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_897c0f5693354801afbbaa4de2ad5252"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.723017 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_897c0f5693354801afbbaa4de2ad5252 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:05.725068 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5345e1fe57284913bcf778715a827515 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:05.726899 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44991.8us
W20260811 18:57:05.727439 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 9f8c0444ec2242299482736a73551f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f8c0444ec2242299482736a73551f80 already in progress: creating tablet
W20260811 18:57:05.727957 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 927fb981490b4cecabe40314d6f71048 because the tablet was not found. No further retry: Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
W20260811 18:57:05.728386 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 634c06d76eca404a88f326e905882930 because the tablet was not found. No further retry: Not found: Tablet not found: 634c06d76eca404a88f326e905882930
I20260811 18:57:05.745002 4165 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:05.745405 4165 ts_tablet_manager.cc:1397] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.586s user 0.008s sys 0.004s
I20260811 18:57:05.746659 4165 raft_consensus.cc:348] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.747061 4165 raft_consensus.cc:374] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.747267 4165 raft_consensus.cc:729] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.747720 4165 consensus_queue.cc:260] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.748982 4165 ts_tablet_manager.cc:1428] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:05.749811 4165 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:05.754689 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.756181 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.756213 4167 ts_tablet_manager.cc:1918] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.756481 4167 log.cc:1196] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e355634612f64c14965d7af2fa442cb7
I20260811 18:57:05.757139 4167 ts_tablet_manager.cc:1939] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.758082 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.758745 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 104) took 1664 ms. Trace:
I20260811 18:57:05.758952 4167 rpcz_store.cc:276] 0811 18:57:04.094316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.102510 (+ 8194us) service_pool.cc:227] Handling call
0811 18:57:05.712159 (+1609649us) ts_tablet_manager.cc:1145] Deleting tablet e355634612f64c14965d7af2fa442cb7
0811 18:57:05.712175 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.723874 (+ 11699us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.755916 (+ 32042us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.758687 (+ 2771us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527175}
W20260811 18:57:05.760650 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
W20260811 18:57:05.761179 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:05.761255 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 105) took 1360 ms. Trace:
I20260811 18:57:05.761406 4167 rpcz_store.cc:276] 0811 18:57:04.400675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.401139 (+ 464us) service_pool.cc:227] Handling call
0811 18:57:05.759427 (+1358288us) ts_tablet_manager.cc:1145] Deleting tablet 60274115fe164d85badcf2e72d640f54
0811 18:57:05.759441 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.761199 (+ 1758us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1357839}
W20260811 18:57:05.761651 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
I20260811 18:57:05.762090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 107) took 1330 ms. Trace:
W20260811 18:57:05.762043 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d9421bc220774f66b90bf537a47f1e13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9421bc220774f66b90bf537a47f1e13 already in progress: creating tablet
I20260811 18:57:05.762284 4167 rpcz_store.cc:276] 0811 18:57:04.431171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.434430 (+ 3259us) service_pool.cc:227] Handling call
0811 18:57:05.761813 (+1327383us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:05.761826 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.762015 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1264275}
W20260811 18:57:05.762553 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
I20260811 18:57:05.762825 4167 tablet_replica.cc:330] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:05.762887 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
W20260811 18:57:05.763334 3541 catalog_manager.cc:4920] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]): Aborted: Op aborted
I20260811 18:57:05.763600 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:05.771917 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.789765 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.790685 4167 raft_consensus.cc:2229] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.791879 4167 raft_consensus.cc:2258] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.793298 4167 ts_tablet_manager.cc:1905] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.800308 4293 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.800654 4293 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.801261 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.801910 4293 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.803069 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 with delete_type TABLET_DATA_DELETED (replaced by tablet f61d3822fc2d4b569f223238a6464e0a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.803676 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.804136 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.804675 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.804962 4169 log.cc:1196] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:05.806217 4361 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.807459 4169 ts_tablet_manager.cc:1939] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.808109 4169 tablet_replica.cc:330] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.808348 4169 raft_consensus.cc:2229] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:05.808501 4162 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.808657 4169 pending_rounds.cc:62] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:05.808686 4162 ts_tablet_manager.cc:1397] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.025s
I20260811 18:57:05.808728 4169 pending_rounds.cc:68] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400066246193152 op_type: NO_OP noop_request { }
I20260811 18:57:05.808895 4169 raft_consensus.cc:2866] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:05.809022 4169 raft_consensus.cc:2258] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.810199 4162 raft_consensus.cc:348] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.810496 4162 raft_consensus.cc:374] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.810648 4162 raft_consensus.cc:729] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.810912 4162 consensus_queue.cc:260] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.811551 4162 ts_tablet_manager.cc:1428] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:05.811920 4162 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.813735 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b70decfaca9c4a9f87915d80838c6061" } modify_external_catalogs: true
I20260811 18:57:05.813961 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b70decfaca9c4a9f87915d80838c6061" } modify_external_catalogs: true
I20260811 18:57:05.814060 4162 tablet_bootstrap.cc:654] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.802270 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:05.815470 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0190cbd859274fcfa09c6fab899c4843 consensus state has no leader
W20260811 18:57:05.815656 3824 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.816035 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5df7a07983174dd1aaf48951c9ff1a1b (table default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]) successfully deleted
I20260811 18:57:05.806510 4361 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.816797 4163 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.819335 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:05.833860 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7b3a14979e84b1e8ec6ed3f49b93b00 consensus state has no leader
I20260811 18:57:05.835023 4163 ts_tablet_manager.cc:1397] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.362s user 0.051s sys 0.111s
W20260811 18:57:05.836998 3824 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.839376 4163 raft_consensus.cc:348] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.840581 3824 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.841813 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.846468 4163 raft_consensus.cc:374] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.846694 4163 raft_consensus.cc:729] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.847237 4165 tablet_bootstrap.cc:654] T 5df7a07983174dd1aaf48951c9ff1a1b P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.847242 4163 consensus_queue.cc:260] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.848321 4293 raft_consensus.cc:2726] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.848657 4169 ts_tablet_manager.cc:1905] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:05.849243 3824 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.850051 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.850824 4361 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.852511 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.853904 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.855125 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 3085d74b0bf7447d9c247b7e8872b215 (table default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]) successfully deleted
I20260811 18:57:05.855394 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e355634612f64c14965d7af2fa442cb7 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]) successfully deleted
W20260811 18:57:05.855660 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 60274115fe164d85badcf2e72d640f54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60274115fe164d85badcf2e72d640f54 already in progress: creating tablet
W20260811 18:57:05.855899 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2852f9e5f67c4025836018af553388a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2852f9e5f67c4025836018af553388a5 already in progress: creating tablet
W20260811 18:57:05.857264 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 6d1d58835ad24749a00ce27b4bc73c59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d1d58835ad24749a00ce27b4bc73c59 already in progress: creating tablet
I20260811 18:57:05.863099 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.863719 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.863950 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.865381 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.865515 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.866580 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.867216 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.868259 4011 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.869450 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.871428 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.871469 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.872066 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.872123 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.873111 4230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:05.874279 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.874934 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.875030 4230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:05.875680 4230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:05.875626 4000 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.876298 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.874114 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.879062 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.879623 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.881070 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.881732 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.883306 4001 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.883769 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.883880 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.885380 4163 ts_tablet_manager.cc:1428] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.001s sys 0.004s
I20260811 18:57:05.887112 4013 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.887954 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.893831 3824 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.895037 3824 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.895191 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.895515 4383 raft_consensus.cc:2726] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.896454 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.896688 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.897257 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.907204 3867 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.908439 3998 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.908874 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:05.910063 3541 catalog_manager.cc:4914] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]),no further retry: Not found: Tablet not found: 6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:05.910423 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b591f65f304846aab2374c6ada0dd0ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.910794 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.911376 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b591f65f304846aab2374c6ada0dd0ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.912631 3996 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.914638 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.915521 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.917433 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.921696 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.921799 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.922777 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.924325 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.929227 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.930433 4167 ts_tablet_manager.cc:1918] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.933758 4167 log.cc:1196] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/634c06d76eca404a88f326e905882930
I20260811 18:57:05.934602 4167 ts_tablet_manager.cc:1939] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.936340 4036 raft_consensus.cc:1261] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:05.936673 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 106) took 1506 ms. Trace:
I20260811 18:57:05.936842 4167 rpcz_store.cc:276] 0811 18:57:04.429880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.429949 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:05.762713 (+1332764us) ts_tablet_manager.cc:1145] Deleting tablet 634c06d76eca404a88f326e905882930
0811 18:57:05.762725 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.868673 (+105948us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.874398 (+ 5725us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.936171 (+ 61773us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55,"tablet-delete.queue_time_us":1265169}
I20260811 18:57:05.938740 4167 tablet_replica.cc:330] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.939134 4167 raft_consensus.cc:2229] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.940948 4330 consensus_queue.cc:1035] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:05.945011 4163 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.947678 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.948354 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.956204 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.957475 4167 raft_consensus.cc:2258] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.960567 4162 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.962468 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.963029 4162 ts_tablet_manager.cc:1397] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.149s user 0.024s sys 0.035s
I20260811 18:57:05.963850 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.965016 3676 consensus_peers.cc:487] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd3b82603ec340ed9cabe9b2ff62a8b3. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:05.965317 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.962829 4167 ts_tablet_manager.cc:1905] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.966702 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.967572 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.969127 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.969841 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.971382 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.972699 3859 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.973729 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.973726 4162 raft_consensus.cc:348] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.974395 4162 raft_consensus.cc:374] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.974606 4162 raft_consensus.cc:729] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.976161 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.977972 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.978642 3892 raft_consensus.cc:1261] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:05.980640 4162 consensus_queue.cc:260] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.981122 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.982858 4162 ts_tablet_manager.cc:1428] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.020s user 0.000s sys 0.004s
I20260811 18:57:05.982683 4330 consensus_queue.cc:1035] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:05.984994 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]) successfully deleted
W20260811 18:57:05.985445 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5ae200f3814e431a8288c30fe94bffcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ae200f3814e431a8288c30fe94bffcf already in progress: creating tablet
I20260811 18:57:05.985698 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_923c4369260d436fafb1aff17a2ec8d4" } modify_external_catalogs: true
I20260811 18:57:05.986123 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_923c4369260d436fafb1aff17a2ec8d4" } modify_external_catalogs: true
W20260811 18:57:05.985762 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 508f0b39cd654aa991a2928a4b067762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 508f0b39cd654aa991a2928a4b067762 already in progress: creating tablet
I20260811 18:57:05.994525 3866 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.016705 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.017720 4162 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.028331 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.030262 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.030961 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.036207 4383 raft_consensus.cc:482] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.036752 4383 raft_consensus.cc:504] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.037345 4361 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.037642 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.038144 4361 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.039094 4361 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.039611 4330 raft_consensus.cc:482] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.039887 4330 raft_consensus.cc:504] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.042038 4330 leader_election.cc:288] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.046414 4383 leader_election.cc:288] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:06.052639 3676 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8efb663060ef457cbece962fce701eb1
I20260811 18:57:06.069773 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db" } modify_external_catalogs: true
I20260811 18:57:06.070166 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db" } modify_external_catalogs: true
W20260811 18:57:06.070314 3678 consensus_peers.cc:487] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd3b82603ec340ed9cabe9b2ff62a8b3. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:06.070830 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 58245us
I20260811 18:57:06.070564 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.071270 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91346cfabb664d0ebfc876dacc125c50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.072400 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91346cfabb664d0ebfc876dacc125c50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.073751 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d22535c5f02a4044aad07546138fca10" } modify_external_catalogs: true
I20260811 18:57:06.074139 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d22535c5f02a4044aad07546138fca10" } modify_external_catalogs: true
I20260811 18:57:06.075070 4330 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.075449 4330 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.080137 4330 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.085486 4162 tablet_bootstrap.cc:654] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.095043 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8397339a34974af9b4e8f20778beb8c7" } modify_external_catalogs: true
I20260811 18:57:06.095407 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8397339a34974af9b4e8f20778beb8c7" } modify_external_catalogs: true
I20260811 18:57:06.098496 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:06.099354 3893 raft_consensus.cc:2454] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.099704 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.104689 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.104722 4035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efb663060ef457cbece962fce701eb1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.112593 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.113306 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f53cc0009ca74325a65d63adf0744d37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:06.113862 3857 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.114264 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f53cc0009ca74325a65d63adf0744d37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.115325 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.119138 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.119303 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.124693 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:06.126528 3824 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
W20260811 18:57:06.127118 3824 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.137518 4330 raft_consensus.cc:482] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.137777 4330 raft_consensus.cc:504] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.138669 4330 leader_election.cc:288] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.138947 3676 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.139667 4330 raft_consensus.cc:2781] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.139832 4330 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.139940 4330 raft_consensus.cc:3037] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.144912 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:06.147769 3893 raft_consensus.cc:2454] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.162649 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.163208 3743 raft_consensus.cc:2454] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:06.164959 3676 leader_election.cc:302] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.165971 4163 tablet_bootstrap.cc:654] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:06.172581 3678 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.172935 3678 leader_election.cc:302] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.173807 4331 raft_consensus.cc:2726] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:06.174474 3678 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.175990 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.184255 3824 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.184926 4331 raft_consensus.cc:2781] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.185660 4383 raft_consensus.cc:2781] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.185902 4383 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:06.186657 3678 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.187047 4383 raft_consensus.cc:3037] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.192780 4331 raft_consensus.cc:482] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.195338 4383 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.197324 4167 ts_tablet_manager.cc:1918] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:06.197702 4167 log.cc:1196] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/41558efe0344434db6e755804442d2de
I20260811 18:57:06.197706 4383 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.198735 4331 raft_consensus.cc:3037] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.201105 4167 ts_tablet_manager.cc:1939] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.202100 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 118) took 1256 ms. Trace:
I20260811 18:57:06.202965 4167 rpcz_store.cc:276] 0811 18:57:04.945685 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.945739 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:05.937896 (+992157us) ts_tablet_manager.cc:1145] Deleting tablet 41558efe0344434db6e755804442d2de
0811 18:57:05.937907 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.083745 (+145838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.195291 (+111546us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.201997 (+ 6706us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":967008}
W20260811 18:57:06.203127 3824 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
I20260811 18:57:06.203626 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 41558efe0344434db6e755804442d2de (table default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]) successfully deleted
I20260811 18:57:06.206404 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 119) took 1171 ms. Trace:
I20260811 18:57:06.212512 3824 leader_election.cc:302] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.215566 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:06.220250 3824 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.221069 4383 raft_consensus.cc:2726] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.221104 4167 rpcz_store.cc:276] 0811 18:57:05.034411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.041946 (+ 7535us) service_pool.cc:227] Handling call
0811 18:57:06.206012 (+1164066us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:06.206030 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.206239 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1145408}
W20260811 18:57:06.222077 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d106e0e1f4b845429b10f4e100f69a8b already in progress: creating tablet
I20260811 18:57:06.222865 4162 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.222856 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:06.223249 3743 raft_consensus.cc:3037] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.223268 4162 ts_tablet_manager.cc:1397] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.240s user 0.009s sys 0.014s
I20260811 18:57:06.223661 4330 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.227701 4330 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.228402 4162 raft_consensus.cc:348] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.229046 4162 raft_consensus.cc:374] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.229205 4162 raft_consensus.cc:729] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.229590 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.229559 4162 consensus_queue.cc:260] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.233726 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.238274 3893 raft_consensus.cc:3037] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.247538 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.257014 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 133501us
I20260811 18:57:06.260535 4162 ts_tablet_manager.cc:1428] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260811 18:57:06.262519 4167 tablet_replica.cc:330] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:06.263005 4167 raft_consensus.cc:2229] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:06.263157 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet d19675f757674450aceb5942e1a51c99
I20260811 18:57:06.263561 4167 pending_rounds.cc:62] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:06.263619 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d3d6daff4b7540309ff58755758e9752
I20260811 18:57:06.263855 4167 pending_rounds.cc:68] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400064111112192 op_type: NO_OP noop_request { }
I20260811 18:57:06.264109 4167 raft_consensus.cc:2866] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.264624 4167 raft_consensus.cc:2258] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.265599 4167 ts_tablet_manager.cc:1905] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.268265 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.271178 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:06.272953 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
I20260811 18:57:06.273943 3893 raft_consensus.cc:2454] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:06.275565 3676 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:06.281838 3678 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.284276 4331 raft_consensus.cc:504] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.285562 4331 leader_election.cc:288] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.287712 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.288713 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:06.289007 3678 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.289453 4330 raft_consensus.cc:2781] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.289837 4330 raft_consensus.cc:686] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:06.291563 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.291625 4330 consensus_queue.cc:237] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.292325 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.293363 3893 raft_consensus.cc:3037] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.294724 3743 raft_consensus.cc:2454] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:06.306098 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:06.307785 3824 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.310585 4383 raft_consensus.cc:2781] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.310982 4383 raft_consensus.cc:686] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:06.312232 4162 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.312981 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:06.315131 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
I20260811 18:57:06.315855 4383 consensus_queue.cc:237] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.320593 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.321323 4330 raft_consensus.cc:482] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.321566 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.322184 4330 raft_consensus.cc:504] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.323642 4330 leader_election.cc:288] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.323998 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b092ff08e7a340bfb2b5ad86895ca81d" } new_table_name: "default.Table_666e839163c3494a9c95e78ff76bb015" modify_external_catalogs: true
I20260811 18:57:06.325218 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df7a07983174dd1aaf48951c9ff1a1b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.326200 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3392d9de7234297a490a976dfe10060"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.326767 3678 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.327031 4169 ts_tablet_manager.cc:1918] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:06.327397 4169 log.cc:1196] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/82d1339366a24a87b74bd1d411a3f6fe
W20260811 18:57:06.327450 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3392d9de7234297a490a976dfe10060 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.328346 4169 ts_tablet_manager.cc:1939] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.329002 4169 tablet_replica.cc:330] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:06.329142 3676 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:06.329264 4169 raft_consensus.cc:2229] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.329332 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:06.329587 3676 leader_election.cc:302] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.329738 4169 raft_consensus.cc:2258] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:06.330328 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.330476 4330 raft_consensus.cc:2726] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.331873 4163 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.332146 4163 ts_tablet_manager.cc:1397] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.446s user 0.004s sys 0.009s
I20260811 18:57:06.332564 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a108c0f554824b94acc46ae9330d1115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.333354 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a108c0f554824b94acc46ae9330d1115 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.333909 4163 raft_consensus.cc:348] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.334234 4163 raft_consensus.cc:374] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.334882 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 82d1339366a24a87b74bd1d411a3f6fe (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:06.335670 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_27837256c8254b8c9b6493a7ca044751" } new_table_name: "default.Table_a5043728e7c842ea851193559d898d80" modify_external_catalogs: true
I20260811 18:57:06.336783 4162 tablet_bootstrap.cc:654] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.339680 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_314cf78c221448d1a5fa8708b3adf8b7" } new_table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" modify_external_catalogs: true
I20260811 18:57:06.343963 4167 ts_tablet_manager.cc:1918] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:06.349668 3541 catalog_manager.cc:4914] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]),no further retry: Not found: Tablet not found: 6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:06.344259 4167 log.cc:1196] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:06.357563 4167 ts_tablet_manager.cc:1939] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.358974 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 120) took 1225 ms. Trace:
I20260811 18:57:06.359273 4167 rpcz_store.cc:276] 0811 18:57:05.133662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.133803 (+ 141us) service_pool.cc:227] Handling call
0811 18:57:06.226478 (+1092675us) ts_tablet_manager.cc:1145] Deleting tablet cd3b82603ec340ed9cabe9b2ff62a8b3
0811 18:57:06.262393 (+ 35915us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.311285 (+ 48892us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.343705 (+ 32420us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.358890 (+ 15185us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091411}
I20260811 18:57:06.360241 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:06.371124 4411 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:06.377825 4169 ts_tablet_manager.cc:1905] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.363629 4163 raft_consensus.cc:729] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.381649 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 121) took 1245 ms. Trace:
I20260811 18:57:06.382066 4167 rpcz_store.cc:276] 0811 18:57:05.134924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.135189 (+ 265us) service_pool.cc:227] Handling call
0811 18:57:06.379474 (+1244285us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:06.379488 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.380712 (+ 1224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1243814}
I20260811 18:57:06.382404 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_178c66b5e0f1479a9a97248ecab4e779" } new_table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" modify_external_catalogs: true
I20260811 18:57:06.383283 3893 raft_consensus.cc:2454] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:06.383301 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9f8c0444ec2242299482736a73551f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f8c0444ec2242299482736a73551f80 already in progress: creating tablet
I20260811 18:57:06.384034 4167 tablet_replica.cc:330] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:06.384001 3676 leader_election.cc:302] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.384857 4331 raft_consensus.cc:2781] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.384253 4163 consensus_queue.cc:260] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.388015 4331 raft_consensus.cc:686] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:06.388525 4163 ts_tablet_manager.cc:1428] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.056s user 0.000s sys 0.004s
I20260811 18:57:06.388682 4331 consensus_queue.cc:237] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.391021 4162 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.391487 4162 ts_tablet_manager.cc:1397] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.079s user 0.003s sys 0.008s
I20260811 18:57:06.393412 4162 raft_consensus.cc:348] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.395352 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 has no consensus state
I20260811 18:57:06.396324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a594cb48b586458783a36cc76df1332c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.397173 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a594cb48b586458783a36cc76df1332c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.402638 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4eaba0dd4bfc4e45878b027d222974fd" } new_table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" modify_external_catalogs: true
I20260811 18:57:06.410452 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.410662 3720 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.411190 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.411438 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.410691 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.415326 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.424027 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.424691 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.424964 4162 raft_consensus.cc:374] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.425091 4162 raft_consensus.cc:729] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.425494 4162 consensus_queue.cc:260] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.394740 4167 raft_consensus.cc:2229] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.425992 4009 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.426434 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.426730 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.427687 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.428725 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.430874 4000 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.431442 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.434604 4167 raft_consensus.cc:2258] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.434924 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.434945 4013 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.435568 4167 ts_tablet_manager.cc:1905] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.435629 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.438763 3996 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.439327 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.443257 4361 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.443607 4361 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.444576 4361 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.450387 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.450911 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.450963 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.451805 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.452594 4162 ts_tablet_manager.cc:1428] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.047s user 0.000s sys 0.003s
I20260811 18:57:06.454839 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.456256 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.457139 4271 raft_consensus.cc:482] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.457401 4271 raft_consensus.cc:504] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.463500 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.464177 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.467823 4162 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.471678 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.472637 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.480831 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.480803 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.481895 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.482993 3998 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.475755 4271 leader_election.cc:288] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.483530 3866 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.484249 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.486671 4001 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.487438 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.490988 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.491231 3856 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.491971 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.494642 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.496052 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.499490 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.499936 4003 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.500304 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.507737 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.508548 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.522727 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.531013 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.531883 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.543491 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.544292 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:06.557112 3965 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:06.559140 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.559700 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.559918 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.559243 3872 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.562032 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.566607 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.572533 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.584656 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.585405 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.588124 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.591889 3713 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.598847 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.607113 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.607641 3824 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:06.608599 3967 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:06.609051 3967 leader_election.cc:302] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.609617 4271 raft_consensus.cc:2726] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.609884 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.623584 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.626215 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.631455 4383 raft_consensus.cc:2726] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.611127 4163 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.632378 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.641072 4413 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:06.669242 4163 tablet_bootstrap.cc:654] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.707798 4162 tablet_bootstrap.cc:654] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.707314 3547 catalog_manager.cc:5495] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:06.711027 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c70d9db064e45af91fd9901d85e211f consensus state has no leader
I20260811 18:57:06.711776 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a671c5d0d75c49779b94baa2b89c9cef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.712745 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a671c5d0d75c49779b94baa2b89c9cef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.712953 4169 ts_tablet_manager.cc:1918] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.713210 4169 log.cc:1196] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/494c09652d9e408880ca606ae0b30773
I20260811 18:57:06.713950 4169 ts_tablet_manager.cc:1939] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.724344 4163 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.724937 4163 ts_tablet_manager.cc:1397] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.336s user 0.004s sys 0.004s
I20260811 18:57:06.730507 4163 raft_consensus.cc:348] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.731127 4163 raft_consensus.cc:374] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.731370 4163 raft_consensus.cc:729] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.735342 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 143) took 1178 ms. Trace:
I20260811 18:57:06.735432 4162 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.735798 4162 ts_tablet_manager.cc:1397] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.283s user 0.005s sys 0.002s
I20260811 18:57:06.735648 4169 rpcz_store.cc:276] 0811 18:57:05.557259 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.557727 (+ 468us) service_pool.cc:227] Handling call
0811 18:57:06.328909 (+771182us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:06.328922 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.672392 (+343470us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.711802 (+ 39410us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.735160 (+ 23358us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":768021}
I20260811 18:57:06.737345 4162 raft_consensus.cc:348] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.737723 4162 raft_consensus.cc:374] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.737665 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 494c09652d9e408880ca606ae0b30773 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:06.737936 4162 raft_consensus.cc:729] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.739239 4163 consensus_queue.cc:260] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.739696 3676 consensus_peers.cc:487] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:06.740806 4167 ts_tablet_manager.cc:1918] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.743165 4167 log.cc:1196] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:06.747335 4167 ts_tablet_manager.cc:1939] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.750172 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 125) took 1533 ms. Trace:
I20260811 18:57:06.750511 4169 tablet_replica.cc:330] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:06.751171 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 consensus state has no leader
I20260811 18:57:06.752132 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4db75b493caf48c0be36b2f872c70dd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.753633 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4db75b493caf48c0be36b2f872c70dd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.754913 4163 ts_tablet_manager.cc:1428] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.030s user 0.004s sys 0.000s
I20260811 18:57:06.755055 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet e215759d48c24d8cb99bbc3fc9009a66 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.755604 4163 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.755450 4167 rpcz_store.cc:276] 0811 18:57:05.216437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.218439 (+ 2002us) service_pool.cc:227] Handling call
0811 18:57:06.383803 (+1165364us) ts_tablet_manager.cc:1145] Deleting tablet 82d1339366a24a87b74bd1d411a3f6fe
0811 18:57:06.383825 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.679746 (+295921us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.740655 (+ 60909us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.750106 (+ 9451us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8445,"tablet-delete.queue_time_us":1150872}
I20260811 18:57:06.756358 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 82d1339366a24a87b74bd1d411a3f6fe (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:06.748232 4162 consensus_queue.cc:260] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.757318 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.758159 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 2852f9e5f67c4025836018af553388a5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.759104 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1a8b47adc7454c64a44013e919b3a485"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.759930 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a8b47adc7454c64a44013e919b3a485 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.761261 4162 ts_tablet_manager.cc:1428] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.025s user 0.002s sys 0.002s
I20260811 18:57:06.761572 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:06.762521 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9bb5acd56644427f9aec05c1ab8e46ee" } modify_external_catalogs: true
I20260811 18:57:06.762768 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9bb5acd56644427f9aec05c1ab8e46ee" } modify_external_catalogs: true
W20260811 18:57:06.764088 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e5fd9fce095e4a769dbb38c7903359cc consensus state has no leader
I20260811 18:57:06.764782 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fa5c52d66624877a73a99af0be2381a" } modify_external_catalogs: true
I20260811 18:57:06.765188 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fa5c52d66624877a73a99af0be2381a" } modify_external_catalogs: true
I20260811 18:57:06.765875 4162 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.768266 4167 tablet_replica.cc:330] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:06.769145 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:06.769198 4167 raft_consensus.cc:2229] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.769591 4167 raft_consensus.cc:2258] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.773092 4169 raft_consensus.cc:2229] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:06.773603 4169 pending_rounds.cc:62] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:06.773838 4169 pending_rounds.cc:68] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400069391585280 op_type: NO_OP noop_request { }
I20260811 18:57:06.774263 4169 raft_consensus.cc:2866] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.774693 4169 raft_consensus.cc:2258] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.775878 4169 ts_tablet_manager.cc:1905] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:06.780644 3678 consensus_peers.cc:487] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:06.782755 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.785867 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.786418 4167 ts_tablet_manager.cc:1905] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:06.789294 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
I20260811 18:57:06.791962 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.793340 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:06.794744 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.795701 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac4a8df2cd7346ffa89549c0750760ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.798846 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.799865 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:06.800100 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
I20260811 18:57:06.800827 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.801237 4163 tablet_bootstrap.cc:654] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.808614 4162 tablet_bootstrap.cc:654] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.819885 4169 ts_tablet_manager.cc:1918] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:06.820827 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 439ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:06.822432 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b4ff060dff9a4bf19670e304147315ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.823446 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4ff060dff9a4bf19670e304147315ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.831694 4169 log.cc:1196] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:06.832481 4169 ts_tablet_manager.cc:1939] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.833504 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 144) took 1272 ms. Trace:
I20260811 18:57:06.833752 4169 rpcz_store.cc:276] 0811 18:57:05.561054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.573245 (+ 12191us) service_pool.cc:227] Handling call
0811 18:57:06.741869 (+1168624us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:06.741887 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.803687 (+ 61800us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.819620 (+ 15933us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.833457 (+ 13837us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21904,"tablet-delete.queue_time_us":1167523}
I20260811 18:57:06.834544 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 08b8dcf7a56043efbbd40bb888616492 (table default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]) successfully deleted
I20260811 18:57:06.835953 4163 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.836228 4163 ts_tablet_manager.cc:1397] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.081s user 0.000s sys 0.007s
I20260811 18:57:06.838407 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:06.837956 4163 raft_consensus.cc:348] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.838660 4163 raft_consensus.cc:374] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.838918 4163 raft_consensus.cc:729] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.839217 3550 catalog_manager.cc:6565] ReplaceTablet: tablet e215759d48c24d8cb99bbc3fc9009a66 deleted and replaced by tablet c08436f33db043b1bc6395cd92102e59
I20260811 18:57:06.839533 4162 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.839874 4162 ts_tablet_manager.cc:1397] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.002s
I20260811 18:57:06.839833 4383 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.839846 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f53cc0009ca74325a65d63adf0744d37" } modify_external_catalogs: true
I20260811 18:57:06.840082 4383 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.840320 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f53cc0009ca74325a65d63adf0744d37" } modify_external_catalogs: true
I20260811 18:57:06.841240 4383 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.841311 4162 raft_consensus.cc:348] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.841806 4162 raft_consensus.cc:374] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.842018 4162 raft_consensus.cc:729] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.839331 4163 consensus_queue.cc:260] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.842526 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.843413 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.843984 3744 raft_consensus.cc:2454] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:06.844880 3824 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.845602 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.845713 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.846925 4383 raft_consensus.cc:2781] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.847114 4383 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.847378 4383 raft_consensus.cc:3037] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.852396 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:06.852981 3824 leader_election.cc:341] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.853869 4383 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.855552 4383 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:06.856303 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
I20260811 18:57:06.856421 4163 ts_tablet_manager.cc:1428] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.002s sys 0.003s
I20260811 18:57:06.856999 4163 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.860281 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.861760 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
W20260811 18:57:06.861850 3824 leader_election.cc:341] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.862187 3744 raft_consensus.cc:3037] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.862999 4162 consensus_queue.cc:260] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.868181 4169 tablet_replica.cc:330] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:06.868636 4169 raft_consensus.cc:2229] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.869032 4169 raft_consensus.cc:2258] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.858072 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7eb5b8b42dae46398c5bf3afe86e1b09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.870915 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eb5b8b42dae46398c5bf3afe86e1b09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.875859 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db39f5cc5ace43a49931b2da095c6037" } new_table_name: "default.Table_e4b52f563c5f4629a03019c85bfbb653" modify_external_catalogs: true
I20260811 18:57:06.877458 4169 ts_tablet_manager.cc:1905] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.879767 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet dde27d8cc3eb4d09879e717fc90305fd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.885160 4163 tablet_bootstrap.cc:654] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.886307 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 5ae200f3814e431a8288c30fe94bffcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.893229 4162 ts_tablet_manager.cc:1428] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.053s user 0.002s sys 0.002s
I20260811 18:57:06.907261 4167 ts_tablet_manager.cc:1918] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.907609 4167 log.cc:1196] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8964f92255884b3aae0cac60f019fdb7
I20260811 18:57:06.908576 4167 ts_tablet_manager.cc:1939] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.909086 3744 raft_consensus.cc:2454] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:06.909945 3824 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.910749 4383 raft_consensus.cc:2781] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.910980 4383 raft_consensus.cc:686] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:06.911339 4162 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.911460 4383 consensus_queue.cc:237] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.915844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 133) took 1447 ms. Trace:
I20260811 18:57:06.916106 4167 rpcz_store.cc:276] 0811 18:57:05.494519 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.500270 (+ 5751us) service_pool.cc:227] Handling call
0811 18:57:06.767949 (+1267679us) ts_tablet_manager.cc:1145] Deleting tablet 8964f92255884b3aae0cac60f019fdb7
0811 18:57:06.767968 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.830599 (+ 62631us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.859093 (+ 28494us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.915738 (+ 56645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1265992}
I20260811 18:57:06.917068 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f501d17d05204712b318f4d95afce6dc
I20260811 18:57:06.916793 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8964f92255884b3aae0cac60f019fdb7 (table default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]) successfully deleted
I20260811 18:57:06.919914 4162 tablet_bootstrap.cc:654] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.920054 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_44def5389adf4d9ca825712dfb862206"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.921095 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44def5389adf4d9ca825712dfb862206 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.921715 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.922586 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1329ea40dbd4f748a0e35686a5e867f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.925225 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 134) took 1355 ms. Trace:
I20260811 18:57:06.925573 4167 rpcz_store.cc:276] 0811 18:57:05.570165 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.594406 (+ 24241us) service_pool.cc:227] Handling call
0811 18:57:06.924850 (+1330444us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:06.924868 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.925095 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1330118}
W20260811 18:57:06.926128 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
I20260811 18:57:06.926538 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 140) took 1269 ms. Trace:
I20260811 18:57:06.926769 4167 rpcz_store.cc:276] 0811 18:57:05.656359 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656890 (+ 531us) service_pool.cc:227] Handling call
0811 18:57:06.926081 (+1269191us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:06.926097 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.926255 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268893}
I20260811 18:57:06.927412 4167 tablet_replica.cc:330] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:06.927665 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
I20260811 18:57:06.927739 4169 ts_tablet_manager.cc:1918] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.927801 4167 raft_consensus.cc:2229] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.928145 4167 pending_rounds.cc:62] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 2 pending ops.
I20260811 18:57:06.928373 4167 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400066380767232 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" new_extra_config { } }
I20260811 18:57:06.929236 4167 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062060187648 op_type: NO_OP noop_request { }
I20260811 18:57:06.929486 4167 raft_consensus.cc:2866] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.929735 4167 raft_consensus.cc:2258] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.942600 4167 ts_tablet_manager.cc:1905] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.944295 4169 log.cc:1196] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/927fb981490b4cecabe40314d6f71048
I20260811 18:57:06.945163 4169 ts_tablet_manager.cc:1939] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.945257 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_abaf87a40e04427fba978e8a67b89f0c" } new_table_name: "default.Table_9e2e268d15894407ab276766ff611c86" modify_external_catalogs: true
I20260811 18:57:06.946604 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 146) took 1370 ms. Trace:
I20260811 18:57:06.946820 4169 rpcz_store.cc:276] 0811 18:57:05.575579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.575653 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:06.867993 (+1292340us) ts_tablet_manager.cc:1145] Deleting tablet 927fb981490b4cecabe40314d6f71048
0811 18:57:06.868012 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.918518 (+ 50506us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.927566 (+ 9048us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.946522 (+ 18956us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1289286}
I20260811 18:57:06.947696 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 927fb981490b4cecabe40314d6f71048 (table default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]) successfully deleted
W20260811 18:57:06.948688 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2247194db704f9abd54a8a0df1ca566 consensus state has no leader
I20260811 18:57:06.949721 4383 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.949992 4383 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.951071 4383 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.952890 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:06.953635 3824 leader_election.cc:341] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.957022 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9c18120da2d245f495c52a670047760d (table default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137]). Expected version 1 got 0
I20260811 18:57:06.957866 4163 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.957824 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137] alter complete (version 1)
I20260811 18:57:06.958124 4163 ts_tablet_manager.cc:1397] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.008s
I20260811 18:57:06.961760 4163 raft_consensus.cc:348] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.962324 4163 raft_consensus.cc:374] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.963348 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 145) took 1387 ms. Trace:
I20260811 18:57:06.963513 4169 rpcz_store.cc:276] 0811 18:57:05.575258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.575697 (+ 439us) service_pool.cc:227] Handling call
0811 18:57:06.953976 (+1378279us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:06.954018 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.962509 (+ 8491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1374978}
I20260811 18:57:06.963690 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
W20260811 18:57:06.964270 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
I20260811 18:57:06.964713 4163 raft_consensus.cc:729] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.965525 4163 consensus_queue.cc:260] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.967276 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.967680 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.969300 4163 ts_tablet_manager.cc:1428] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.002s
I20260811 18:57:06.969878 4383 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.969887 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.970130 4383 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.971338 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.971280 4383 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.973009 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.973453 4163 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:06.974073 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.975153 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:06.975934 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.976274 3824 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.976821 4383 raft_consensus.cc:2726] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.979139 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7eb5b8b42dae46398c5bf3afe86e1b09" } new_table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" modify_external_catalogs: true
I20260811 18:57:06.980630 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 147) took 1236 ms. Trace:
I20260811 18:57:06.980839 4169 rpcz_store.cc:276] 0811 18:57:05.742830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.742926 (+ 96us) service_pool.cc:227] Handling call
0811 18:57:06.965215 (+1222289us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:06.965235 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.979524 (+ 14289us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1210267}
I20260811 18:57:06.981176 4331 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.981499 4331 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.981715 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:06.982761 4331 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.984125 4163 tablet_bootstrap.cc:654] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.984102 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.984710 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:06.985260 3678 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.985397 3890 raft_consensus.cc:2454] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.983662 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bcaa2e21fe064cbea9a28f034c63eae6" } new_table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" modify_external_catalogs: true
I20260811 18:57:06.986171 3676 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.986553 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 150) took 1230 ms. Trace:
I20260811 18:57:06.987087 4331 raft_consensus.cc:2781] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.987293 4331 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.987614 4331 raft_consensus.cc:3037] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.991866 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.992666 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.992815 4331 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.994577 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.994952 3890 raft_consensus.cc:3037] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.995550 4331 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.996605 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:06.997588 3678 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.998912 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.999979 3744 raft_consensus.cc:2454] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:07.000787 3824 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:07.006521 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:07.007215 4383 raft_consensus.cc:2781] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:07.007515 4383 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:07.007843 4383 raft_consensus.cc:3037] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:07.009267 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0c4cb29b3ec48749d83adaa68477c84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.013247 4001 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.014168 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.015259 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.015705 4383 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.016098 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.016835 4383 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:07.017166 3890 raft_consensus.cc:2454] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:07.018141 3676 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:07.026662 4331 raft_consensus.cc:2781] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:07.026574 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:07.028757 3824 leader_election.cc:341] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:07.035142 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:07.035568 3744 raft_consensus.cc:3037] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:07.036183 3866 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.036906 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.036983 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.037549 4010 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.037822 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.038203 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.050654 4383 raft_consensus.cc:482] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:07.051119 4383 raft_consensus.cc:504] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.053963 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2852f9e5f67c4025836018af553388a5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:07.054858 3824 leader_election.cc:341] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:07.064560 3859 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.065440 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.065407 3712 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.066228 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.070412 4383 leader_election.cc:288] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:07.074100 3855 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.074908 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.986917 4169 rpcz_store.cc:276] 0811 18:57:05.755930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.755997 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:06.986275 (+1230278us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:06.986292 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.986486 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1229902}
I20260811 18:57:07.075860 4169 tablet_replica.cc:330] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:07.076179 4169 raft_consensus.cc:2229] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:07.080042 4167 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:07.089579 4169 raft_consensus.cc:2258] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:07.090569 4169 ts_tablet_manager.cc:1905] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:07.080467 4167 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:07.099387 4167 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:07.100695 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 137) took 1445 ms. Trace:
I20260811 18:57:07.100862 4167 rpcz_store.cc:276] 0811 18:57:05.655533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656133 (+ 600us) service_pool.cc:227] Handling call
0811 18:57:06.927244 (+1271111us) ts_tablet_manager.cc:1145] Deleting tablet 6ef0a376939d4d57b0d95db644f25dd9
0811 18:57:06.927256 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.976403 (+ 49147us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.074948 (+ 98545us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.100626 (+ 25678us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1269804}
I20260811 18:57:07.101555 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:07.101805 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 139) took 1445 ms. Trace:
I20260811 18:57:07.102034 4167 rpcz_store.cc:276] 0811 18:57:05.655926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656194 (+ 268us) service_pool.cc:227] Handling call
0811 18:57:07.101456 (+1445262us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:07.101487 (+ 31us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.101674 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1443618}
I20260811 18:57:07.140240 4163 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:07.312342 4471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:38889 every 8 connection(s)
I20260811 18:57:07.140578 4163 ts_tablet_manager.cc:1397] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.171s user 0.010s sys 0.012s
I20260811 18:57:08.303539 4163 raft_consensus.cc:348] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.303872 4163 raft_consensus.cc:374] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.303993 4163 raft_consensus.cc:729] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.304265 4163 consensus_queue.cc:260] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.956779 4485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:08.304965 4163 ts_tablet_manager.cc:1428] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:08.305365 4163 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:08.305069 4485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:07.313058 4230 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:38889
I20260811 18:57:08.307153 4485 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
W20260811 18:57:07.103190 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
W20260811 18:57:08.307569 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 1.20463e+06us
I20260811 18:57:08.308525 4230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:57:07.167467 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.167640 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.248467 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.249888 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.250545 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.283938 4169 ts_tablet_manager.cc:1918] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:08.309765 4169 log.cc:1196] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:08.309899 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb (DEFAULT_TABLE table=default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.310328 4169 ts_tablet_manager.cc:1939] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:07.097087 3744 raft_consensus.cc:2454] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:08.310465 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515d5b5afbd74ec5ab90a503aef2f4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.310560 3744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:51189 (request call id 24) took 1287 ms. Trace:
I20260811 18:57:08.310705 3744 rpcz_store.cc:276] 0811 18:57:07.023209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.033128 (+ 9919us) service_pool.cc:227] Handling call
0811 18:57:08.310511 (+1277383us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:07.107930 3998 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.311267 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 153) took 2553 ms. Trace:
I20260811 18:57:08.311393 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.311395 4169 rpcz_store.cc:276] 0811 18:57:05.757745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.757801 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:07.075670 (+1317869us) ts_tablet_manager.cc:1145] Deleting tablet 6d1d58835ad24749a00ce27b4bc73c59
0811 18:57:07.075685 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.179233 (+103548us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.283770 (+104537us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.311210 (+1027440us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1317387}
I20260811 18:57:08.311821 4169 tablet_replica.cc:330] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:08.312068 4169 raft_consensus.cc:2229] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:08.312249 4169 pending_rounds.cc:62] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 2 pending ops.
I20260811 18:57:08.312335 4169 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400066380767232 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" new_extra_config { } }
I20260811 18:57:08.312858 4169 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062060187648 op_type: NO_OP noop_request { }
I20260811 18:57:08.313041 4169 raft_consensus.cc:2866] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:08.313182 4169 raft_consensus.cc:2258] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:08.313868 4169 ts_tablet_manager.cc:1905] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:07.308692 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (DEFAULT_TABLE table=default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.314710 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.120746 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9409f5b4b0034412aedc183b302556a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:08.317207 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 4230
W20260811 18:57:08.317308 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9409f5b4b0034412aedc183b302556a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.097311 4331 raft_consensus.cc:686] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:07.310384 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 (DEFAULT_TABLE table=default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.311396 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4687b95728fc485fa0e842bc447eb83f (DEFAULT_TABLE table=default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.318305 3721 tablet_service.cc:1467] Processing CreateTablet for tablet b4c7f500a5444bae9a015a5e903c0622 (DEFAULT_TABLE table=default.Table_a594cb48b586458783a36cc76df1332c [id=7c0d12bf15b945c68177bfb057015870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.312368 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b4c7f500a5444bae9a015a5e903c0622 (DEFAULT_TABLE table=default.Table_a594cb48b586458783a36cc76df1332c [id=7c0d12bf15b945c68177bfb057015870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.318768 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6ec419ed7747c3b24dc11992518e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.318827 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c7f500a5444bae9a015a5e903c0622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.318662 4331 consensus_queue.cc:237] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.319042 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c7f500a5444bae9a015a5e903c0622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.319389 3717 tablet_service.cc:1467] Processing CreateTablet for tablet c08436f33db043b1bc6395cd92102e59 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.108460 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2852f9e5f67c4025836018af553388a5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:08.319792 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08436f33db043b1bc6395cd92102e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.320065 4034 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:47987 (request call id 25) took 1213 ms. Trace:
I20260811 18:57:08.320333 4034 rpcz_store.cc:276] 0811 18:57:07.106986 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.108224 (+ 1238us) service_pool.cc:227] Handling call
0811 18:57:08.319978 (+1211754us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.321075 3712 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.321524 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.313647 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9efd2f19d74a68a8536fd057377738 (DEFAULT_TABLE table=default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 [id=64009634513f46eeae6442cfadb7b0f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.322710 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9efd2f19d74a68a8536fd057377738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.322728 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (DEFAULT_TABLE table=default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.322808 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9efd2f19d74a68a8536fd057377738 (DEFAULT_TABLE table=default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 [id=64009634513f46eeae6442cfadb7b0f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.323134 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.323493 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 867259e37fe44c33a35823ec2d896754 (DEFAULT_TABLE table=default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.323930 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867259e37fe44c33a35823ec2d896754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.314942 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 867259e37fe44c33a35823ec2d896754 (DEFAULT_TABLE table=default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.326895 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4687b95728fc485fa0e842bc447eb83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.326915 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 (DEFAULT_TABLE table=default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.327203 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 4687b95728fc485fa0e842bc447eb83f (DEFAULT_TABLE table=default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.327369 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02808605ee6c4634af47fc3fd57e9cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.327615 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4687b95728fc485fa0e842bc447eb83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.327608 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867259e37fe44c33a35823ec2d896754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.328104 3723 tablet_service.cc:1467] Processing CreateTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.328660 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b199f4c9e14b468cb26cbd61fc8586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.161681 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:08.330719 3714 tablet_service.cc:1467] Processing CreateTablet for tablet a902ad70a25546be926ff8f7bf2ed64f (DEFAULT_TABLE table=default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.330926 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0190cbd859274fcfa09c6fab899c4843 deleted and replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:08.331113 3546 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 261) took 1865 ms. Trace:
I20260811 18:57:08.331205 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902ad70a25546be926ff8f7bf2ed64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.331261 3546 rpcz_store.cc:276] 0811 18:57:06.465290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.757148 (+291858us) service_pool.cc:227] Handling call
0811 18:57:08.331047 (+1573899us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.057800 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.057899 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.057905 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:07.057907 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:07.058094 (+ 187us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.058105 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.058110 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.058181 (+ 71us) tablet.cc:699] Row locks acquired
0811 18:57:07.058183 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.058214 (+ 31us) write_op.cc:270] Start()
0811 18:57:07.058242 (+ 28us) write_op.cc:276] Timestamp: P: 1786474627058172 usec, L: 0
0811 18:57:07.058245 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.058465 (+ 220us) log.cc:842] Serialized 622 byte log entry
0811 18:57:07.158160 (+ 99695us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.159887 (+ 1727us) write_op.cc:301] APPLY: starting
0811 18:57:07.159897 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.159931 (+ 34us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.159954 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.160059 (+ 105us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.160132 (+ 73us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.160139 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:07.160191 (+ 52us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.160228 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160242 (+ 14us) write_op.cc:454] Released schema lock
0811 18:57:07.160247 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.160291 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160295 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1696,"apply.run_cpu_time_us":448,"apply.run_wall_time_us":448,"num_ops":2,"prepare.queue_time_us":101718,"prepare.run_cpu_time_us":866,"prepare.run_wall_time_us":65551,"raft.queue_time_us":185,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":9,"replication_time_us":99899}]]}
I20260811 18:57:08.334065 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb0316f2e1db4353b9919edf3d8e1c90" } modify_external_catalogs: true
I20260811 18:57:08.334172 3856 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 191) took 1026 ms. Trace:
I20260811 18:57:07.107638 4162 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:08.334311 3856 rpcz_store.cc:276] 0811 18:57:07.307913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.308031 (+ 118us) service_pool.cc:227] Handling call
0811 18:57:08.314483 (+1006452us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.314512 (+ 29us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.317893 (+ 3381us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.333905 (+ 16012us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.334128 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.334537 4162 ts_tablet_manager.cc:1397] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 1.440s user 0.016s sys 0.005s
I20260811 18:57:08.334757 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f335e50ae57547ed80b144129faaffbb (DEFAULT_TABLE table=default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.335215 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f335e50ae57547ed80b144129faaffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.335461 4162 raft_consensus.cc:348] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.335708 4162 raft_consensus.cc:374] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.335801 4162 raft_consensus.cc:729] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.336017 4162 consensus_queue.cc:260] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:07.378022 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 734ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:07.316912 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 (DEFAULT_TABLE table=default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.337309 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02808605ee6c4634af47fc3fd57e9cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.338110 3860 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 195) took 1024 ms. Trace:
I20260811 18:57:07.162387 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:07.162844 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:08.338718 3711 tablet_service.cc:1467] Processing CreateTablet for tablet cbf6ce167fb348be8d61372791f0cbca (DEFAULT_TABLE table=default.Table_a671c5d0d75c49779b94baa2b89c9cef [id=dda7fb95dc1549ddbc88d8525ff023e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.338840 3551 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 265) took 1858 ms. Trace:
I20260811 18:57:08.339219 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbf6ce167fb348be8d61372791f0cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.338975 3551 rpcz_store.cc:276] 0811 18:57:06.479865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.764486 (+284621us) service_pool.cc:227] Handling call
0811 18:57:06.765358 (+ 872us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.765528 (+ 170us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:06.765598 (+ 70us) catalog_manager.cc:2761] Locking tablets
0811 18:57:06.960011 (+194413us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:07.162575 (+202564us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:07.162649 (+ 74us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:07.162708 (+ 59us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:08.338769 (+1176061us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.136544 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.136752 (+ 208us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.136761 (+ 9us) write_op.cc:435] Acquired schema lock
0811 18:57:07.136766 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.136988 (+ 222us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.137005 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.137014 (+ 9us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.137158 (+ 144us) tablet.cc:699] Row locks acquired
0811 18:57:07.137161 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:07.137214 (+ 53us) write_op.cc:270] Start()
0811 18:57:07.137262 (+ 48us) write_op.cc:276] Timestamp: P: 1786474627137159 usec, L: 0
0811 18:57:07.137268 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.137591 (+ 323us) log.cc:842] Serialized 756 byte log entry
0811 18:57:07.158710 (+ 21119us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.160917 (+ 2207us) write_op.cc:301] APPLY: starting
0811 18:57:07.160932 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.160965 (+ 33us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.160989 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.161054 (+ 65us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.161092 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:07.161098 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.161153 (+ 55us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.161190 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.161208 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:07.161213 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.161260 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.161264 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2148,"apply.run_cpu_time_us":400,"apply.run_wall_time_us":401,"num_ops":2,"prepare.queue_time_us":175744,"prepare.run_cpu_time_us":1321,"prepare.run_wall_time_us":33947,"raft.queue_time_us":211,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":210,"replication_time_us":21427}]]}
I20260811 18:57:08.339929 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.340549 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e85b84e5b0e45bf80321dfb55c1704d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.165973 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dcfccd3951dc4875a1d70530eb514fb2" } new_table_name: "default.Table_53cd8000542843c984903cd25e451bdf" modify_external_catalogs: true
I20260811 18:57:07.138746 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 138) took 1478 ms. Trace:
W20260811 18:57:07.234654 3678 consensus_peers.cc:487] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:07.320669 3865 tablet_service.cc:1467] Processing CreateTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.342517 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 1.10801e+06us
I20260811 18:57:07.332844 3858 tablet_service.cc:1467] Processing CreateTablet for tablet a902ad70a25546be926ff8f7bf2ed64f (DEFAULT_TABLE table=default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.342775 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 65234acc3d5a462683edbe8f223eeefc (DEFAULT_TABLE table=default.Table_4db75b493caf48c0be36b2f872c70dd8 [id=c7f1c01155c044af8ffdf358de715e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.345032 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f335e50ae57547ed80b144129faaffbb (DEFAULT_TABLE table=default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.436466 4446 consensus_queue.cc:1035] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:07.168037 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ace00158b744bd8a97dc1c47fc6049c consensus state has no leader
I20260811 18:57:08.343235 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65234acc3d5a462683edbe8f223eeefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.343348 3548 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 275) took 1425 ms. Trace:
I20260811 18:57:07.347676 3861 tablet_service.cc:1467] Processing CreateTablet for tablet cbf6ce167fb348be8d61372791f0cbca (DEFAULT_TABLE table=default.Table_a671c5d0d75c49779b94baa2b89c9cef [id=dda7fb95dc1549ddbc88d8525ff023e8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:07.550316 3664 kernel_stack_watchdog.cc:198] Thread 3678 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 153ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fcdd5c86980 (unknown) at ??:0
@ 0x7fcdd5c809c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fcdcfd7769c google::LogMessage::Flush() at ??:0
@ 0x7fcdcfd776d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fcddddbbb37 kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7fcddddbac46 kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7fcddddc0ff1 kudu::consensus::Peer::SendNextRequest()::$_3::operator()() at ??:0
@ 0x7fcddddc0dfe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9426f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fcdd9426b6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9412dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fcdd9413321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fcdd93b0aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fcdd93af5a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:08.343506 3548 rpcz_store.cc:276] 0811 18:57:06.917826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.944831 (+ 27005us) service_pool.cc:227] Handling call
0811 18:57:06.945521 (+ 690us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.945869 (+ 348us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.946137 (+ 268us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.946245 (+ 108us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.162974 (+216729us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.343303 (+1180329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.056563 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.056703 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.056710 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:07.056713 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:07.056815 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.056827 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.056831 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.056892 (+ 61us) tablet.cc:699] Row locks acquired
0811 18:57:07.056894 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.056997 (+ 103us) write_op.cc:270] Start()
0811 18:57:07.057046 (+ 49us) write_op.cc:276] Timestamp: P: 1786474627056952 usec, L: 0
0811 18:57:07.057050 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.057251 (+ 201us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.158069 (+100818us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.159054 (+ 985us) write_op.cc:301] APPLY: starting
0811 18:57:07.159077 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.159159 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.159213 (+ 54us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.159280 (+ 67us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.159335 (+ 55us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.159351 (+ 16us) write_op.cc:312] APPLY: finished
0811 18:57:07.159533 (+ 182us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.159648 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.159684 (+ 36us) write_op.cc:454] Released schema lock
0811 18:57:07.159701 (+ 17us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.159798 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.159802 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":911,"apply.run_cpu_time_us":811,"apply.run_wall_time_us":811,"num_ops":1,"prepare.queue_time_us":109584,"prepare.run_cpu_time_us":795,"prepare.run_wall_time_us":1222,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":100982}]]}
I20260811 18:57:07.360954 3854 tablet_service.cc:1467] Processing CreateTablet for tablet c08436f33db043b1bc6395cd92102e59 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.344475 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet d106e0e1f4b845429b10f4e100f69a8b from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:07.647948 4476 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:08.344962 3548 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 284) took 1310 ms. Trace:
I20260811 18:57:08.344805 4476 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.345063 3548 rpcz_store.cc:276] 0811 18:57:07.034425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.344406 (+1309981us) service_pool.cc:227] Handling call
0811 18:57:08.344903 (+ 497us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.345502 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_60c6f6ccfa2747e1b4fff22a5779d63e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:08.345674 4476 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:07.362970 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 65234acc3d5a462683edbe8f223eeefc (DEFAULT_TABLE table=default.Table_4db75b493caf48c0be36b2f872c70dd8 [id=c7f1c01155c044af8ffdf358de715e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.439107 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 490f34d7a403416fbab64e46973746de (DEFAULT_TABLE table=default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.346163 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60c6f6ccfa2747e1b4fff22a5779d63e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.441349 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.186271 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dde27d8cc3eb4d09879e717fc90305fd
W20260811 18:57:07.730131 3824 consensus_peers.cc:487] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:08.326920 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 (DEFAULT_TABLE table=default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.346803 3553 catalog_manager.cc:6565] ReplaceTablet: tablet dde27d8cc3eb4d09879e717fc90305fd deleted and replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5
I20260811 18:57:08.346940 3553 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 271) took 1697 ms. Trace:
I20260811 18:57:08.347044 3553 rpcz_store.cc:276] 0811 18:57:06.649596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.879512 (+229916us) service_pool.cc:227] Handling call
0811 18:57:08.346886 (+1467374us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.170540 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.170979 (+ 439us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.170991 (+ 12us) write_op.cc:435] Acquired schema lock
0811 18:57:07.170996 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.171211 (+ 215us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.171236 (+ 25us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.171244 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.171379 (+ 135us) tablet.cc:699] Row locks acquired
0811 18:57:07.171383 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:07.171441 (+ 58us) write_op.cc:270] Start()
0811 18:57:07.171498 (+ 57us) write_op.cc:276] Timestamp: P: 1786474627171382 usec, L: 0
0811 18:57:07.171505 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.171834 (+ 329us) log.cc:842] Serialized 622 byte log entry
0811 18:57:07.184823 (+ 12989us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.185006 (+ 183us) write_op.cc:301] APPLY: starting
0811 18:57:07.185023 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.185070 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.185113 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.185230 (+ 117us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.185304 (+ 74us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.185310 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.185459 (+ 149us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.185565 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.185599 (+ 34us) write_op.cc:454] Released schema lock
0811 18:57:07.185611 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.185702 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.185708 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":133,"apply.run_cpu_time_us":777,"apply.run_wall_time_us":775,"num_ops":2,"prepare.queue_time_us":203155,"prepare.run_cpu_time_us":1576,"prepare.run_wall_time_us":20180,"raft.queue_time_us":265,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":435,"replication_time_us":13288}]]}
W20260811 18:57:07.236001 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61c8443df21949ff9bdc63fa6aa34108 consensus state has no leader
I20260811 18:57:07.251451 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_542b2f198c2a42abb286bbdcf948584b" } modify_external_catalogs: true
I20260811 18:57:08.350589 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet ed3346bd12d04c45a511308c427856e5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:08.350626 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_542b2f198c2a42abb286bbdcf948584b" } modify_external_catalogs: true
W20260811 18:57:07.571264 3528 kernel_stack_watchdog.cc:198] Thread 3541 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 158ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f61e8213980 (unknown) at ??:0
@ 0x7f61e118d135 __write at ??:0
@ 0x7f61e110815d _IO_file_write at ??:0
@ 0x7f61e1108acf _IO_file_xsputn at ??:0
@ 0x7f61e10fc927 fwrite at ??:0
@ 0x80f130 __interceptor_fwrite.part.57 at ??:?
@ 0x7f61e2307661 google::LogMessage::SendToLog() at ??:0
@ 0x7f61e23044de google::LogMessage::Flush() at ??:0
@ 0x7f61e23046d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f61f7e30bab kudu::master::AsyncDeleteReplica::HandleResponse() at ??:0
@ 0x7f61f7d899a7 kudu::master::RetryingTSRpcTask::RpcCallback() at ??:0
@ 0x7f61f7e33540 _ZZN4kudu6master18AsyncDeleteReplica11SendRequestEiENKUlvE_clEv at ??:0
@ 0x7f61f7e3335e _ZNSt17_Function_handlerIFvvEZN4kudu6master18AsyncDeleteReplica11SendRequestEiEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61eb9b3f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f61eb9b3b6e std::_Function_handler<>::_M_invoke() at ??:0
W20260811 18:57:07.827332 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7f640c60e81b4a369458b1497a269c9c consensus state has no leader
W20260811 18:57:07.209591 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f335e50ae57547ed80b144129faaffbb has no consensus state
I20260811 18:57:08.352555 3547 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 276) took 1373 ms. Trace:
I20260811 18:57:08.352669 3547 rpcz_store.cc:276] 0811 18:57:06.978633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.978749 (+ 116us) service_pool.cc:227] Handling call
0811 18:57:06.979464 (+ 715us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.980773 (+ 1309us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.980995 (+ 222us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.981114 (+ 119us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.209119 (+228005us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.352520 (+1143401us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.190743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.190866 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.190872 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:07.190874 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:07.190982 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.190996 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.191001 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.191092 (+ 91us) tablet.cc:699] Row locks acquired
0811 18:57:07.191094 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.191126 (+ 32us) write_op.cc:270] Start()
0811 18:57:07.191159 (+ 33us) write_op.cc:276] Timestamp: P: 1786474627191075 usec, L: 0
0811 18:57:07.191162 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.191361 (+ 199us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.208088 (+ 16727us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.208315 (+ 227us) write_op.cc:301] APPLY: starting
0811 18:57:07.208340 (+ 25us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.208408 (+ 68us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.208492 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.208550 (+ 58us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.208592 (+ 42us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.208599 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:07.208696 (+ 97us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.208763 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.208786 (+ 23us) write_op.cc:454] Released schema lock
0811 18:57:07.208794 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.208852 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.208856 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":145,"apply.run_cpu_time_us":611,"apply.run_wall_time_us":610,"num_ops":1,"prepare.queue_time_us":208810,"prepare.run_cpu_time_us":877,"prepare.run_wall_time_us":15273,"raft.queue_time_us":254,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":454,"replication_time_us":16885}]]}
I20260811 18:57:08.353579 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0cb87feb6db445d792ba8021cbe75510"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.354187 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cb87feb6db445d792ba8021cbe75510 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.354415 3555 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 277) took 1367 ms. Trace:
I20260811 18:57:08.354499 3555 rpcz_store.cc:276] 0811 18:57:06.982626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.983161 (+ 535us) service_pool.cc:227] Handling call
0811 18:57:06.986104 (+ 2943us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.986560 (+ 456us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.986817 (+ 257us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.986903 (+ 86us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.235670 (+248767us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.350522 (+1114852us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.206108 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.206280 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.206290 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:07.206295 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.216559 (+ 10264us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.216586 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.216597 (+ 11us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.216713 (+ 116us) tablet.cc:699] Row locks acquired
0811 18:57:07.216718 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:07.216791 (+ 73us) write_op.cc:270] Start()
0811 18:57:07.216854 (+ 63us) write_op.cc:276] Timestamp: P: 1786474627216767 usec, L: 0
0811 18:57:07.216862 (+ 8us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.217223 (+ 361us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.234625 (+ 17402us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.234827 (+ 202us) write_op.cc:301] APPLY: starting
0811 18:57:07.234852 (+ 25us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.234919 (+ 67us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.234966 (+ 47us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.235055 (+ 89us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.235127 (+ 72us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.235139 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:07.235267 (+ 128us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.235362 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.235389 (+ 27us) write_op.cc:454] Released schema lock
0811 18:57:07.235401 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.235481 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.235487 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":736,"apply.run_wall_time_us":736,"num_ops":1,"prepare.queue_time_us":197670,"prepare.run_cpu_time_us":1380,"prepare.run_wall_time_us":14302,"raft.queue_time_us":204,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":4964,"replication_time_us":17720}]]}
I20260811 18:57:08.355379 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_335e9fdd802644cc93a0657a33e74605"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.355978 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_335e9fdd802644cc93a0657a33e74605 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:08.346657 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 616658us
I20260811 18:57:08.349364 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:08.338835 3554 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 264) took 1862 ms. Trace:
I20260811 18:57:08.357090 3554 rpcz_store.cc:276] 0811 18:57:06.476032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.762199 (+286167us) service_pool.cc:227] Handling call
0811 18:57:06.762957 (+ 758us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.763367 (+ 410us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:06.763429 (+ 62us) catalog_manager.cc:2761] Locking tablets
0811 18:57:06.958946 (+195517us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:07.161881 (+202935us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:07.161962 (+ 81us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:07.162020 (+ 58us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:08.338769 (+1176749us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.123394 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.123550 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.123557 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:07.123560 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:07.123723 (+ 163us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.123737 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.123743 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.123998 (+ 255us) tablet.cc:699] Row locks acquired
0811 18:57:07.124013 (+ 15us) write_op.cc:260] PREPARE: finished
0811 18:57:07.124106 (+ 93us) write_op.cc:270] Start()
0811 18:57:07.124159 (+ 53us) write_op.cc:276] Timestamp: P: 1786474627124043 usec, L: 0
0811 18:57:07.124164 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.124401 (+ 237us) log.cc:842] Serialized 725 byte log entry
0811 18:57:07.158637 (+ 34236us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.160361 (+ 1724us) write_op.cc:301] APPLY: starting
0811 18:57:07.160370 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.160396 (+ 26us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.160422 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.160462 (+ 40us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.160485 (+ 23us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:07.160491 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.160531 (+ 40us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.160560 (+ 29us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160570 (+ 10us) write_op.cc:454] Released schema lock
0811 18:57:07.160574 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.160610 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160614 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":896,"child_traces":[["op",{"apply.queue_time_us":1680,"apply.run_cpu_time_us":287,"apply.run_wall_time_us":287,"num_ops":2,"prepare.queue_time_us":163713,"prepare.run_cpu_time_us":1249,"prepare.run_wall_time_us":13097,"raft.queue_time_us":2180,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":34447}]]}
I20260811 18:57:08.357918 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_220d78aeb25b4b64b594e0839928c35e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.358516 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_220d78aeb25b4b64b594e0839928c35e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.358907 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6d1d58835ad24749a00ce27b4bc73c59 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:08.346876 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6ec419ed7747c3b24dc11992518e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.360396 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9efd2f19d74a68a8536fd057377738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.346797 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 490f34d7a403416fbab64e46973746de (DEFAULT_TABLE table=default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.342445 4167 rpcz_store.cc:276] 0811 18:57:05.655787 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656164 (+ 377us) service_pool.cc:227] Handling call
0811 18:57:07.134055 (+1477891us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:07.134090 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.134288 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1475939}
I20260811 18:57:08.361593 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490f34d7a403416fbab64e46973746de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.347358 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.347918 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb (DEFAULT_TABLE table=default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.361821 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 136) took 2706 ms. Trace:
W20260811 18:57:08.271448 3811 kernel_stack_watchdog.cc:198] Thread 3824 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 212ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f4d0c40b980 (unknown) at ??:0
@ 0x7f4d0c4059c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f4d064fc69c google::LogMessage::Flush() at ??:0
@ 0x7f4d064fc6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f4d14540b37 kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7f4d1453fc46 kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7f4d14545ff1 kudu::consensus::Peer::SendNextRequest()::$_3::operator()() at ??:0
@ 0x7f4d14545dfe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fbabf86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f4d0fbabb6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fb97dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f4d0fb98321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f4d0fb35aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f4d0fb345a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:08.377089 3857 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 192) took 1067 ms. Trace:
I20260811 18:57:08.378234 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3ec38f9b55489ba00ed76f9e4456f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.382746 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515d5b5afbd74ec5ab90a503aef2f4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.380399 4167 rpcz_store.cc:276] 0811 18:57:05.655254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656094 (+ 840us) service_pool.cc:227] Handling call
0811 18:57:08.361494 (+2705400us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:08.361509 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.361651 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2703201}
I20260811 18:57:08.334311 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb0316f2e1db4353b9919edf3d8e1c90" } modify_external_catalogs: true
I20260811 18:57:08.386706 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 135) took 2729 ms. Trace:
I20260811 18:57:08.386874 4167 rpcz_store.cc:276] 0811 18:57:05.654691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.663221 (+ 8530us) service_pool.cc:227] Handling call
0811 18:57:08.383916 (+2720695us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:08.383930 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.384097 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2720437}
I20260811 18:57:08.393983 3998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55448 (request call id 188) took 1286 ms. Trace:
W20260811 18:57:08.395701 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
W20260811 18:57:08.396023 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:08.417135 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.417614 4167 tablet_replica.cc:330] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:08.417922 4167 raft_consensus.cc:2229] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:08.418659 4167 pending_rounds.cc:62] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:08.394320 3998 rpcz_store.cc:276] 0811 18:57:07.107292 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.107375 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.311103 (+1203728us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.311133 (+ 30us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.379441 (+ 68308us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.393643 (+ 14202us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.393892 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.446990 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.460008 3867 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 194) took 1147 ms. Trace:
I20260811 18:57:08.446633 4167 pending_rounds.cc:68] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400069692174336 op_type: NO_OP noop_request { }
I20260811 18:57:08.467969 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:08.476346 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.476501 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.479525 4167 raft_consensus.cc:2866] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:08.462198 3871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 193) took 1151 ms. Trace:
I20260811 18:57:08.482314 4167 raft_consensus.cc:2258] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:08.482754 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e51a08e6dbf04e3fa58689ee0cd45db3 consensus state has no leader
I20260811 18:57:08.483183 3549 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 280) took 1480 ms. Trace:
I20260811 18:57:08.484136 4167 ts_tablet_manager.cc:1905] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:08.483435 3549 rpcz_store.cc:276] 0811 18:57:07.002091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.165716 (+163625us) service_pool.cc:227] Handling call
0811 18:57:08.342045 (+1176329us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.342307 (+ 262us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:08.342506 (+ 199us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:08.342591 (+ 85us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:08.482307 (+139716us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.482947 (+ 640us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.390790 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.390944 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.390952 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:08.390954 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:08.391104 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.391119 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.391126 (+ 7us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:08.391200 (+ 74us) tablet.cc:699] Row locks acquired
0811 18:57:08.391202 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:08.391255 (+ 53us) write_op.cc:270] Start()
0811 18:57:08.391297 (+ 42us) write_op.cc:276] Timestamp: P: 1786474628391129 usec, L: 0
0811 18:57:08.391301 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.391545 (+ 244us) log.cc:842] Serialized 405 byte log entry
0811 18:57:08.451373 (+ 59828us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.478407 (+ 27034us) write_op.cc:301] APPLY: starting
0811 18:57:08.478437 (+ 30us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.478518 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.478689 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.478847 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.478900 (+ 53us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:08.478920 (+ 20us) write_op.cc:312] APPLY: finished
0811 18:57:08.479095 (+ 175us) log.cc:842] Serialized 23 byte log entry
0811 18:57:08.481601 (+ 2506us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.481632 (+ 31us) write_op.cc:454] Released schema lock
0811 18:57:08.481643 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":26897,"num_ops":1,"prepare.queue_time_us":47689,"prepare.run_cpu_time_us":920,"prepare.run_wall_time_us":18187,"replication_time_us":60034,"thread_start_us":85027,"threads_started":2}]]}
I20260811 18:57:08.499396 3550 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 279) took 1516 ms. Trace:
I20260811 18:57:08.504637 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b199f4c9e14b468cb26cbd61fc8586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.505115 3872 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 196) took 1190 ms. Trace:
I20260811 18:57:08.567466 4162 ts_tablet_manager.cc:1428] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.224s user 0.003s sys 0.016s
I20260811 18:57:08.577222 4162 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:08.799654 4503 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:08.949668 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap starting.
W20260811 18:57:08.903290 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 290ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:08.668385 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e902efbb774244d18e3ba165bfb7c695 (DEFAULT_TABLE table=default.Table_60c6f6ccfa2747e1b4fff22a5779d63e [id=11c6327a2ff84680bcd60893f05b3493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.668962 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c6852aaef1d5486b87e5462eae532719 (DEFAULT_TABLE table=default.Table_9409f5b4b0034412aedc183b302556a3 [id=75ea77c989e746f88a94e72de6cd11ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.673111 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 71df3738a5cf4b12820dda131ced62e1 (DEFAULT_TABLE table=default.Table_220d78aeb25b4b64b594e0839928c35e [id=b9face92d47f453ab5de5af21691edb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.674369 4506 log.cc:924] Time spent T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260811 18:57:09.015095 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e902efbb774244d18e3ba165bfb7c695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.676241 3714 tablet_service.cc:1467] Processing CreateTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 (DEFAULT_TABLE table=default.Table_9d18dc31cde54577b8bcc329191b74ef [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.016002 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2718cd4a98742dfa8014a8a38c69c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.679345 3709 tablet_service.cc:1467] Processing CreateTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f (DEFAULT_TABLE table=default.Table_335e9fdd802644cc93a0657a33e74605 [id=14b4d412d33c43fdb3d20c85b82c1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.018792 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe203387bc9c45ba82afe6b058aa5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.685612 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 (DEFAULT_TABLE table=default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.023077 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43ad6993b63e4bbd98c8cc52f1a53451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.698483 4162 tablet_bootstrap.cc:654] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.726984 4167 ts_tablet_manager.cc:1918] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:08.851405 4476 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:09.027940 4167 log.cc:1196] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:09.028386 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6852aaef1d5486b87e5462eae532719. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:08.905362 3388 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.125:38889 timed out after 0.587s (SENT) [suppressed 8 similar messages]
I20260811 18:57:09.268386 4503 log.cc:824] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:09.692754 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=14 ignored=0} inserts{seen=23 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 3 replicates
W20260811 18:57:09.496275 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 2429ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:08.499670 3550 rpcz_store.cc:276] 0811 18:57:06.983097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.120052 (+136955us) service_pool.cc:227] Handling call
0811 18:57:08.317424 (+1197372us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.317911 (+ 487us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.498796 (+180885us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.498964 (+ 168us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.499116 (+ 152us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.409061 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.409239 (+ 178us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.409250 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:08.409254 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.409393 (+ 139us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.409411 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.409419 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.409585 (+ 166us) tablet.cc:699] Row locks acquired
0811 18:57:08.409587 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:08.409639 (+ 52us) write_op.cc:270] Start()
0811 18:57:08.409689 (+ 50us) write_op.cc:276] Timestamp: P: 1786474628409512 usec, L: 0
0811 18:57:08.409695 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.409971 (+ 276us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.479801 (+ 69830us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.497808 (+ 18007us) write_op.cc:301] APPLY: starting
0811 18:57:08.497829 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.497880 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.497911 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.498055 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.498095 (+ 40us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.498105 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:08.498242 (+ 137us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.498642 (+ 400us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.498668 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:08.498679 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":17949,"num_ops":2,"prepare.queue_time_us":57171,"prepare.run_cpu_time_us":1043,"prepare.run_wall_time_us":9455,"replication_time_us":70076,"threads_started":1}]]}
I20260811 18:57:09.711858 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap complete.
W20260811 18:57:09.712406 3550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 33) took 1124 ms (client timeout 1000 ms). Trace:
W20260811 18:57:09.712558 3550 rpcz_store.cc:269] 0811 18:57:08.588416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.712099 (+1123683us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:09.712333 (+ 234us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:09.713227 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_919f7bdbd76f4707813e30d37dfc3767"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:09.221671 3388 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 316178us
I20260811 18:57:08.507503 3860 rpcz_store.cc:276] 0811 18:57:07.313181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.313263 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:08.322476 (+1009213us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.322502 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.334444 (+ 11942us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.337888 (+ 3444us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.338057 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.510281 4163 tablet_bootstrap.cc:654] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.511628 3857 rpcz_store.cc:276] 0811 18:57:07.309736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.309857 (+ 121us) service_pool.cc:227] Handling call
0811 18:57:08.318522 (+1008665us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318551 (+ 29us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.326064 (+ 7513us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.376822 (+ 50758us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.377039 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.513374 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65234acc3d5a462683edbe8f223eeefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.521647 3893 raft_consensus.cc:2454] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:09.779363 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490f34d7a403416fbab64e46973746de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.779507 3893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.65:36347 (request call id 44) took 1431 ms. Trace:
I20260811 18:57:09.779712 3893 rpcz_store.cc:276] 0811 18:57:08.348367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.349231 (+ 864us) service_pool.cc:227] Handling call
0811 18:57:09.779450 (+1430219us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.521890 3871 rpcz_store.cc:276] 0811 18:57:07.310816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.310920 (+ 104us) service_pool.cc:227] Handling call
0811 18:57:08.318585 (+1007665us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318611 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.333715 (+ 15104us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.461850 (+128135us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.462042 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.513533 3872 rpcz_store.cc:276] 0811 18:57:07.314387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.314466 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:08.326530 (+1012064us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.326587 (+ 57us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.439374 (+112787us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.504845 (+ 65471us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.505031 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.873997 4503 raft_consensus.cc:348] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:57:09.809768 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3255ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:08.523305 3867 rpcz_store.cc:276] 0811 18:57:07.311920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.312003 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.318827 (+1006824us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318851 (+ 24us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.338642 (+ 19791us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.458897 (+120255us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.459477 (+ 580us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.529765 3865 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 198) took 1209 ms. Trace:
I20260811 18:57:10.856699 3865 rpcz_store.cc:276] 0811 18:57:07.320102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.320200 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.342649 (+1022449us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.342674 (+ 25us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.518755 (+176081us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.529494 (+ 10739us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.529675 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.788858 3370 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: OK, instance_id { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" instance_seqno: 1786474627171025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:10.857882 3370 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:09.028261 4476 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:10.858765 4476 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:09.714048 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_919f7bdbd76f4707813e30d37dfc3767 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.635121 3940 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:08.810937 3870 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 197) took 1494 ms. Trace:
I20260811 18:57:10.861056 3870 rpcz_store.cc:276] 0811 18:57:07.316415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.316490 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:08.336907 (+1020417us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.336939 (+ 32us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.433783 (+ 96844us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.810532 (+376749us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.810837 (+ 305us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.510810 3824 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:09.773559 4565 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:10.861858 4565 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:09.783396 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f335e50ae57547ed80b144129faaffbb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:10.364069 3388 leader_election.cc:334] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889): Timed out: RequestConsensusVote RPC to 127.3.72.125:38889 timed out after 0.587s (SENT)
I20260811 18:57:10.862694 3388 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:09.005527 3707 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1923ce0049448fe4e28f2dd258dc (DEFAULT_TABLE table=default.Table_0cb87feb6db445d792ba8021cbe75510 [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.557698 3548 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 285) took 1491 ms. Trace:
I20260811 18:57:10.863961 3548 rpcz_store.cc:276] 0811 18:57:07.066107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.345251 (+1279144us) service_pool.cc:227] Handling call
0811 18:57:08.346290 (+ 1039us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.367395 (+ 21105us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.557224 (+189829us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.557377 (+ 153us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.557513 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.424337 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.424527 (+ 190us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.424537 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:08.424542 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:08.424683 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.424698 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.424706 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.424827 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:08.424831 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.424882 (+ 51us) write_op.cc:270] Start()
0811 18:57:08.424927 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628424753 usec, L: 0
0811 18:57:08.424933 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.429693 (+ 4760us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.555947 (+126254us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.556034 (+ 87us) write_op.cc:301] APPLY: starting
0811 18:57:08.556048 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.556110 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.556189 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.556521 (+ 332us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.556573 (+ 52us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.556581 (+ 8us) write_op.cc:312] APPLY: finished
0811 18:57:08.556743 (+ 162us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.557079 (+ 336us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.557107 (+ 28us) write_op.cc:454] Released schema lock
0811 18:57:08.557116 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.557904 (+ 788us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.557909 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":38,"apply.run_cpu_time_us":879,"apply.run_wall_time_us":1950,"num_ops":2,"prepare.queue_time_us":56294,"prepare.run_cpu_time_us":1125,"prepare.run_wall_time_us":6517,"replication_time_us":130994}]]}
W20260811 18:57:10.864952 3548 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.66:35625 (request call id 30) took 1080 ms (client timeout 1000 ms). Trace:
W20260811 18:57:10.865061 3548 rpcz_store.cc:269] 0811 18:57:09.784892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:10.864839 (+1079947us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:10.864893 (+ 54us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:10.865523 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_79396737f1a343c98184686b7eda44ab" } new_table_name: "default.Table_45c439603b1140e998b5d9199026da49" modify_external_catalogs: true
I20260811 18:57:09.827950 4163 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:10.887220 4503 raft_consensus.cc:729] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85eb97df7824bc9bad9dca349b0c180, State: Initialized, Role: FOLLOWER
W20260811 18:57:11.039822 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1284ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:08.559392 3551 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 283) took 1525 ms. Trace:
I20260811 18:57:11.321699 4503 consensus_queue.cc:260] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 1.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:11.321344 3551 rpcz_store.cc:276] 0811 18:57:07.034050 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.339680 (+1305630us) service_pool.cc:227] Handling call
0811 18:57:08.340674 (+ 994us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.341277 (+ 603us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.559015 (+217738us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.559137 (+ 122us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.559236 (+ 99us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.430915 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.431089 (+ 174us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.431098 (+ 9us) write_op.cc:435] Acquired schema lock
0811 18:57:08.431103 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:08.431224 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.431238 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.431246 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.431355 (+ 109us) tablet.cc:699] Row locks acquired
0811 18:57:08.431359 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.431407 (+ 48us) write_op.cc:270] Start()
0811 18:57:08.431451 (+ 44us) write_op.cc:276] Timestamp: P: 1786474628431277 usec, L: 0
0811 18:57:08.431457 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.431744 (+ 287us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.558098 (+126354us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.558196 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:08.558210 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.558258 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.558280 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.558494 (+ 214us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.558529 (+ 35us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.558536 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.558622 (+ 86us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.558885 (+ 263us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.558903 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:08.558911 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.559830 (+ 919us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.559837 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42,"apply.run_cpu_time_us":660,"apply.run_wall_time_us":1692,"num_ops":2,"prepare.queue_time_us":61989,"prepare.run_cpu_time_us":989,"prepare.run_wall_time_us":14055,"replication_time_us":126611}]]}
I20260811 18:57:11.328462 4503 sys_catalog.cc:564] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:10.866535 4163 ts_tablet_manager.cc:1397] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 2.561s user 0.005s sys 0.012s
I20260811 18:57:11.322451 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet f61d3822fc2d4b569f223238a6464e0a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:10.877125 4593 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:11.370034 4163 raft_consensus.cc:348] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:10.005074 3811 kernel_stack_watchdog.cc:198] Thread 3824 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 1009ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f4d0c40b980 (unknown) at ??:0
@ 0x7f4d0c4059c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f4d064fc69c google::LogMessage::Flush() at ??:0
@ 0x7f4d064fc6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f4d145a79f7 kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7f4d145a8ce1 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7f4d145ab73a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7f4d145ab4fe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fbabf86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f4d0fbabb6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fb97dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f4d0fb98321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f4d0fb35aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f4d0fb345a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:09.029347 4167 ts_tablet_manager.cc:1939] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:10.877368 3388 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 513282us
I20260811 18:57:08.574699 3554 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 289) took 1476 ms. Trace:
I20260811 18:57:11.370949 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 142) took 5599 ms. Trace:
W20260811 18:57:10.889950 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 2.5329e+06us
I20260811 18:57:11.371064 4167 rpcz_store.cc:276] 0811 18:57:05.771325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.771685 (+ 360us) service_pool.cc:227] Handling call
0811 18:57:08.387238 (+2615553us) ts_tablet_manager.cc:1145] Deleting tablet 6d1d58835ad24749a00ce27b4bc73c59
0811 18:57:08.387252 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.627570 (+240318us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.726739 (+ 99169us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.370883 (+2644144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2615102}
I20260811 18:57:10.037185 4572 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.371551 4167 tablet_replica.cc:330] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:11.328495 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:11.371536 4572 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:11.372249 4438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43854 (request call id 1) took 1596 ms (client timeout 587 ms). Trace:
I20260811 18:57:11.372567 4572 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:10.052956 4573 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:11.372822 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 71df3738a5cf4b12820dda131ced62e1 (DEFAULT_TABLE table=default.Table_220d78aeb25b4b64b594e0839928c35e [id=b9face92d47f453ab5de5af21691edb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:11.548520 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3550ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:11.370733 3554 rpcz_store.cc:276] 0811 18:57:07.098547 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.357708 (+1259161us) service_pool.cc:227] Handling call
0811 18:57:08.358620 (+ 912us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.373671 (+ 15051us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.374122 (+ 451us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.574240 (+200118us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.574514 (+ 274us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.574618 (+ 104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.448721 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.448837 (+ 116us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.448844 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:08.448848 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.448949 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.448961 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.448968 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.449058 (+ 90us) tablet.cc:699] Row locks acquired
0811 18:57:08.449062 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.449106 (+ 44us) write_op.cc:270] Start()
0811 18:57:08.449137 (+ 31us) write_op.cc:276] Timestamp: P: 1786474628448980 usec, L: 0
0811 18:57:08.449143 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.449313 (+ 170us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.573603 (+124290us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.573663 (+ 60us) write_op.cc:301] APPLY: starting
0811 18:57:08.573672 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.573710 (+ 38us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.573740 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.573835 (+ 95us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.573869 (+ 34us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.573878 (+ 9us) write_op.cc:312] APPLY: finished
0811 18:57:08.573948 (+ 70us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.574151 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.574164 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:08.574170 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.575573 (+ 1403us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.575579 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27846912,"child_traces":[["op",{"apply.queue_time_us":28,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":1954,"num_ops":2,"prepare.queue_time_us":74175,"prepare.run_cpu_time_us":729,"prepare.run_wall_time_us":38374,"replication_time_us":124445}]]}
I20260811 18:57:11.651948 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a82335ab20e412c93a5ac0938a96012"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:09.030548 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71df3738a5cf4b12820dda131ced62e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:11.652922 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a82335ab20e412c93a5ac0938a96012 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.372884 4573 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.653365 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1923ce0049448fe4e28f2dd258dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:11.372421 4438 rpcz_store.cc:269] 0811 18:57:09.776212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.776292 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:11.372178 (+1595886us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.836782 3852 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 204) took 2398 ms. Trace:
I20260811 18:57:11.334926 4441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
W20260811 18:57:11.654244 4441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43854 (request call id 0) took 2984 ms (client timeout 587 ms). Trace:
W20260811 18:57:11.654470 4441 rpcz_store.cc:269] 0811 18:57:08.669507 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.669568 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:11.654198 (+2984630us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.574985 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ed3346bd12d04c45a511308c427856e5
I20260811 18:57:10.873214 3864 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 200) took 3528 ms. Trace:
I20260811 18:57:10.879350 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902ad70a25546be926ff8f7bf2ed64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.655200 3553 catalog_manager.cc:6565] ReplaceTablet: tablet ed3346bd12d04c45a511308c427856e5 deleted and replaced by tablet b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:11.655349 3553 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 286) took 4557 ms. Trace:
I20260811 18:57:10.879983 3853 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 203) took 3517 ms. Trace:
I20260811 18:57:10.889523 4592 raft_consensus.cc:2781] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:10.891278 4169 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:11.373798 3871 tablet_service.cc:1467] Processing CreateTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f (DEFAULT_TABLE table=default.Table_335e9fdd802644cc93a0657a33e74605 [id=14b4d412d33c43fdb3d20c85b82c1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.655467 3553 rpcz_store.cc:276] 0811 18:57:07.097824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.350461 (+1252637us) service_pool.cc:227] Handling call
0811 18:57:08.371038 (+ 20577us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7dada82 00007f61f7f4c6c7 00007f61ef23eda7 00007f61ef23ead5 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:11.655306 (+3284268us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.445044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.445223 (+ 179us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.445233 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:08.445237 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.445436 (+ 199us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.445453 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.445461 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.445566 (+ 105us) tablet.cc:699] Row locks acquired
0811 18:57:08.445571 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.445618 (+ 47us) write_op.cc:270] Start()
0811 18:57:08.445660 (+ 42us) write_op.cc:276] Timestamp: P: 1786474628445490 usec, L: 0
0811 18:57:08.445666 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.446055 (+ 389us) log.cc:842] Serialized 622 byte log entry
0811 18:57:08.571255 (+125200us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.571353 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:08.571371 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.571421 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.571505 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.571633 (+ 128us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.571752 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:08.571763 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:08.571870 (+ 107us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.572224 (+ 354us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.572250 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:08.572258 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.572588 (+ 330us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.572595 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2383,"spinlock_wait_cycles":45226240,"child_traces":[["op",{"apply.queue_time_us":41,"apply.run_cpu_time_us":784,"apply.run_wall_time_us":1306,"num_ops":2,"prepare.queue_time_us":72796,"prepare.run_cpu_time_us":1006,"prepare.run_wall_time_us":1854,"replication_time_us":125554}]]}
I20260811 18:57:11.656610 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a85e1485efcd445a972fe25c37f2bb9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:11.657272 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a85e1485efcd445a972fe25c37f2bb9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.328147 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:09.044541 4162 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:11.658514 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:11.658643 4162 ts_tablet_manager.cc:1397] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 3.087s user 0.002s sys 0.005s
I20260811 18:57:10.759351 4586 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.412786 4443 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.14->[1.15-1.278] Dedup: 1.17->[1.18-1.278]
I20260811 18:57:09.106843 3796 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:11.374663 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 (DEFAULT_TABLE table=default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.375042 3856 tablet_service.cc:1467] Processing CreateTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 (DEFAULT_TABLE table=default.Table_9d18dc31cde54577b8bcc329191b74ef [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.375172 4163 raft_consensus.cc:374] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:11.659547 4162 raft_consensus.cc:348] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.375473 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.659860 4162 raft_consensus.cc:374] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:11.659947 4162 raft_consensus.cc:729] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:11.375681 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.375846 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.375998 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:10.866863 4565 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.376595 3857 tablet_service.cc:1467] Processing CreateTablet for tablet e902efbb774244d18e3ba165bfb7c695 (DEFAULT_TABLE table=default.Table_60c6f6ccfa2747e1b4fff22a5779d63e [id=11c6327a2ff84680bcd60893f05b3493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.660218 4162 consensus_queue.cc:260] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.377022 3860 tablet_service.cc:1467] Processing CreateTablet for tablet c6852aaef1d5486b87e5462eae532719 (DEFAULT_TABLE table=default.Table_9409f5b4b0034412aedc183b302556a3 [id=75ea77c989e746f88a94e72de6cd11ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.379426 4572 consensus_queue.cc:1035] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:08.597003 3547 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 287) took 1498 ms. Trace:
I20260811 18:57:08.599251 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:11.661315 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 282) took 4658 ms. Trace:
I20260811 18:57:11.660813 3547 rpcz_store.cc:276] 0811 18:57:07.097992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.353343 (+1255351us) service_pool.cc:227] Handling call
0811 18:57:08.354280 (+ 937us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.373692 (+ 19412us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.374121 (+ 429us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.596594 (+222473us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.596745 (+ 151us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.596859 (+ 114us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.487200 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.487382 (+ 182us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.487393 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:08.487397 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.487545 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.487561 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.487569 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.487815 (+ 246us) tablet.cc:699] Row locks acquired
0811 18:57:08.487820 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.487878 (+ 58us) write_op.cc:270] Start()
0811 18:57:08.487929 (+ 51us) write_op.cc:276] Timestamp: P: 1786474628487743 usec, L: 0
0811 18:57:08.487935 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.561192 (+ 73257us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.561488 (+ 296us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0418d02 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c 00007f61e4f50344 00007f61e4f5391b 00007f61e4f53798 00000000009edb7c
0811 18:57:08.594556 (+ 33068us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.595105 (+ 549us) write_op.cc:301] APPLY: starting
0811 18:57:08.595124 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.595177 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.595179 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.595384 (+ 205us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.595436 (+ 52us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.595445 (+ 9us) write_op.cc:312] APPLY: finished
0811 18:57:08.595559 (+ 114us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.596442 (+ 883us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.596466 (+ 24us) write_op.cc:454] Released schema lock
0811 18:57:08.596476 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.597175 (+ 699us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.597181 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":37933824,"child_traces":[["op",{"apply.queue_time_us":460,"apply.run_cpu_time_us":804,"apply.run_wall_time_us":2147,"mutex_wait_us":23,"num_ops":2,"prepare.queue_time_us":112617,"prepare.run_cpu_time_us":2436,"prepare.run_wall_time_us":74373,"raft.queue_time_us":78,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":382,"replication_time_us":106582,"spinlock_wait_cycles":167904000}]]}
I20260811 18:57:11.661455 3546 rpcz_store.cc:276] 0811 18:57:07.002632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.333874 (+1331242us) service_pool.cc:227] Handling call
0811 18:57:08.386724 (+ 52850us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.386973 (+ 249us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:08.387061 (+ 88us) catalog_manager.cc:2761] Locking tablets
0811 18:57:08.573999 (+186938us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:08.598897 (+ 24898us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:08.599023 (+ 126us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:08.599079 (+ 56us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:11.661266 (+3062187us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.575682 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.575832 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.575838 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.575842 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.576002 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.576016 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.576023 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.576274 (+ 251us) tablet.cc:699] Row locks acquired
0811 18:57:08.576277 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:08.576328 (+ 51us) write_op.cc:270] Start()
0811 18:57:08.576410 (+ 82us) write_op.cc:276] Timestamp: P: 1786474628576190 usec, L: 0
0811 18:57:08.576414 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.582094 (+ 5680us) log.cc:842] Serialized 725 byte log entry
0811 18:57:08.582817 (+ 723us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0418d02 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c 00007f61e4f50344 00007f61e4f5391b 00007f61e4f53798 00000000009edb7c
0811 18:57:08.597811 (+ 14994us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.597941 (+ 130us) write_op.cc:301] APPLY: starting
0811 18:57:08.597955 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.597999 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.598002 (+ 3us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.598068 (+ 66us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.598101 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:08.598107 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:08.598192 (+ 85us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.598778 (+ 586us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.598803 (+ 25us) write_op.cc:454] Released schema lock
0811 18:57:08.598811 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.604806 (+ 5995us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.604816 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":13696,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":595,"apply.run_wall_time_us":6953,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":1670,"prepare.run_wall_time_us":8181,"raft.queue_time_us":40,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":325,"replication_time_us":21356,"spinlock_wait_cycles":12143488}]]}
I20260811 18:57:11.661950 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_056e34af1fcd4642abd908808acf8a67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:11.662451 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" } modify_external_catalogs: true
W20260811 18:57:11.662650 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_056e34af1fcd4642abd908808acf8a67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.662731 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" } modify_external_catalogs: true
W20260811 18:57:08.843233 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 consensus state has no leader
W20260811 18:57:11.664884 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet e5fd9fce095e4a769dbb38c7903359cc consensus state has no leader
I20260811 18:57:09.518744 4554 consensus_queue.cc:1035] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260811 18:57:09.504180 3522 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: OK, instance_id { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" instance_seqno: 1786474627171025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:11.667068 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:09.602849 4555 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:09.965920 4556 raft_consensus.cc:482] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.667270 4555 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.667609 4167 raft_consensus.cc:2229] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:10.276178 4574 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:11.667830 4574 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.668063 4167 pending_rounds.cc:62] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:11.668183 4167 pending_rounds.cc:68] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400066460418048 op_type: NO_OP noop_request { }
I20260811 18:57:10.336215 4576 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.668463 4576 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.555601 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:11.669837 3552 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 281) took 4667 ms. Trace:
I20260811 18:57:11.669982 3552 rpcz_store.cc:276] 0811 18:57:07.002427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.250686 (+248259us) service_pool.cc:227] Handling call
0811 18:57:08.350789 (+1100103us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.350965 (+ 176us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:08.351018 (+ 53us) catalog_manager.cc:2761] Locking tablets
0811 18:57:08.351170 (+ 152us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:08.555139 (+203969us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:08.555262 (+ 123us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:08.555323 (+ 61us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:11.669791 (+3114468us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.418556 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.418682 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.418688 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.418691 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:08.418868 (+ 177us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.418885 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.418893 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.419098 (+ 205us) tablet.cc:699] Row locks acquired
0811 18:57:08.419102 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.419149 (+ 47us) write_op.cc:270] Start()
0811 18:57:08.419194 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628419022 usec, L: 0
0811 18:57:08.419198 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.419430 (+ 232us) log.cc:842] Serialized 725 byte log entry
0811 18:57:08.553934 (+134504us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.554065 (+ 131us) write_op.cc:301] APPLY: starting
0811 18:57:08.554094 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.554161 (+ 67us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.554236 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.554358 (+ 122us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.554400 (+ 42us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:08.554407 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.554584 (+ 177us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.554974 (+ 390us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.555014 (+ 40us) write_op.cc:454] Released schema lock
0811 18:57:08.555023 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.555791 (+ 768us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.555797 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":57,"apply.run_cpu_time_us":804,"apply.run_wall_time_us":1803,"num_ops":2,"prepare.queue_time_us":66615,"prepare.run_cpu_time_us":987,"prepare.run_wall_time_us":5718,"replication_time_us":134700}]]}
I20260811 18:57:11.670979 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6d55543667e9439eb0420ca7ec6e8e74" } new_table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" modify_external_catalogs: true
I20260811 18:57:09.782995 3676 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:10.387156 3664 kernel_stack_watchdog.cc:198] Thread 3676 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 265ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fcdd5c86980 (unknown) at ??:0
@ 0x7fcdd5c809c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fcdcfd7769c google::LogMessage::Flush() at ??:0
@ 0x7fcdcfd776d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fcddde229f7 kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7fcddde23ce1 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fcddde2673a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fcddde264fe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9426f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fcdd9426b6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9412dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fcdd9413321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fcdd93b0aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fcdd93af5a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:10.380551 4578 raft_consensus.cc:482] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.672713 4578 raft_consensus.cc:504] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:10.222826 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 89b5c0f3673445bbba9136a040ca9bbb consensus state has no leader
I20260811 18:57:10.782948 4580 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.674500 4580 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:11.087174 4476 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.675932 4476 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.560760 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc0e1659d31645b28fe322618e22cb6c" } modify_external_catalogs: true
I20260811 18:57:11.677273 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc0e1659d31645b28fe322618e22cb6c" } modify_external_catalogs: true
I20260811 18:57:11.377933 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:11.667346 4556 raft_consensus.cc:504] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:11.378454 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:08.574854 3555 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 288) took 1476 ms. Trace:
I20260811 18:57:11.377341 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6d1d58835ad24749a00ce27b4bc73c59 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
W20260811 18:57:11.681180 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 303848us
I20260811 18:57:11.682976 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:11.688743 4555 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.699486 4167 raft_consensus.cc:2866] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:11.699774 4167 raft_consensus.cc:2258] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:12.471638 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1460ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:11.723701 3864 rpcz_store.cc:276] 0811 18:57:07.344536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.344630 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:08.342976 (+998346us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.343075 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:10.868245 (+2525170us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.872936 (+ 4691us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:10.873136 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.723459 3853 rpcz_store.cc:276] 0811 18:57:07.362128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.362226 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.345928 (+983702us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.345951 (+ 23us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:10.867425 (+2521474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.879446 (+ 12021us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:10.879849 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.741096 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71df3738a5cf4b12820dda131ced62e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.700870 4167 ts_tablet_manager.cc:1905] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:11.741484 3852 rpcz_store.cc:276] 0811 18:57:07.438203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.438286 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.346150 (+907864us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.346168 (+ 18us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:09.783687 (+1437519us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:09.836415 (+ 52728us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:09.836685 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.741743 4163 raft_consensus.cc:729] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:11.723608 4586 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.770839 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08436f33db043b1bc6395cd92102e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.771343 4586 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.743510 4169 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:12.771960 4169 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:12.772406 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 149) took 7017 ms. Trace:
I20260811 18:57:12.772526 4169 rpcz_store.cc:276] 0811 18:57:05.755415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.755624 (+ 209us) service_pool.cc:227] Handling call
0811 18:57:08.311745 (+2556121us) ts_tablet_manager.cc:1145] Deleting tablet 6ef0a376939d4d57b0d95db644f25dd9
0811 18:57:08.311759 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.429688 (+117929us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.891127 (+2461439us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.772340 (+1881213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2508479}
I20260811 18:57:11.681022 3555 rpcz_store.cc:276] 0811 18:57:07.098217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.355150 (+1256933us) service_pool.cc:227] Handling call
0811 18:57:08.356103 (+ 953us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.372451 (+ 16348us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.372931 (+ 480us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.574552 (+201621us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.574625 (+ 73us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.574684 (+ 59us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.446965 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.447246 (+ 281us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.447252 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.447268 (+ 16us) tablet.cc:660] Decoding operations
0811 18:57:08.447395 (+ 127us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.447408 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.447424 (+ 16us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.447539 (+ 115us) tablet.cc:699] Row locks acquired
0811 18:57:08.447544 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.447596 (+ 52us) write_op.cc:270] Start()
0811 18:57:08.447641 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628447458 usec, L: 0
0811 18:57:08.447647 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.447925 (+ 278us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.572767 (+124842us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.572851 (+ 84us) write_op.cc:301] APPLY: starting
0811 18:57:08.572866 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.572910 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.572940 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.573060 (+ 120us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.573093 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.573100 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.573170 (+ 70us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.573418 (+ 248us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.573438 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:08.573446 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.573502 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.573506 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":33415040,"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":519,"apply.run_wall_time_us":700,"num_ops":2,"prepare.queue_time_us":73449,"prepare.run_cpu_time_us":1086,"prepare.run_wall_time_us":1733,"replication_time_us":125095}]]}
I20260811 18:57:12.773059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 151) took 7016 ms. Trace:
I20260811 18:57:12.773175 4169 rpcz_store.cc:276] 0811 18:57:05.756857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.756916 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:12.772830 (+7015914us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:12.772868 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.772995 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6968997}
I20260811 18:57:12.773535 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 152) took 7016 ms. Trace:
I20260811 18:57:12.773634 4169 rpcz_store.cc:276] 0811 18:57:05.757290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.757356 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:12.773389 (+7016033us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:12.773398 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.773476 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6968546}
I20260811 18:57:12.773977 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 148) took 7024 ms. Trace:
I20260811 18:57:12.774068 4169 rpcz_store.cc:276] 0811 18:57:05.749406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.749503 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:12.773827 (+7024324us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:12.773837 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.773918 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6959143}
I20260811 18:57:11.744118 3872 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1923ce0049448fe4e28f2dd258dc (DEFAULT_TABLE table=default.Table_0cb87feb6db445d792ba8021cbe75510 [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.774667 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1923ce0049448fe4e28f2dd258dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.775630 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3ec38f9b55489ba00ed76f9e4456f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.745339 4573 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.744593 4592 raft_consensus.cc:686] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:12.778896 4592 consensus_queue.cc:237] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.701783 3707 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38286 (request call id 208) took 2697 ms. Trace:
I20260811 18:57:12.780272 3707 rpcz_store.cc:276] 0811 18:57:09.003914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.004891 (+ 977us) service_pool.cc:227] Handling call
0811 18:57:10.863369 (+1858478us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:10.863402 (+ 33us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.657490 (+794088us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.701376 (+ 43886us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.701646 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.860515 3858 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 199) took 4528 ms. Trace:
I20260811 18:57:11.843371 3550 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 291) took 4686 ms. Trace:
I20260811 18:57:12.780773 3858 rpcz_store.cc:276] 0811 18:57:07.332193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.332291 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.426502 (+1094211us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.426544 (+ 42us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.750107 (+3323563us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.860220 (+110113us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.860461 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:11.742302 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:12.581609 4658 raft_consensus.cc:482] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
W20260811 18:57:12.781216 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 1.05441e+06us
I20260811 18:57:12.781235 4658 raft_consensus.cc:504] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.782198 4658 leader_election.cc:288] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.699005 4162 ts_tablet_manager.cc:1428] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.040s user 0.001s sys 0.001s
W20260811 18:57:12.783990 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.017s (SENT)
W20260811 18:57:12.784042 4162 ts_tablet_manager.cc:1460] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Tablet startup took 1125ms
W20260811 18:57:12.784173 4162 ts_tablet_manager.cc:1462] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Trace:
0811 18:57:03.144267 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.144338 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:03.145184 (+ 846us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:03.145224 (+ 40us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:03.150679 (+ 5455us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.158855 (+ 8176us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:03.159011 (+ 156us) inbound_call.cc:173] Queueing success response
0811 18:57:08.571485 (+5412474us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0811 18:57:11.658747 (+3087262us) ts_tablet_manager.cc:1429] Starting tablet replica
0811 18:57:11.658996 (+ 249us) tablet_replica.cc:259] Starting instrumentation
0811 18:57:11.659147 (+ 151us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":6368,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":53,"tablet-open.queue_time_us":5412394,"thread_start_us":30122,"threads_started":1}
I20260811 18:57:12.784718 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3060822021431584a494e6941be756 (DEFAULT_TABLE table=default.Table_a85e1485efcd445a972fe25c37f2bb9e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.784870 4162 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:12.784924 3867 tablet_service.cc:1467] Processing CreateTablet for tablet c26851245cab4a2d80e94d4fe278fd50 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.786003 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.770850 4163 consensus_queue.cc:260] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.786197 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3060822021431584a494e6941be756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.707816 4660 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
I20260811 18:57:12.787289 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.787292 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.788442 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 (DEFAULT_TABLE table=default.Table_056e34af1fcd4642abd908808acf8a67 [id=1bcc678c540844b38d761cec1618c852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.905488 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:12.789505 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.702407 4574 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702447 4576 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702677 4578 leader_election.cc:288] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702881 4580 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.930943 3554 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 294) took 4705 ms. Trace:
I20260811 18:57:11.702955 4476 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:11.932816 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8eb041f3d704762868a048faec0e0fc consensus state has no leader
I20260811 18:57:11.934561 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f335e50ae57547ed80b144129faaffbb
I20260811 18:57:11.703188 3744 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:11.703454 4556 leader_election.cc:288] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.703598 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:11.703456 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:12.790515 3744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:51189 (request call id 27) took 1414 ms. Trace:
W20260811 18:57:11.703984 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 1.8895e+06us
I20260811 18:57:12.790666 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc5448a555944dfba8ccae09e25cbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.790663 3744 rpcz_store.cc:276] 0811 18:57:11.376369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.377764 (+ 1395us) service_pool.cc:227] Handling call
0811 18:57:12.790455 (+1412691us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:11.951768 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e1efffade0f486d8d627e56e0be6ff6 consensus state has no leader
I20260811 18:57:11.704895 4637 raft_consensus.cc:2781] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:12.791616 4637 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:12.791929 4637 raft_consensus.cc:3037] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:11.952519 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:11.958628 3547 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 296) took 4505 ms. Trace:
I20260811 18:57:12.004621 3553 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 295) took 4677 ms. Trace:
W20260811 18:57:12.673421 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0ace00158b744bd8a97dc1c47fc6049c consensus state has no leader
I20260811 18:57:11.704561 3719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38286 (request call id 205) took 3036 ms. Trace:
I20260811 18:57:12.001572 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c26851245cab4a2d80e94d4fe278fd50 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.794628 3719 rpcz_store.cc:276] 0811 18:57:08.667626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.669424 (+ 1798us) service_pool.cc:227] Handling call
0811 18:57:09.014788 (+345364us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:09.014825 (+ 37us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.662562 (+2647737us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.704206 (+ 41644us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.704443 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.794845 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26851245cab4a2d80e94d4fe278fd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.001849 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 (DEFAULT_TABLE table=default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.795262 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 221d0d8d69f142a29ec7065a82b3f769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.796968 4637 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.002604 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.056010 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 (DEFAULT_TABLE table=default.Table_056e34af1fcd4642abd908808acf8a67 [id=1bcc678c540844b38d761cec1618c852]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:11.828282 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.269s (SENT)
I20260811 18:57:12.798051 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122b7898fea44ef78f269b1505d8ddc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.095178 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3060822021431584a494e6941be756 (DEFAULT_TABLE table=default.Table_a85e1485efcd445a972fe25c37f2bb9e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.798628 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3060822021431584a494e6941be756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.798656 4637 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.799652 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122b7898fea44ef78f269b1505d8ddc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.799806 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc5448a555944dfba8ccae09e25cbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.799808 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 (DEFAULT_TABLE table=default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.801436 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26851245cab4a2d80e94d4fe278fd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.801967 4163 ts_tablet_manager.cc:1428] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 1.419s user 0.000s sys 0.002s
I20260811 18:57:12.804090 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:12.810135 4163 ts_tablet_manager.cc:1460] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Tablet startup took 1433ms
W20260811 18:57:12.810418 4163 ts_tablet_manager.cc:1462] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trace:
0811 18:57:03.114182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.114276 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:03.115152 (+ 876us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:03.115179 (+ 27us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:03.156195 (+ 41016us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.176303 (+ 20108us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:03.176459 (+ 156us) inbound_call.cc:173] Queueing success response
0811 18:57:08.305182 (+5128723us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0811 18:57:11.369027 (+3063845us) ts_tablet_manager.cc:1429] Starting tablet replica
0811 18:57:11.369305 (+ 278us) tablet_replica.cc:259] Starting instrumentation
0811 18:57:11.369513 (+ 208us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":270,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"tablet-open.queue_time_us":5128756}
I20260811 18:57:12.811581 4163 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:12.811898 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 166) took 6503 ms. Trace:
I20260811 18:57:12.812115 4169 rpcz_store.cc:276] 0811 18:57:06.285266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.286322 (+ 1056us) service_pool.cc:227] Handling call
0811 18:57:12.774285 (+6487963us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:12.774295 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.788159 (+ 13864us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6485272}
I20260811 18:57:12.814651 4162 tablet_bootstrap.cc:654] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:12.822791 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 167) took 6491 ms. Trace:
I20260811 18:57:12.824594 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 221d0d8d69f142a29ec7065a82b3f769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.823091 4169 rpcz_store.cc:276] 0811 18:57:06.321194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.321258 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:12.812444 (+6491186us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:12.812459 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.812714 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6490620}
I20260811 18:57:12.832489 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43ad6993b63e4bbd98c8cc52f1a53451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.809335 4660 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.833508 4658 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:12.843942 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbf6ce167fb348be8d61372791f0cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.846686 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.849061 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.849776 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 179) took 6064 ms. Trace:
I20260811 18:57:12.847469 4658 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.853428 3892 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:12.857013 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.857899 4169 rpcz_store.cc:276] 0811 18:57:06.769668 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.791769 (+ 22101us) service_pool.cc:227] Handling call
0811 18:57:12.826488 (+6034719us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:12.826532 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.834014 (+ 7482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6034216}
I20260811 18:57:12.858383 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe203387bc9c45ba82afe6b058aa5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:12.864652 3676 leader_election.cc:341] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:12.865103 3676 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.865450 4556 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:57:12.865607 4556 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
W20260811 18:57:12.865545 3676 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:12.865734 4556 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Advancing to term 2
W20260811 18:57:12.866108 3676 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8efb663060ef457cbece962fce701eb1
I20260811 18:57:12.870564 4556 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.872483 4556 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.863276 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.875278 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e902efbb774244d18e3ba165bfb7c695. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:12.862075 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 70817.5us
I20260811 18:57:12.878609 4167 ts_tablet_manager.cc:1918] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:12.878840 4660 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:12.879171 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 180) took 6066 ms. Trace:
I20260811 18:57:12.879318 4169 rpcz_store.cc:276] 0811 18:57:06.797118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.800501 (+ 3383us) service_pool.cc:227] Handling call
0811 18:57:12.858224 (+6057723us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:12.858237 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.863866 (+ 5629us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6056725}
I20260811 18:57:12.880057 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 181) took 6027 ms. Trace:
I20260811 18:57:12.880174 4169 rpcz_store.cc:276] 0811 18:57:06.852109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.852232 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:12.879706 (+6027474us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:12.879721 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.879948 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6027108}
I20260811 18:57:12.880594 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 187) took 5713 ms. Trace:
I20260811 18:57:12.880769 4169 rpcz_store.cc:276] 0811 18:57:07.167102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.167217 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:12.880421 (+5713204us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:12.880432 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.880519 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4571098}
I20260811 18:57:12.881422 4169 tablet_replica.cc:330] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:12.882900 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 1.05457e+06us
W20260811 18:57:12.884316 3676 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:12.886624 4167 log.cc:1196] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/494c09652d9e408880ca606ae0b30773
I20260811 18:57:12.892618 4167 ts_tablet_manager.cc:1939] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:12.893275 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2718cd4a98742dfa8014a8a38c69c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.893891 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 141) took 7122 ms. Trace:
I20260811 18:57:12.894747 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6852aaef1d5486b87e5462eae532719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.896159 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:12.894084 4167 rpcz_store.cc:276] 0811 18:57:05.771018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.771538 (+ 520us) service_pool.cc:227] Handling call
0811 18:57:11.371452 (+5599914us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:11.371467 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.802176 (+1430709us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.878439 (+ 76263us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.893813 (+ 15374us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":295795,"tablet-delete.queue_time_us":5581187}
I20260811 18:57:12.899655 4162 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:12.902207 4162 ts_tablet_manager.cc:1397] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.117s user 0.008s sys 0.003s
I20260811 18:57:12.902238 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:12.902508 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.902722 3890 raft_consensus.cc:3037] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:12.903749 4162 raft_consensus.cc:348] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.903996 4162 raft_consensus.cc:374] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:12.904718 4556 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:12.904968 4556 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.908274 3824 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.874951 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.910493 3889 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:12.914321 4162 raft_consensus.cc:729] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:13.119899 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3106ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:12.915189 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:12.915545 4162 consensus_queue.cc:260] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.914685 4167 tablet_replica.cc:330] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.137910 4162 ts_tablet_manager.cc:1428] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.236s user 0.004s sys 0.000s
I20260811 18:57:12.914844 3676 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.914624 4556 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.920097 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:12.920986 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.139981 4167 raft_consensus.cc:2229] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:12.924568 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:12.924937 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.140413 4167 raft_consensus.cc:2258] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:12.925601 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:12.998378 4651 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 15995523294368684499) tablet 00000000000000000000000000000000 cur holder 0x617000072a80
I20260811 18:57:12.925302 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
W20260811 18:57:13.141040 3539 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 351204us
I20260811 18:57:13.141392 4167 ts_tablet_manager.cc:1905] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:13.141444 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
I20260811 18:57:12.956321 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:13.141772 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
I20260811 18:57:13.141904 3740 raft_consensus.cc:2379] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
W20260811 18:57:13.141992 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
W20260811 18:57:13.138245 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 223688us
W20260811 18:57:13.142411 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
W20260811 18:57:13.142656 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
W20260811 18:57:13.142899 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
W20260811 18:57:13.143124 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
W20260811 18:57:13.143317 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
W20260811 18:57:13.143565 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 08b8dcf7a56043efbbd40bb888616492 because the tablet was not found. No further retry: Not found: Tablet not found: 08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:12.912178 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:13.137239 3744 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:12.925493 3552 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 298) took 4233 ms. Trace:
I20260811 18:57:13.145462 4556 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:57:13.145980 4556 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:13.146155 4556 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.145514 3552 rpcz_store.cc:276] 0811 18:57:08.557320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.670771 (+3113451us) service_pool.cc:227] Handling call
0811 18:57:11.671210 (+ 439us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.671496 (+ 286us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:11.671740 (+ 244us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:11.671812 (+ 72us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:11.695381 (+ 23569us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d70cc2 00007f61f7d54693 00007f61f7f426e9 00007f61ef235de6
0811 18:57:11.951283 (+255902us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:12.791017 (+839734us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.697919 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.698017 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.698023 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:11.698027 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.698120 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.698130 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.698136 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:11.698182 (+ 46us) tablet.cc:699] Row locks acquired
0811 18:57:11.698184 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:11.698208 (+ 24us) write_op.cc:270] Start()
0811 18:57:11.698229 (+ 21us) write_op.cc:276] Timestamp: P: 1786474631698170 usec, L: 0
0811 18:57:11.698232 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.699121 (+ 889us) log.cc:842] Serialized 405 byte log entry
0811 18:57:11.701390 (+ 2269us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f03677cf 00007f61f0366c7e 00007f61f0417e45 00007f61f0418cd9 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c
0811 18:57:11.948867 (+247477us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.950327 (+ 1460us) write_op.cc:301] APPLY: starting
0811 18:57:11.950364 (+ 37us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.950479 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.950485 (+ 6us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.950564 (+ 79us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.950615 (+ 51us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.950627 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:11.950757 (+ 130us) log.cc:842] Serialized 23 byte log entry
0811 18:57:11.951118 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951144 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:11.951156 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.951237 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951242 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":52106624,"child_traces":[["op",{"apply.queue_time_us":1355,"apply.run_cpu_time_us":723,"apply.run_wall_time_us":995,"num_ops":1,"prepare.queue_time_us":2478,"prepare.run_cpu_time_us":1028,"prepare.run_wall_time_us":3512,"replication_time_us":250589,"spinlock_wait_cycles":4404992}]]}
I20260811 18:57:13.148312 3552 catalog_manager.cc:5495] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.148912 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.149518 4556 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.917147 3888 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.150130 4556 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:13.152194 3676 leader_election.cc:341] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.153234 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:13.153486 3891 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.153848 3888 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 2.
I20260811 18:57:13.154006 4592 raft_consensus.cc:2741] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260811 18:57:12.930791 3554 rpcz_store.cc:276] 0811 18:57:07.225481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.651484 (+4426003us) service_pool.cc:227] Handling call
0811 18:57:11.653080 (+ 1596us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.653741 (+ 661us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.930446 (+276705us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.930616 (+ 170us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.930752 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.694910 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.695166 (+ 256us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.695177 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:11.695182 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:11.695324 (+ 142us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.695342 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.695350 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.695476 (+ 126us) tablet.cc:699] Row locks acquired
0811 18:57:11.695480 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:11.695534 (+ 54us) write_op.cc:270] Start()
0811 18:57:11.695583 (+ 49us) write_op.cc:276] Timestamp: P: 1786474631695488 usec, L: 0
0811 18:57:11.695590 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.695964 (+ 374us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.915620 (+219656us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.927947 (+ 12327us) write_op.cc:301] APPLY: starting
0811 18:57:11.927976 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.928148 (+ 172us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.928306 (+ 158us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.928672 (+ 366us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.928756 (+ 84us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.928768 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:11.929047 (+ 279us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.929928 (+ 881us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.930083 (+ 155us) write_op.cc:454] Released schema lock
0811 18:57:11.930097 (+ 14us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.930563 (+ 466us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.930635 (+ 72us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12198,"apply.run_cpu_time_us":2083,"apply.run_wall_time_us":38127,"num_ops":2,"prepare.queue_time_us":96,"prepare.run_cpu_time_us":1176,"prepare.run_wall_time_us":1177,"replication_time_us":219982}]]}
I20260811 18:57:12.932365 3547 rpcz_store.cc:276] 0811 18:57:07.453306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.661671 (+4208365us) service_pool.cc:227] Handling call
0811 18:57:11.662760 (+ 1089us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.689596 (+ 26836us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.697715 (+ 8119us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d4024d 00007f61f7f4046a 00007f61ef232466 00007f61ef232195
0811 18:57:11.958242 (+260527us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.958408 (+ 166us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.958538 (+ 130us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.730323 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.730941 (+ 618us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.730952 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:11.731009 (+ 57us) tablet.cc:660] Decoding operations
0811 18:57:11.731595 (+ 586us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.731668 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.731739 (+ 71us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.732259 (+ 520us) tablet.cc:699] Row locks acquired
0811 18:57:11.732264 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.732485 (+ 221us) write_op.cc:270] Start()
0811 18:57:11.732647 (+ 162us) write_op.cc:276] Timestamp: P: 1786474631732377 usec, L: 0
0811 18:57:11.732707 (+ 60us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.734554 (+ 1847us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.949490 (+214936us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.952026 (+ 2536us) write_op.cc:301] APPLY: starting
0811 18:57:11.952039 (+ 13us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.952077 (+ 38us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.952130 (+ 53us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.952286 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.952324 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.952334 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:11.952399 (+ 65us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.952920 (+ 521us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.953004 (+ 84us) write_op.cc:454] Released schema lock
0811 18:57:11.953075 (+ 71us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.954267 (+ 1192us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.954277 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":10548992,"child_traces":[["op",{"apply.queue_time_us":2444,"apply.run_cpu_time_us":970,"apply.run_wall_time_us":2543,"num_ops":2,"prepare.queue_time_us":32357,"prepare.run_cpu_time_us":2588,"prepare.run_wall_time_us":5337,"replication_time_us":216704}]]}
I20260811 18:57:13.155179 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d" } new_table_name: "default.Table_34c1bb6042d54870bb236a2204ce71ed" modify_external_catalogs: true
I20260811 18:57:13.155412 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_666e839163c3494a9c95e78ff76bb015" } new_table_name: "default.Table_fce143fe6d0849a0a3af8939df804986" modify_external_catalogs: true
I20260811 18:57:12.934023 3553 rpcz_store.cc:276] 0811 18:57:07.326971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.656289 (+4329318us) service_pool.cc:227] Handling call
0811 18:57:11.657385 (+ 1096us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.657947 (+ 562us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:12.002962 (+345015us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:12.004356 (+ 1394us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:12.004499 (+ 143us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.697102 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.697270 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.697278 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:11.697282 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.697432 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.697443 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.697450 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.697542 (+ 92us) tablet.cc:699] Row locks acquired
0811 18:57:11.697547 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.697581 (+ 34us) write_op.cc:270] Start()
0811 18:57:11.697619 (+ 38us) write_op.cc:276] Timestamp: P: 1786474631697539 usec, L: 0
0811 18:57:11.697624 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.697816 (+ 192us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.915797 (+217981us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.935105 (+ 19308us) write_op.cc:301] APPLY: starting
0811 18:57:11.935185 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.935416 (+ 231us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.935577 (+ 161us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.940932 (+ 5355us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.942990 (+ 2058us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.972971 (+ 29981us) write_op.cc:312] APPLY: finished
0811 18:57:11.982130 (+ 9159us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.998703 (+ 16573us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.999025 (+ 322us) write_op.cc:454] Released schema lock
0811 18:57:11.999193 (+ 168us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:12.003570 (+ 4377us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:12.004387 (+ 817us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19064,"apply.run_cpu_time_us":3835,"apply.run_wall_time_us":74508,"num_ops":2,"prepare.queue_time_us":1955,"prepare.run_cpu_time_us":773,"prepare.run_wall_time_us":772,"replication_time_us":218154,"thread_start_us":32908,"threads_started":1}]]}
I20260811 18:57:13.156795 3891 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 2.
I20260811 18:57:12.923978 4658 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.157152 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f" } modify_external_catalogs: true
I20260811 18:57:13.157549 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f" } modify_external_catalogs: true
I20260811 18:57:13.157661 4162 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:12.931778 3872 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 217) took 1195 ms. Trace:
I20260811 18:57:12.935269 3854 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 202) took 5554 ms. Trace:
I20260811 18:57:12.937847 3863 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 214) took 1560 ms. Trace:
I20260811 18:57:13.157913 3863 rpcz_store.cc:276] 0811 18:57:11.372370 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.372427 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:11.373085 (+ 658us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.373112 (+ 27us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.788968 (+1415856us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.932308 (+143340us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.932473 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.967528 4163 tablet_bootstrap.cc:654] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.159142 3676 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.980262 3890 raft_consensus.cc:2454] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:12.935139 3555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 34) took 1110 ms (client timeout 1000 ms). Trace:
I20260811 18:57:12.989931 3851 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 205) took 5548 ms. Trace:
I20260811 18:57:13.157846 3854 rpcz_store.cc:276] 0811 18:57:07.359923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.360052 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:08.344516 (+984464us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.344539 (+ 23us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.787033 (+4442494us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.914429 (+127396us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.914679 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.159490 3851 rpcz_store.cc:276] 0811 18:57:07.440720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.440822 (+ 102us) service_pool.cc:227] Handling call
0811 18:57:08.346383 (+905561us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.346401 (+ 18us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.856364 (+4509963us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.983964 (+127600us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.988820 (+ 4856us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:13.159444 3555 rpcz_store.cc:269] 0811 18:57:11.682920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.793122 (+1110202us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:12.793198 (+ 76us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:13.157826 3872 rpcz_store.cc:276] 0811 18:57:11.726261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.726354 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:12.774457 (+1048103us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:12.774476 (+ 19us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.777896 (+ 3420us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.921324 (+143428us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.921560 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.044679 3848 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 68) took 1159 ms. Trace:
I20260811 18:57:13.159992 3848 rpcz_store.cc:276] 0811 18:57:11.884726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.884802 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:12.777201 (+892399us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x615000193aa4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.783564 (+ 6363us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x6150001610a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.799887 (+ 16323us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.846333 (+ 46446us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.021782 (+175449us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.044495 (+ 22713us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":268899328}
I20260811 18:57:13.160223 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6460e311a6da4deaba4efce4c6dbd883" } new_table_name: "default.Table_1996443a5de84ded8919fb5c51dc6554" modify_external_catalogs: true
I20260811 18:57:13.047983 3871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 211) took 1675 ms. Trace:
I20260811 18:57:13.160516 3871 rpcz_store.cc:276] 0811 18:57:11.371970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.373455 (+ 1485us) service_pool.cc:227] Handling call
0811 18:57:11.723958 (+350503us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.723998 (+ 40us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.021685 (+1297687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.047548 (+ 25863us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.047815 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.051541 3857 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 212) took 1679 ms. Trace:
I20260811 18:57:13.053131 3856 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 215) took 1680 ms. Trace:
I20260811 18:57:13.093641 3847 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 67) took 4533 ms. Trace:
I20260811 18:57:13.160800 3857 rpcz_store.cc:276] 0811 18:57:11.372106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.376316 (+ 4210us) service_pool.cc:227] Handling call
0811 18:57:11.660363 (+284047us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.660394 (+ 31us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.027361 (+1366967us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.051104 (+ 23743us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.051380 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.160828 3847 rpcz_store.cc:276] 0811 18:57:08.556166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.556538 (+ 372us) service_pool.cc:227] Handling call
0811 18:57:11.374499 (+2817961us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x6150001d2524. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.788413 (+1413914us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.852339 (+ 63926us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.920534 (+ 68195us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x6150000a3824. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.992166 (+ 71632us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.089609 (+ 97443us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1375992704}
I20260811 18:57:13.106549 4169 raft_consensus.cc:2229] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:13.161906 4169 raft_consensus.cc:2258] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:12.935541 3548 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 292) took 5632 ms. Trace:
I20260811 18:57:12.938139 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 297) took 4367 ms. Trace:
I20260811 18:57:12.938419 3549 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 290) took 5793 ms. Trace:
I20260811 18:57:13.107273 3850 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 66) took 4538 ms. Trace:
I20260811 18:57:13.162842 3850 rpcz_store.cc:276] 0811 18:57:08.556006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.556231 (+ 225us) service_pool.cc:227] Handling call
0811 18:57:12.774534 (+4218303us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x615000193aa4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.782527 (+ 7993us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x6150001610a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.790853 (+ 8326us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.847058 (+ 56205us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.022077 (+175019us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.094798 (+ 72721us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4644815488}
I20260811 18:57:13.162587 3549 rpcz_store.cc:276] 0811 18:57:07.131470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.485073 (+1353603us) service_pool.cc:227] Handling call
0811 18:57:08.560532 (+ 75459us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f412e6 00007f61ef234126 00007f61ef233e55 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:11.677507 (+3116975us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.677718 (+ 211us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:11.677803 (+ 85us) catalog_manager.cc:2761] Locking tablets
0811 18:57:11.677984 (+ 181us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:11.695827 (+ 17843us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d58d4c 00007f61f7d4dc70 00007f61f7d4c1bf 00007f61f7f41446
0811 18:57:11.951994 (+256167us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:11.952115 (+ 121us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:11.952350 (+ 235us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:12.918958 (+966608us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.701486 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.701595 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.701602 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:11.701606 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.701769 (+ 163us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.701780 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.701787 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.701916 (+ 129us) tablet.cc:699] Row locks acquired
0811 18:57:11.701921 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.701955 (+ 34us) write_op.cc:270] Start()
0811 18:57:11.702001 (+ 46us) write_op.cc:276] Timestamp: P: 1786474631701914 usec, L: 0
0811 18:57:11.702005 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.702166 (+ 161us) log.cc:842] Serialized 725 byte log entry
0811 18:57:11.703865 (+ 1699us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f03677cf 00007f61f0366c7e 00007f61f0417e45 00007f61f0418cd9 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c
0811 18:57:11.948970 (+245105us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.951359 (+ 2389us) write_op.cc:301] APPLY: starting
0811 18:57:11.951373 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.951419 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.951423 (+ 4us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.951490 (+ 67us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.951534 (+ 44us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:11.951544 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:11.951618 (+ 74us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.951877 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951890 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:11.951896 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.951935 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951938 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":208608512,"child_traces":[["op",{"apply.queue_time_us":2348,"apply.run_cpu_time_us":421,"apply.run_wall_time_us":627,"num_ops":2,"prepare.queue_time_us":5585,"prepare.run_cpu_time_us":5480,"prepare.run_wall_time_us":28502,"raft.queue_time_us":9584,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":246945,"spinlock_wait_cycles":2499456,"thread_start_us":25961,"threads_started":1}]]}
I20260811 18:57:13.163357 3856 rpcz_store.cc:276] 0811 18:57:11.372485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.374329 (+ 1844us) service_pool.cc:227] Handling call
0811 18:57:11.659695 (+285366us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.659716 (+ 21us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.986291 (+1326575us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.052865 (+ 66574us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.053078 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.119066 3861 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 201) took 5771 ms. Trace:
I20260811 18:57:13.121114 3860 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 213) took 1748 ms. Trace:
I20260811 18:57:13.167724 3860 rpcz_store.cc:276] 0811 18:57:11.372246 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.376774 (+ 4528us) service_pool.cc:227] Handling call
0811 18:57:11.724237 (+347463us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.724261 (+ 24us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.101735 (+1377474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.120879 (+ 19144us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.121041 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:13.141623 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.110463 3862 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 216) took 1737 ms. Trace:
I20260811 18:57:13.168118 3862 rpcz_store.cc:276] 0811 18:57:11.372614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.374396 (+ 1782us) service_pool.cc:227] Handling call
0811 18:57:11.659540 (+285144us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.659577 (+ 37us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.965609 (+1306032us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.109894 (+144285us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.110131 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.168365 3824 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.163771 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.164542 4169 ts_tablet_manager.cc:1905] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.168823 4658 raft_consensus.cc:2741] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260811 18:57:13.169132 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_511d4c7dbd2b41ae95fc5f910147870d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.169559 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.169863 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.170006 3861 rpcz_store.cc:276] 0811 18:57:07.347057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.347162 (+ 105us) service_pool.cc:227] Handling call
0811 18:57:08.344002 (+996840us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.344028 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.091897 (+4747869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.118720 (+ 26823us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.118874 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.780762 3550 rpcz_store.cc:276] 0811 18:57:07.157109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.712862 (+2555753us) service_pool.cc:227] Handling call
0811 18:57:10.859077 (+1146215us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:10.859839 (+ 762us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.842737 (+982898us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.842905 (+ 168us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.843049 (+ 144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:10.891801 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:10.891966 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:10.891976 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:10.891981 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:10.892110 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:10.892130 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:10.892138 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:10.892254 (+ 116us) tablet.cc:699] Row locks acquired
0811 18:57:10.892258 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:10.892315 (+ 57us) write_op.cc:270] Start()
0811 18:57:10.892361 (+ 46us) write_op.cc:276] Timestamp: P: 1786474630892286 usec, L: 0
0811 18:57:10.892368 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:10.892686 (+ 318us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.700434 (+807748us) spinlock_profiling.cc:243] Waited 938 us on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0363390 00007f61f037aa59 00007f61f037a7c1 00007f61f7ea947d 00007f61f03c68a1 00007f61f03cb5ae 00007f61f03cb331 00007f61f7ea947d 00007f61efe03308 00007f61efdeb9c3
0811 18:57:11.796921 (+ 96487us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.839233 (+ 42312us) write_op.cc:301] APPLY: starting
0811 18:57:11.839266 (+ 33us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.839350 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.839355 (+ 5us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.839590 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.839658 (+ 68us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.839669 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.839852 (+ 183us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.841916 (+ 2064us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.842083 (+ 167us) write_op.cc:454] Released schema lock
0811 18:57:11.842271 (+ 188us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.903045 (+ 60774us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.903056 (+ 11us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42190,"apply.run_cpu_time_us":2034,"apply.run_wall_time_us":63946,"num_ops":2,"prepare.queue_time_us":31162,"prepare.run_cpu_time_us":2541,"prepare.run_wall_time_us":28111,"replication_time_us":904513,"spinlock_wait_cycles":3560960,"thread_start_us":101672,"threads_started":3,"wal-append.queue_time_us":24399}]]}
I20260811 18:57:12.946574 3551 catalog_manager.cc:6565] ReplaceTablet: tablet f61d3822fc2d4b569f223238a6464e0a deleted and replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:13.172159 3551 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 293) took 6014 ms. Trace:
I20260811 18:57:13.172366 3551 rpcz_store.cc:276] 0811 18:57:07.157840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.322305 (+4164465us) service_pool.cc:227] Handling call
0811 18:57:13.172103 (+1849798us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.374038 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.374210 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.374222 (+ 12us) write_op.cc:435] Acquired schema lock
0811 18:57:11.374226 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.374431 (+ 205us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.374447 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.374455 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.374576 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:11.374581 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.374634 (+ 53us) write_op.cc:270] Start()
0811 18:57:11.374681 (+ 47us) write_op.cc:276] Timestamp: P: 1786474631374615 usec, L: 0
0811 18:57:11.374686 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.374991 (+ 305us) log.cc:842] Serialized 622 byte log entry
0811 18:57:11.883917 (+508926us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.903236 (+ 19319us) write_op.cc:301] APPLY: starting
0811 18:57:11.903259 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.903309 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.903472 (+ 163us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.903625 (+ 153us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.903702 (+ 77us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.903713 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.903820 (+ 107us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.904716 (+ 896us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.904806 (+ 90us) write_op.cc:454] Released schema lock
0811 18:57:11.904878 (+ 72us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.905352 (+ 474us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.905359 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19250,"apply.run_cpu_time_us":1310,"apply.run_wall_time_us":2442,"num_ops":2,"prepare.queue_time_us":3352,"prepare.run_cpu_time_us":1114,"prepare.run_wall_time_us":1114,"replication_time_us":509206,"thread_start_us":31504,"threads_started":1}]]}
I20260811 18:57:13.173753 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c87170e0a947438da8d0091a78e401c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.162463 3546 rpcz_store.cc:276] 0811 18:57:08.553033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.662250 (+3109217us) service_pool.cc:227] Handling call
0811 18:57:11.662858 (+ 608us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.663101 (+ 243us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:11.663170 (+ 69us) catalog_manager.cc:2761] Locking tablets
0811 18:57:11.663361 (+ 191us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:11.695041 (+ 31680us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d58d4c 00007f61f7d4dc70 00007f61f7d4c1bf 00007f61f7f41446
0811 18:57:11.934199 (+239158us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:11.934322 (+ 123us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:11.934400 (+ 78us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:12.920176 (+985776us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.696137 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.696269 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.696279 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:11.696283 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.696420 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.696434 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.696442 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.696688 (+ 246us) tablet.cc:699] Row locks acquired
0811 18:57:11.696693 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.696748 (+ 55us) write_op.cc:270] Start()
0811 18:57:11.696787 (+ 39us) write_op.cc:276] Timestamp: P: 1786474631696702 usec, L: 0
0811 18:57:11.696793 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.696998 (+ 205us) log.cc:842] Serialized 756 byte log entry
0811 18:57:11.915749 (+218751us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.932566 (+ 16817us) write_op.cc:301] APPLY: starting
0811 18:57:11.932585 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.932644 (+ 59us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.932723 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.932824 (+ 101us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.932870 (+ 46us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:11.932881 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.932984 (+ 103us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.933785 (+ 801us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.933869 (+ 84us) write_op.cc:454] Released schema lock
0811 18:57:11.933940 (+ 71us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.934257 (+ 317us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.934323 (+ 66us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":70439168,"child_traces":[["op",{"apply.queue_time_us":16772,"apply.run_cpu_time_us":1142,"apply.run_wall_time_us":2031,"num_ops":2,"prepare.queue_time_us":1023,"prepare.run_cpu_time_us":926,"prepare.run_wall_time_us":925,"replication_time_us":218936}]]}
I20260811 18:57:13.162505 3548 rpcz_store.cc:276] 0811 18:57:07.157709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:10.865282 (+3707573us) service_pool.cc:227] Handling call
0811 18:57:10.865753 (+ 471us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:10.866086 (+ 333us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:10.868398 (+ 2312us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:10.868536 (+ 138us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:11.932469 (+1063933us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:12.790318 (+857849us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:10.919971 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:10.920148 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:10.920156 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:10.920161 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:10.920314 (+ 153us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:10.920333 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:10.920341 (+ 8us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:10.920433 (+ 92us) tablet.cc:699] Row locks acquired
0811 18:57:10.920437 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:10.920489 (+ 52us) write_op.cc:270] Start()
0811 18:57:10.920536 (+ 47us) write_op.cc:276] Timestamp: P: 1786474630920461 usec, L: 0
0811 18:57:10.920543 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:10.920827 (+ 284us) log.cc:842] Serialized 405 byte log entry
0811 18:57:11.840848 (+920021us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.879192 (+ 38344us) write_op.cc:301] APPLY: starting
0811 18:57:11.879215 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.879449 (+ 234us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.880107 (+ 658us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.881283 (+ 1176us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.881591 (+ 308us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.881668 (+ 77us) write_op.cc:312] APPLY: finished
0811 18:57:11.882942 (+ 1274us) log.cc:842] Serialized 23 byte log entry
0811 18:57:11.884740 (+ 1798us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.932325 (+ 47585us) write_op.cc:454] Released schema lock
0811 18:57:11.932338 (+ 13us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.932436 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.932442 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":38225,"apply.run_cpu_time_us":2248,"apply.run_wall_time_us":53335,"num_ops":1,"prepare.queue_time_us":50871,"prepare.run_cpu_time_us":945,"prepare.run_wall_time_us":944,"replication_time_us":920270,"thread_start_us":42902,"threads_started":1}]]}
I20260811 18:57:13.174809 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c33809efd2a943a3bfd700f8cf0750c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.175370 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dae19af473234626ac121fc618e92a4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.175590 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c33809efd2a943a3bfd700f8cf0750c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:13.175956 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dae19af473234626ac121fc618e92a4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:13.178987 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c87170e0a947438da8d0091a78e401c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.181710 3940 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:13.185855 3849 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 65) took 4630 ms. Trace:
I20260811 18:57:13.189114 4637 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:13.190274 4556 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.191359 4556 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.198628 3676 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.186162 3849 rpcz_store.cc:276] 0811 18:57:08.555773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.555866 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:13.159914 (+4604048us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x6150001629a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.185700 (+ 25786us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":586254080}
I20260811 18:57:13.203586 3676 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.203666 4556 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.204286 4556 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:13.205647 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:13.206919 4576 raft_consensus.cc:2781] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.207011 4556 raft_consensus.cc:686] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.207978 4169 ts_tablet_manager.cc:1918] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.208544 4637 raft_consensus.cc:686] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.208475 4556 consensus_queue.cc:237] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.209064 4637 consensus_queue.cc:237] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.218993 4576 raft_consensus.cc:686] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.219574 4576 consensus_queue.cc:237] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:13.228225 3676 leader_election.cc:341] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.235982 4169 log.cc:1196] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d106e0e1f4b845429b10f4e100f69a8b
I20260811 18:57:13.222106 4163 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.237460 4163 ts_tablet_manager.cc:1397] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.426s user 0.011s sys 0.011s
I20260811 18:57:13.239217 4162 tablet_bootstrap.cc:654] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.239246 4163 raft_consensus.cc:348] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.239605 4163 raft_consensus.cc:374] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.239866 4163 raft_consensus.cc:729] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.251436 4163 consensus_queue.cc:260] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.256218 4163 ts_tablet_manager.cc:1428] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260811 18:57:13.260279 4163 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.264389 4163 tablet_bootstrap.cc:654] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.254071 4169 ts_tablet_manager.cc:1939] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.284416 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 186) took 6119 ms. Trace:
I20260811 18:57:13.295426 4169 rpcz_store.cc:276] 0811 18:57:07.164788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.167248 (+ 2460us) service_pool.cc:227] Handling call
0811 18:57:12.881170 (+5713922us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:12.881181 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.161591 (+280410us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x6150000c9024. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:13.187833 (+ 26242us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.207729 (+ 19896us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.284338 (+ 76609us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":501723520,"tablet-delete.queue_time_us":4571817}
I20260811 18:57:13.295974 4167 ts_tablet_manager.cc:1918] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.296280 4167 log.cc:1196] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:13.296358 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d106e0e1f4b845429b10f4e100f69a8b (table default.Table_8e4b85a9b7d5460b83544a3677cde81a [id=17b08a7402a34713a7329c297b221da9]) successfully deleted
I20260811 18:57:13.296916 4169 tablet_replica.cc:330] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.304411 4169 raft_consensus.cc:2229] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.304880 4169 raft_consensus.cc:2258] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.305879 4169 ts_tablet_manager.cc:1905] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.306319 4162 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.306720 4162 ts_tablet_manager.cc:1397] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.149s user 0.007s sys 0.005s
I20260811 18:57:13.308143 4162 raft_consensus.cc:348] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.308810 4162 raft_consensus.cc:374] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.308997 4162 raft_consensus.cc:729] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.309716 4162 consensus_queue.cc:260] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.314314 4167 ts_tablet_manager.cc:1939] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.315169 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 143) took 7453 ms. Trace:
I20260811 18:57:13.315562 4167 rpcz_store.cc:276] 0811 18:57:05.861871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.863597 (+ 1726us) service_pool.cc:227] Handling call
0811 18:57:12.906106 (+7042509us) ts_tablet_manager.cc:1145] Deleting tablet 5df7a07983174dd1aaf48951c9ff1a1b
0811 18:57:12.906122 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.154080 (+247958us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.295658 (+141578us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.315117 (+ 19459us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2345,"tablet-delete.queue_time_us":7042212}
I20260811 18:57:13.316628 4167 tablet_replica.cc:330] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.316905 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5df7a07983174dd1aaf48951c9ff1a1b (table default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]) successfully deleted
I20260811 18:57:13.317132 4167 raft_consensus.cc:2229] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.317684 4167 raft_consensus.cc:2258] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.318830 4167 ts_tablet_manager.cc:1905] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.333734 4163 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.334079 4163 ts_tablet_manager.cc:1397] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.003s
I20260811 18:57:13.335685 4163 raft_consensus.cc:348] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.336220 4163 raft_consensus.cc:374] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.336484 4163 raft_consensus.cc:729] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.336901 4163 consensus_queue.cc:260] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.334156 4162 ts_tablet_manager.cc:1428] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:13.346987 4163 ts_tablet_manager.cc:1428] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.013s user 0.000s sys 0.003s
I20260811 18:57:13.349975 4163 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.351442 4162 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.366493 4162 tablet_bootstrap.cc:654] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.362299 4169 ts_tablet_manager.cc:1918] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.393146 4169 log.cc:1196] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2852f9e5f67c4025836018af553388a5
I20260811 18:57:13.395973 4169 ts_tablet_manager.cc:1939] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.401141 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9292c8eb06fb4a0d8e33b6796398b740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.402561 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 188) took 6148 ms. Trace:
W20260811 18:57:13.404186 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9292c8eb06fb4a0d8e33b6796398b740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.402694 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
I20260811 18:57:13.404874 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:13.405205 4169 rpcz_store.cc:276] 0811 18:57:07.247876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.247995 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:13.296745 (+6048750us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:13.296763 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.328880 (+ 32117us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.362133 (+ 33253us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.396659 (+ 34526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4987213}
I20260811 18:57:13.402882 4163 tablet_bootstrap.cc:654] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.406821 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2852f9e5f67c4025836018af553388a5 (table default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]) successfully deleted
I20260811 18:57:13.407227 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.411258 4169 tablet_replica.cc:330] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.411299 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.411551 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84" } modify_external_catalogs: true
I20260811 18:57:13.411708 4169 raft_consensus.cc:2229] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.411860 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84" } modify_external_catalogs: true
I20260811 18:57:13.413919 4169 raft_consensus.cc:2258] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.414942 4169 ts_tablet_manager.cc:1905] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.423571 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.427713 4658 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.428050 4162 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.428022 4658 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.428258 4162 ts_tablet_manager.cc:1397] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.001s
I20260811 18:57:13.428889 4658 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.429888 4162 raft_consensus.cc:348] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.430065 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.430460 4162 raft_consensus.cc:374] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.430572 3744 raft_consensus.cc:2454] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:13.433139 4162 raft_consensus.cc:729] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.433792 3824 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.434479 4658 raft_consensus.cc:2781] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:13.434748 4658 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:13.435081 4658 raft_consensus.cc:3037] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.435075 4162 consensus_queue.cc:260] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.440433 4658 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.440974 4443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43838 (request call id 0) took 4835 ms. Trace:
I20260811 18:57:13.442373 4658 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.442437 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:13.443089 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:13.443938 4655 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:13.444655 4655 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:57:13.447786 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 799ba9a044e145ec89d0025c03f391ea consensus state has no leader
I20260811 18:57:13.448135 3554 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 299) took 4887 ms. Trace:
I20260811 18:57:13.448308 3554 rpcz_store.cc:276] 0811 18:57:08.560504 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.154949 (+4594445us) service_pool.cc:227] Handling call
0811 18:57:13.155417 (+ 468us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.155665 (+ 248us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.155887 (+ 222us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.155979 (+ 92us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.447369 (+291390us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.447998 (+ 629us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.229301 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.229429 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.229436 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.229439 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.229559 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.229575 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.229580 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.229661 (+ 81us) tablet.cc:699] Row locks acquired
0811 18:57:13.229664 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.229706 (+ 42us) write_op.cc:270] Start()
0811 18:57:13.229743 (+ 37us) write_op.cc:276] Timestamp: P: 1786474633229673 usec, L: 0
0811 18:57:13.229746 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.230007 (+ 261us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.444257 (+214250us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.445328 (+ 1071us) write_op.cc:301] APPLY: starting
0811 18:57:13.445350 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.445395 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.445423 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.445485 (+ 62us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.445522 (+ 37us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.445528 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.445625 (+ 97us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.447145 (+ 1520us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.447167 (+ 22us) write_op.cc:454] Released schema lock
0811 18:57:13.447175 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":996,"num_ops":1,"prepare.queue_time_us":72678,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":784,"replication_time_us":214476}]]}
W20260811 18:57:13.452656 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:13.452972 3547 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 300) took 4892 ms. Trace:
I20260811 18:57:13.453269 3547 rpcz_store.cc:276] 0811 18:57:08.560750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.154998 (+4594248us) service_pool.cc:227] Handling call
0811 18:57:13.155724 (+ 726us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.155936 (+ 212us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.156113 (+ 177us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.156226 (+ 113us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.452111 (+295885us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.452841 (+ 730us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.230132 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.230265 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.230273 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.230278 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.240800 (+ 10522us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.240823 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.240834 (+ 11us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.241001 (+ 167us) tablet.cc:699] Row locks acquired
0811 18:57:13.241006 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:13.241071 (+ 65us) write_op.cc:270] Start()
0811 18:57:13.241126 (+ 55us) write_op.cc:276] Timestamp: P: 1786474633241029 usec, L: 0
0811 18:57:13.241133 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.241497 (+ 364us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.444353 (+202856us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.450723 (+ 6370us) write_op.cc:301] APPLY: starting
0811 18:57:13.450742 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.450787 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.450830 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.450895 (+ 65us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.450939 (+ 44us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.450949 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:13.451035 (+ 86us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.451950 (+ 915us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.451971 (+ 21us) write_op.cc:454] Released schema lock
0811 18:57:13.451981 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.452067 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.452073 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6321,"apply.run_cpu_time_us":590,"apply.run_wall_time_us":1421,"num_ops":1,"prepare.queue_time_us":73414,"prepare.run_cpu_time_us":1168,"prepare.run_wall_time_us":11469,"replication_time_us":203191,"threads_started":1}]]}
I20260811 18:57:13.454771 4655 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:13.442518 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:13.457726 3744 raft_consensus.cc:3037] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.460044 4162 ts_tablet_manager.cc:1428] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.030s user 0.002s sys 0.002s
I20260811 18:57:13.432768 4167 ts_tablet_manager.cc:1918] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.461547 4167 log.cc:1196] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8efb663060ef457cbece962fce701eb1
I20260811 18:57:13.462180 4162 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.466747 3744 raft_consensus.cc:2454] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:13.469419 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
W20260811 18:57:13.470714 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 125757b8c1a6448b841c63339f392339 consensus state has no leader
I20260811 18:57:13.470942 3555 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 302) took 4784 ms. Trace:
I20260811 18:57:13.471446 3549 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 303) took 3554 ms. Trace:
I20260811 18:57:13.471719 3549 rpcz_store.cc:276] 0811 18:57:09.916809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.163536 (+3246727us) service_pool.cc:227] Handling call
0811 18:57:13.169303 (+ 5767us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.169963 (+ 660us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.470876 (+300913us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.471003 (+ 127us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.471158 (+ 155us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.243636 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.243717 (+ 81us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.243722 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:13.243725 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.243809 (+ 84us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.243817 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.243821 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.243878 (+ 57us) tablet.cc:699] Row locks acquired
0811 18:57:13.243882 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.243909 (+ 27us) write_op.cc:270] Start()
0811 18:57:13.243932 (+ 23us) write_op.cc:276] Timestamp: P: 1786474633243879 usec, L: 0
0811 18:57:13.243935 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.244068 (+ 133us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.467514 (+223446us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.469818 (+ 2304us) write_op.cc:301] APPLY: starting
0811 18:57:13.469833 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.469870 (+ 37us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.469932 (+ 62us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.470356 (+ 424us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.470399 (+ 43us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.470409 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:13.470484 (+ 75us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.470742 (+ 258us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.470763 (+ 21us) write_op.cc:454] Released schema lock
0811 18:57:13.470772 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.470838 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.470844 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2267,"apply.run_cpu_time_us":669,"apply.run_wall_time_us":1076,"num_ops":2,"prepare.queue_time_us":73034,"prepare.run_cpu_time_us":491,"prepare.run_wall_time_us":489,"replication_time_us":223560,"threads_started":1}]]}
I20260811 18:57:13.472764 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.472906 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.470248 3553 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 301) took 4909 ms. Trace:
I20260811 18:57:13.473605 3553 rpcz_store.cc:276] 0811 18:57:08.560905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.156843 (+4595938us) service_pool.cc:227] Handling call
0811 18:57:13.157735 (+ 892us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.157898 (+ 163us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.157957 (+ 59us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.158232 (+ 275us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:13.469065 (+310833us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:13.469170 (+ 105us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:13.469241 (+ 71us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:13.470040 (+ 799us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.241668 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.241889 (+ 221us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.241899 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:13.241904 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.242061 (+ 157us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.242077 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.242084 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.242489 (+ 405us) tablet.cc:699] Row locks acquired
0811 18:57:13.242495 (+ 6us) write_op.cc:260] PREPARE: finished
0811 18:57:13.242545 (+ 50us) write_op.cc:270] Start()
0811 18:57:13.242587 (+ 42us) write_op.cc:276] Timestamp: P: 1786474633242507 usec, L: 0
0811 18:57:13.242593 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.242886 (+ 293us) log.cc:842] Serialized 725 byte log entry
0811 18:57:13.467364 (+224478us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.467868 (+ 504us) write_op.cc:301] APPLY: starting
0811 18:57:13.467891 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.467944 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.468002 (+ 58us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.468108 (+ 106us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.468157 (+ 49us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:13.468168 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.468264 (+ 96us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.468595 (+ 331us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.468621 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:13.468632 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.468706 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.468712 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":423,"apply.run_cpu_time_us":674,"apply.run_wall_time_us":919,"num_ops":2,"prepare.queue_time_us":82465,"prepare.run_cpu_time_us":1101,"prepare.run_wall_time_us":1288,"replication_time_us":224735}]]}
I20260811 18:57:13.471139 3555 rpcz_store.cc:276] 0811 18:57:08.685984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.159850 (+4473866us) service_pool.cc:227] Handling call
0811 18:57:13.160428 (+ 578us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.160638 (+ 210us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.160841 (+ 203us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.160924 (+ 83us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.470449 (+309525us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.470872 (+ 423us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.242992 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.243128 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.243133 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:13.243136 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.243225 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.243233 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.243238 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.243292 (+ 54us) tablet.cc:699] Row locks acquired
0811 18:57:13.243295 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.243324 (+ 29us) write_op.cc:270] Start()
0811 18:57:13.243350 (+ 26us) write_op.cc:276] Timestamp: P: 1786474633243292 usec, L: 0
0811 18:57:13.243356 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.243491 (+ 135us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.467465 (+223974us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.468839 (+ 1374us) write_op.cc:301] APPLY: starting
0811 18:57:13.468855 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.468894 (+ 39us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.468925 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.469065 (+ 140us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.469120 (+ 55us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.469131 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.469210 (+ 79us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.469628 (+ 418us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.469647 (+ 19us) write_op.cc:454] Released schema lock
0811 18:57:13.469656 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.469717 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.469723 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1323,"apply.run_cpu_time_us":535,"apply.run_wall_time_us":939,"num_ops":1,"prepare.queue_time_us":81490,"prepare.run_cpu_time_us":550,"prepare.run_wall_time_us":549,"replication_time_us":224090}]]}
I20260811 18:57:13.475342 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet e5fd9fce095e4a769dbb38c7903359cc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.476998 3824 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.477720 4658 raft_consensus.cc:2781] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.478263 4658 raft_consensus.cc:686] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:13.478652 4658 consensus_queue.cc:237] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.482007 4162 tablet_bootstrap.cc:654] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.484774 4167 ts_tablet_manager.cc:1939] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.500510 4163 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.500862 4163 ts_tablet_manager.cc:1397] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.151s user 0.008s sys 0.001s
I20260811 18:57:13.502545 4163 raft_consensus.cc:348] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.503026 4163 raft_consensus.cc:374] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.503279 4163 raft_consensus.cc:729] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.504020 4163 consensus_queue.cc:260] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.504935 4163 ts_tablet_manager.cc:1428] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:13.505388 4163 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.509908 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 144) took 7627 ms. Trace:
I20260811 18:57:13.519702 3546 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 305) took 1637 ms. Trace:
I20260811 18:57:13.520025 3546 rpcz_store.cc:276] 0811 18:57:11.881979 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.174483 (+1292504us) service_pool.cc:227] Handling call
0811 18:57:13.175679 (+ 1196us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.176284 (+ 605us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.519130 (+342846us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.519331 (+ 201us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.519458 (+ 127us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.244170 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.244343 (+ 173us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.244351 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.244356 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.244450 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.244461 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.244468 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.244554 (+ 86us) tablet.cc:699] Row locks acquired
0811 18:57:13.244558 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.244589 (+ 31us) write_op.cc:270] Start()
0811 18:57:13.244618 (+ 29us) write_op.cc:276] Timestamp: P: 1786474633244557 usec, L: 0
0811 18:57:13.244624 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.244799 (+ 175us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.513302 (+268503us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.513607 (+ 305us) write_op.cc:301] APPLY: starting
0811 18:57:13.513640 (+ 33us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.513718 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.513775 (+ 57us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.514041 (+ 266us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.514086 (+ 45us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.514092 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.514272 (+ 180us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.518001 (+ 3729us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.518053 (+ 52us) write_op.cc:454] Released schema lock
0811 18:57:13.518067 (+ 14us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.518188 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.518194 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":227,"apply.run_cpu_time_us":1106,"apply.run_wall_time_us":4674,"num_ops":2,"prepare.queue_time_us":67181,"prepare.run_cpu_time_us":685,"prepare.run_wall_time_us":683,"replication_time_us":268633}]]}
I20260811 18:57:13.521090 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 405773849fb94800be61ad8a7a6055a4 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.521818 3548 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 306) took 1639 ms. Trace:
I20260811 18:57:13.522035 3548 rpcz_store.cc:276] 0811 18:57:11.882163 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.175053 (+1292890us) service_pool.cc:227] Handling call
0811 18:57:13.176049 (+ 996us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.176616 (+ 567us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.521346 (+344730us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.521493 (+ 147us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.521616 (+ 123us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.244902 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.245009 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.245017 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.245021 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.245113 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.245120 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.245124 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.245197 (+ 73us) tablet.cc:699] Row locks acquired
0811 18:57:13.245202 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:13.245230 (+ 28us) write_op.cc:270] Start()
0811 18:57:13.245259 (+ 29us) write_op.cc:276] Timestamp: P: 1786474633245199 usec, L: 0
0811 18:57:13.245262 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.245418 (+ 156us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.513472 (+268054us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.518372 (+ 4900us) write_op.cc:301] APPLY: starting
0811 18:57:13.518392 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.518454 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.518555 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.518730 (+ 175us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.518787 (+ 57us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.518798 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.518918 (+ 120us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.519459 (+ 541us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.519486 (+ 27us) write_op.cc:454] Released schema lock
0811 18:57:13.519497 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.519738 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.519744 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":9472,"child_traces":[["op",{"apply.queue_time_us":4808,"apply.run_cpu_time_us":884,"apply.run_wall_time_us":1478,"num_ops":2,"prepare.queue_time_us":67882,"prepare.run_cpu_time_us":579,"prepare.run_wall_time_us":577,"replication_time_us":268139}]]}
I20260811 18:57:13.523504 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 125757b8c1a6448b841c63339f392339 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.531648 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]). Expected version 2 got 0
I20260811 18:57:13.532048 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.533983 3551 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 304) took 1652 ms. Trace:
I20260811 18:57:13.534425 3551 rpcz_store.cc:276] 0811 18:57:11.881785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.173409 (+1291624us) service_pool.cc:227] Handling call
0811 18:57:13.179112 (+ 5703us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.179564 (+ 452us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.533493 (+353929us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.533679 (+ 186us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.533834 (+ 155us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.245535 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.245652 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.245660 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.245664 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.245778 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.245790 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.245797 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.245888 (+ 91us) tablet.cc:699] Row locks acquired
0811 18:57:13.245892 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.245926 (+ 34us) write_op.cc:270] Start()
0811 18:57:13.245957 (+ 31us) write_op.cc:276] Timestamp: P: 1786474633245892 usec, L: 0
0811 18:57:13.245963 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.246217 (+ 254us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.519917 (+273700us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.520019 (+ 102us) write_op.cc:301] APPLY: starting
0811 18:57:13.520036 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.520099 (+ 63us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.520146 (+ 47us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.520359 (+ 213us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.520408 (+ 49us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.520419 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.520522 (+ 103us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.521558 (+ 1036us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.521583 (+ 25us) write_op.cc:454] Released schema lock
0811 18:57:13.521593 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.521681 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.521687 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48,"apply.run_cpu_time_us":806,"apply.run_wall_time_us":1727,"num_ops":2,"prepare.queue_time_us":65321,"prepare.run_cpu_time_us":749,"prepare.run_wall_time_us":748,"replication_time_us":273929}]]}
I20260811 18:57:13.535766 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet a0ba169c32ef48b984e649b211ee2af1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.537520 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5332d093ac81423fa33cb860f7d9a460" } new_table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" modify_external_catalogs: true
I20260811 18:57:13.541165 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3392d9de7234297a490a976dfe10060" } modify_external_catalogs: true
I20260811 18:57:13.541610 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3392d9de7234297a490a976dfe10060" } modify_external_catalogs: true
I20260811 18:57:13.542629 4658 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.542693 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_46eb50be61c143f2930a20efd2cd56b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.542945 4658 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:13.543783 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46eb50be61c143f2930a20efd2cd56b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.544108 3552 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 307) took 1661 ms. Trace:
I20260811 18:57:13.544691 4658 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.544409 3552 rpcz_store.cc:276] 0811 18:57:11.882761 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.400448 (+1517687us) service_pool.cc:227] Handling call
0811 18:57:13.407237 (+ 6789us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.408069 (+ 832us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.543785 (+135716us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.543908 (+ 123us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.544024 (+ 116us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.408936 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.409092 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.409102 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:13.409106 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.409291 (+ 185us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.409308 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.409315 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.409434 (+ 119us) tablet.cc:699] Row locks acquired
0811 18:57:13.409438 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.409492 (+ 54us) write_op.cc:270] Start()
0811 18:57:13.409537 (+ 45us) write_op.cc:276] Timestamp: P: 1786474633409452 usec, L: 0
0811 18:57:13.409544 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.409818 (+ 274us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.543011 (+133193us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.543109 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:13.543125 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.543197 (+ 72us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.543269 (+ 72us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.543427 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.543463 (+ 36us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.543469 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.543551 (+ 82us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.543656 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.543680 (+ 24us) write_op.cc:454] Released schema lock
0811 18:57:13.543692 (+ 12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42,"num_ops":2,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":991,"prepare.run_wall_time_us":991,"replication_time_us":133439}]]}
I20260811 18:57:13.545043 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:13.547976 3824 leader_election.cc:341] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.510160 4167 rpcz_store.cc:276] 0811 18:57:05.865078 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.865240 (+ 162us) service_pool.cc:227] Handling call
0811 18:57:13.316414 (+7451174us) ts_tablet_manager.cc:1145] Deleting tablet 8efb663060ef457cbece962fce701eb1
0811 18:57:13.316430 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.374156 (+ 57726us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.432625 (+ 58469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.489672 (+ 57047us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7450872}
I20260811 18:57:13.548826 4167 tablet_replica.cc:330] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.549222 4167 raft_consensus.cc:2229] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.549636 4167 raft_consensus.cc:2258] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.550640 4167 ts_tablet_manager.cc:1905] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.550598 4592 consensus_queue.cc:1035] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:13.551260 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8efb663060ef457cbece962fce701eb1 (table default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]) successfully deleted
I20260811 18:57:13.556663 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 490f34d7a403416fbab64e46973746de
I20260811 18:57:13.557101 3796 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:13.557536 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 308) took 1667 ms. Trace:
I20260811 18:57:13.557893 3550 rpcz_store.cc:276] 0811 18:57:11.889647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.411120 (+1521473us) service_pool.cc:227] Handling call
0811 18:57:13.412038 (+ 918us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.412257 (+ 219us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.412326 (+ 69us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.412555 (+ 229us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:13.556323 (+143768us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:13.556443 (+ 120us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:13.556504 (+ 61us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:13.557380 (+ 876us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.413741 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.413894 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.413901 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.413914 (+ 13us) tablet.cc:660] Decoding operations
0811 18:57:13.414073 (+ 159us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.414086 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.414091 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.414275 (+ 184us) tablet.cc:699] Row locks acquired
0811 18:57:13.414278 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.414324 (+ 46us) write_op.cc:270] Start()
0811 18:57:13.414995 (+ 671us) write_op.cc:276] Timestamp: P: 1786474633414287 usec, L: 0
0811 18:57:13.415000 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.415231 (+ 231us) log.cc:842] Serialized 725 byte log entry
0811 18:57:13.555485 (+140254us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.555637 (+ 152us) write_op.cc:301] APPLY: starting
0811 18:57:13.555654 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.555709 (+ 55us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.555794 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.555884 (+ 90us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.555930 (+ 46us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:13.555936 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.556063 (+ 127us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.556177 (+ 114us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.556197 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:13.556205 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"num_ops":2,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":986,"prepare.run_wall_time_us":1572,"replication_time_us":140448}]]}
I20260811 18:57:13.561574 4163 tablet_bootstrap.cc:654] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.564535 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.564441 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.582893 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 125757b8c1a6448b841c63339f392339
I20260811 18:57:13.583564 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 125757b8c1a6448b841c63339f392339 deleted and replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:13.584197 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_77c2e3d67a494888be8b3405d4a06cf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.585074 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77c2e3d67a494888be8b3405d4a06cf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.587833 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.589478 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cec1a44bf54f48019648c9338236fcc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.590325 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cec1a44bf54f48019648c9338236fcc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.595247 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.602370 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:13.603245 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]). Expected version 2 got 0
I20260811 18:57:13.603835 3551 catalog_manager.cc:6565] ReplaceTablet: tablet a0ba169c32ef48b984e649b211ee2af1 deleted and replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:13.605499 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba169c32ef48b984e649b211ee2af1 with delete_type TABLET_DATA_DELETED (replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.608748 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba169c32ef48b984e649b211ee2af1 with delete_type TABLET_DATA_DELETED (replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.603698 3547 catalog_manager.cc:5495] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 1 to 2, leader changed from 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67) to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 2 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.609588 3547 catalog_manager.cc:5495] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 1 to 2, leader changed from 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67) to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 2 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.610446 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:13.610895 3547 catalog_manager.cc:5495] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.611500 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
I20260811 18:57:13.619586 4704 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:13.621838 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:13.622689 3555 catalog_manager.cc:6565] ReplaceTablet: tablet e5fd9fce095e4a769dbb38c7903359cc deleted and replaced by tablet 1b73a97b7abf4bd58c792426be6e938d
I20260811 18:57:13.623844 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_57bac5d4e666472fbab036524cbbed7d" } new_table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" modify_external_catalogs: true
I20260811 18:57:13.624265 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.626677 4162 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.627148 4162 ts_tablet_manager.cc:1397] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.165s user 0.007s sys 0.004s
I20260811 18:57:13.628805 4162 raft_consensus.cc:348] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.629249 4162 raft_consensus.cc:374] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.629488 4163 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.629776 4162 raft_consensus.cc:729] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.629835 4163 ts_tablet_manager.cc:1397] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.000s sys 0.008s
I20260811 18:57:13.630508 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.631237 4163 raft_consensus.cc:348] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.630592 4162 consensus_queue.cc:260] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.631654 4163 raft_consensus.cc:374] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.631801 4163 raft_consensus.cc:729] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.632349 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea" } modify_external_catalogs: true
I20260811 18:57:13.632244 4163 consensus_queue.cc:260] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.632975 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_575c5aca059048d68252cca6583a1f67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.636691 4162 ts_tablet_manager.cc:1428] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260811 18:57:13.642937 4162 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.636406 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44def5389adf4d9ca825712dfb862206" } modify_external_catalogs: true
I20260811 18:57:13.645272 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44def5389adf4d9ca825712dfb862206" } modify_external_catalogs: true
I20260811 18:57:13.647336 4658 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.647696 4658 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.648703 4658 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.650171 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.650646 3744 raft_consensus.cc:2454] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:13.636658 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea" } modify_external_catalogs: true
I20260811 18:57:13.651857 3824 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.652382 4658 raft_consensus.cc:2781] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:13.652523 4658 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:13.653267 4658 raft_consensus.cc:3037] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.656355 4162 tablet_bootstrap.cc:654] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:13.634315 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_575c5aca059048d68252cca6583a1f67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.662536 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4dfa50c8631f4b70b6b743837d5a9032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.663880 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dfa50c8631f4b70b6b743837d5a9032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.664041 4163 ts_tablet_manager.cc:1428] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:13.664716 4163 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.670598 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet ef393c341bb54baf87176db5b5254ec7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.672142 4167 ts_tablet_manager.cc:1918] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.672119 4169 ts_tablet_manager.cc:1918] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.672313 4167 log.cc:1196] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:13.672375 4169 log.cc:1196] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:13.672914 4167 ts_tablet_manager.cc:1939] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.673187 4162 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.673220 4169 ts_tablet_manager.cc:1939] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.673483 4162 ts_tablet_manager.cc:1397] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.005s
I20260811 18:57:13.675022 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 145) took 7809 ms. Trace:
I20260811 18:57:13.677207 4167 rpcz_store.cc:276] 0811 18:57:05.865275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.865355 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:13.548531 (+7683176us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:13.548546 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.576546 (+ 28000us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.671908 (+ 95362us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.674980 (+ 3072us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7682834}
I20260811 18:57:13.678050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 159) took 7410 ms. Trace:
I20260811 18:57:13.678269 4167 rpcz_store.cc:276] 0811 18:57:06.267916 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.268001 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:13.677784 (+7409783us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:13.677798 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.677970 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7409312}
W20260811 18:57:13.679384 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 01d564ad80064912a8ce8e31c1b6c4d2 consensus state has no leader
I20260811 18:57:13.679390 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 160) took 7360 ms. Trace:
I20260811 18:57:13.679661 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 08b8dcf7a56043efbbd40bb888616492 (table default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]) successfully deleted
I20260811 18:57:13.679666 4167 rpcz_store.cc:276] 0811 18:57:06.319248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.319896 (+ 648us) service_pool.cc:227] Handling call
0811 18:57:13.679218 (+7359322us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:13.679233 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.679354 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7358386}
W20260811 18:57:13.680050 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
W20260811 18:57:13.680400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
I20260811 18:57:13.680464 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 172) took 6898 ms. Trace:
I20260811 18:57:13.680650 4167 rpcz_store.cc:276] 0811 18:57:06.781802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.781879 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:13.680224 (+6898345us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:13.680236 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.680389 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6897169}
W20260811 18:57:13.681234 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
I20260811 18:57:13.681700 4420 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" instance_seqno: 1786474620633424) as {username='slave'} at 127.3.72.66:50257; Asking this server to re-register.
I20260811 18:57:13.682138 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 173) took 6882 ms. Trace:
I20260811 18:57:13.682439 4167 rpcz_store.cc:276] 0811 18:57:06.799430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.799507 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:13.681565 (+6882058us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:13.681577 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.681785 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6881396}
W20260811 18:57:13.682999 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
I20260811 18:57:13.683467 3940 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:13.683337 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" } new_table_name: "default.Table_0d93e11fb1464fcbb4999abaebc3d611" modify_external_catalogs: true
I20260811 18:57:13.686295 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 174) took 6838 ms. Trace:
I20260811 18:57:13.686599 4167 rpcz_store.cc:276] 0811 18:57:06.845125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.845191 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:13.683538 (+6838347us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:13.683550 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.683665 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6837618}
W20260811 18:57:13.687600 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
I20260811 18:57:13.688077 4167 tablet_replica.cc:330] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.688932 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 189) took 6426 ms. Trace:
I20260811 18:57:13.689318 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.689314 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_335e9fdd802644cc93a0657a33e74605" } new_table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" modify_external_catalogs: true
I20260811 18:57:13.689364 4169 rpcz_store.cc:276] 0811 18:57:07.249545 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.249659 (+ 114us) service_pool.cc:227] Handling call
0811 18:57:13.411008 (+6161349us) ts_tablet_manager.cc:1145] Deleting tablet 8af3e74889b64c408c9ed9c54916f890
0811 18:57:13.411021 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.424749 (+ 13728us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.671900 (+247151us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.675778 (+ 3878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5101430}
I20260811 18:57:13.688529 4167 raft_consensus.cc:2229] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.689908 4169 tablet_replica.cc:330] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.690052 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8af3e74889b64c408c9ed9c54916f890 (table default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]) successfully deleted
I20260811 18:57:13.690294 4167 raft_consensus.cc:2258] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.690447 4169 raft_consensus.cc:2229] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:13.690913 4169 raft_consensus.cc:2258] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.691676 4167 ts_tablet_manager.cc:1905] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.692245 4169 ts_tablet_manager.cc:1905] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.698552 4423 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" instance_seqno: 1786474620110993) as {username='slave'} at 127.3.72.65:40455; Asking this server to re-register.
I20260811 18:57:13.701712 3796 heartbeater.cc:464] Registering TS with master...
W20260811 18:57:13.705026 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b3ec38f9b55489ba00ed76f9e4456f5 consensus state has no leader
I20260811 18:57:13.700306 4162 raft_consensus.cc:348] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.716173 4162 raft_consensus.cc:374] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.718685 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:13.716786 4162 raft_consensus.cc:729] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.719659 4162 consensus_queue.cc:260] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.720508 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.721724 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 867259e37fe44c33a35823ec2d896754
I20260811 18:57:13.725807 4162 ts_tablet_manager.cc:1428] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.035s user 0.000s sys 0.004s
I20260811 18:57:13.726478 4162 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.729589 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.729983 4162 tablet_bootstrap.cc:654] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.730568 4420 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:13.732438 4423 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.738698 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0859cafbc76b4eae9fe4453842a95178"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.741366 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a671c5d0d75c49779b94baa2b89c9cef" } new_table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" modify_external_catalogs: true
I20260811 18:57:13.739760 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" } modify_external_catalogs: true
I20260811 18:57:13.746007 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" } modify_external_catalogs: true
W20260811 18:57:13.740597 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0859cafbc76b4eae9fe4453842a95178 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.757792 4719 tablet.cc:1722] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:13.769804 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b7f48893a8614cacb6393af837f050f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.775899 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7f48893a8614cacb6393af837f050f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.790066 4658 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:13.793139 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe203387bc9c45ba82afe6b058aa5e5f consensus state has no leader
W20260811 18:57:13.797595 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1240ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:13.799332 4658 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.802728 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:13.803891 4162 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.805228 4556 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.805662 3744 raft_consensus.cc:3037] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.806118 4556 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.806757 4162 ts_tablet_manager.cc:1397] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.078s user 0.003s sys 0.006s
I20260811 18:57:13.807161 4720 tablet.cc:1722] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:13.808480 4556 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:13.812098 3676 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:13.812846 4162 raft_consensus.cc:348] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.817711 4163 tablet_bootstrap.cc:654] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.819516 4162 raft_consensus.cc:374] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.819762 4162 raft_consensus.cc:729] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.820611 4162 consensus_queue.cc:260] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.823699 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 278 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
W20260811 18:57:13.824391 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9bc5448a555944dfba8ccae09e25cbe9 consensus state has no leader
I20260811 18:57:13.825238 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_efa46b4a159b4805b21635ef65051022"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.826292 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efa46b4a159b4805b21635ef65051022 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.830107 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_00f2f8bec82f44f3a42420818d7dc6a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.831875 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00f2f8bec82f44f3a42420818d7dc6a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.844980 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:13.846293 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e75a55498d4c4072968ee03aaf48b831"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.847009 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e75a55498d4c4072968ee03aaf48b831 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.846819 3550 catalog_manager.cc:5495] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:13.847594 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.849684 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (DEFAULT_TABLE table=default.Table_dae19af473234626ac121fc618e92a4b [id=afadeb62ce0d4ceda9714b891c134737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.849937 3861 tablet_service.cc:1467] Processing CreateTablet for tablet b3f0cdcee681460084662e01152ad9dd (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.849756 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 (DEFAULT_TABLE table=default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.852821 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.852972 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.853420 4162 ts_tablet_manager.cc:1428] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.002s sys 0.002s
I20260811 18:57:13.853982 3722 tablet_service.cc:1467] Processing CreateTablet for tablet b3f0cdcee681460084662e01152ad9dd (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.855419 4162 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.859035 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 (DEFAULT_TABLE table=default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.860132 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (DEFAULT_TABLE table=default.Table_dae19af473234626ac121fc618e92a4b [id=afadeb62ce0d4ceda9714b891c134737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.860832 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.862090 4162 tablet_bootstrap.cc:654] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.886907 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 278 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:13.888021 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:13.888797 4655 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:13.889751 3744 raft_consensus.cc:2454] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:13.890894 4167 ts_tablet_manager.cc:1918] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.891240 4167 log.cc:1196] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d106e0e1f4b845429b10f4e100f69a8b
I20260811 18:57:13.893381 4167 ts_tablet_manager.cc:1939] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.895357 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f0cdcee681460084662e01152ad9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.895815 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebe3fed5aaf45918dbadfde06b2c24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.896189 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c07150e31994bdf9a00ac2e7a954691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.907356 4169 ts_tablet_manager.cc:1918] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.911188 3555 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1b73a97b7abf4bd58c792426be6e938d
I20260811 18:57:13.913022 3824 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:13.913395 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cbf6ce167fb348be8d61372791f0cbca consensus state has no leader
I20260811 18:57:13.913662 4658 raft_consensus.cc:2781] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.935737 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_267303174d744009a1803a363c1e431f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.936950 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_267303174d744009a1803a363c1e431f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.939234 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_13fbac538d16421e921aa37f54683662"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.940448 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13fbac538d16421e921aa37f54683662 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.943337 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e851b9ff216483bb8768c88e4b6e1b9" } modify_external_catalogs: true
I20260811 18:57:13.943702 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e851b9ff216483bb8768c88e4b6e1b9" } modify_external_catalogs: true
I20260811 18:57:13.951709 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:13.953792 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 9bc5448a555944dfba8ccae09e25cbe9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.954331 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.956473 4658 raft_consensus.cc:686] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:13.957080 4658 consensus_queue.cc:237] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.967084 4162 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.967983 4162 ts_tablet_manager.cc:1397] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.113s user 0.001s sys 0.009s
I20260811 18:57:13.969866 4169 log.cc:1196] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
I20260811 18:57:13.971570 4169 ts_tablet_manager.cc:1939] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.976089 4163 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.976431 4163 ts_tablet_manager.cc:1397] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.312s user 0.001s sys 0.009s
I20260811 18:57:13.982795 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f0cdcee681460084662e01152ad9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.987346 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebe3fed5aaf45918dbadfde06b2c24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.990610 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c07150e31994bdf9a00ac2e7a954691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.995344 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 175) took 7028 ms. Trace:
I20260811 18:57:13.995604 4167 rpcz_store.cc:276] 0811 18:57:06.966713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.967008 (+ 295us) service_pool.cc:227] Handling call
0811 18:57:13.687964 (+6720956us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:13.687978 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.802712 (+114734us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.890718 (+ 88006us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.995120 (+104402us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6720508}
I20260811 18:57:13.996711 4167 tablet_replica.cc:330] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.997069 4167 raft_consensus.cc:2229] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.997534 4167 raft_consensus.cc:2258] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.997790 3891 raft_consensus.cc:1261] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 18:57:13.998579 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d106e0e1f4b845429b10f4e100f69a8b (table default.Table_8e4b85a9b7d5460b83544a3677cde81a [id=17b08a7402a34713a7329c297b221da9]) successfully deleted
I20260811 18:57:13.998974 4556 consensus_queue.cc:1035] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 18:57:14.001370 4163 raft_consensus.cc:348] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.004766 4167 ts_tablet_manager.cc:1905] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.441311 4443 rpcz_store.cc:276] 0811 18:57:08.606364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.606645 (+ 281us) service_pool.cc:227] Handling call
0811 18:57:11.328738 (+2722093us) raft_consensus.cc:1457] Updating replica for 264 ops
0811 18:57:11.659409 (+330671us) raft_consensus.cc:1520] Early marking committed up to index 17
0811 18:57:11.827079 (+167670us) raft_consensus.cc:1525] Triggering prepare for 261 ops
0811 18:57:11.827658 (+ 579us) write_op.cc:270] Start()
0811 18:57:11.827748 (+ 90us) write_op.cc:276] Timestamp: P: 1786474621520868 usec, L: 0
0811 18:57:11.918883 (+ 91135us) write_op.cc:270] Start()
0811 18:57:11.918934 (+ 51us) write_op.cc:276] Timestamp: P: 1786474621522053 usec, L: 0
0811 18:57:11.919202 (+ 268us) write_op.cc:270] Start()
0811 18:57:11.919229 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621523450 usec, L: 0
0811 18:57:11.919381 (+ 152us) write_op.cc:270] Start()
0811 18:57:11.919403 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621524997 usec, L: 0
0811 18:57:11.919587 (+ 184us) write_op.cc:270] Start()
0811 18:57:11.919612 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621527610 usec, L: 0
0811 18:57:11.919872 (+ 260us) write_op.cc:270] Start()
0811 18:57:11.919899 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621528508 usec, L: 0
0811 18:57:11.920054 (+ 155us) write_op.cc:270] Start()
0811 18:57:11.920109 (+ 55us) write_op.cc:276] Timestamp: P: 1786474621529384 usec, L: 0
0811 18:57:11.920352 (+ 243us) write_op.cc:270] Start()
0811 18:57:11.920381 (+ 29us) write_op.cc:276] Timestamp: P: 1786474621544629 usec, L: 0
0811 18:57:11.920595 (+ 214us) write_op.cc:270] Start()
0811 18:57:11.920618 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621651974 usec, L: 0
0811 18:57:11.920865 (+ 247us) write_op.cc:270] Start()
0811 18:57:11.920892 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621652817 usec, L: 0
0811 18:57:11.921122 (+ 230us) write_op.cc:270] Start()
0811 18:57:11.921150 (+ 28us) write_op.cc:276] Timestamp: P: 1786474621676972 usec, L: 0
0811 18:57:11.921319 (+ 169us) write_op.cc:270] Start()
0811 18:57:11.921365 (+ 46us) write_op.cc:276] Timestamp: P: 1786474621677939 usec, L: 0
0811 18:57:11.921532 (+ 167us) write_op.cc:270] Start()
0811 18:57:11.921556 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621678607 usec, L: 0
0811 18:57:11.921780 (+ 224us) write_op.cc:270] Start()
0811 18:57:11.921807 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621711143 usec, L: 0
0811 18:57:11.922019 (+ 212us) write_op.cc:270] Start()
0811 18:57:11.922044 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621727651 usec, L: 0
0811 18:57:11.922235 (+ 191us) write_op.cc:270] Start()
0811 18:57:11.922260 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621731810 usec, L: 0
0811 18:57:11.990413 (+ 68153us) write_op.cc:270] Start()
0811 18:57:11.990519 (+ 106us) write_op.cc:276] Timestamp: P: 1786474621748443 usec, L: 0
0811 18:57:11.990902 (+ 383us) write_op.cc:270] Start()
0811 18:57:11.990939 (+ 37us) write_op.cc:276] Timestamp: P: 1786474621751639 usec, L: 0
0811 18:57:11.991142 (+ 203us) write_op.cc:270] Start()
0811 18:57:11.991166 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621774538 usec, L: 0
0811 18:57:11.991325 (+ 159us) write_op.cc:270] Start()
0811 18:57:11.991349 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621788092 usec, L: 0
0811 18:57:11.991494 (+ 145us) write_op.cc:270] Start()
0811 18:57:11.991517 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621792386 usec, L: 0
0811 18:57:11.991664 (+ 147us) write_op.cc:270] Start()
0811 18:57:11.991686 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621814381 usec, L: 0
0811 18:57:11.991828 (+ 142us) write_op.cc:270] Start()
0811 18:57:11.991853 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621818940 usec, L: 0
0811 18:57:11.991990 (+ 137us) write_op.cc:270] Start()
0811 18:57:11.992012 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621820953 usec, L: 0
0811 18:57:11.992186 (+ 174us) write_op.cc:270] Start()
0811 18:57:11.992209 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621824136 usec, L: 0
0811 18:57:11.992384 (+ 175us) write_op.cc:270] Start()
0811 18:57:11.992415 (+ 31us) write_op.cc:276] Timestamp: P: 1786474621826027 usec, L: 0
0811 18:57:11.992572 (+ 157us) write_op.cc:270] Start()
0811 18:57:11.992596 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621847361 usec, L: 0
0811 18:57:11.992735 (+ 139us) write_op.cc:270] Start()
0811 18:57:11.992752 (+ 17us) write_op.cc:276] Timestamp: P: 1786474621848770 usec, L: 0
0811 18:57:11.992908 (+ 156us) write_op.cc:270] Start()
0811 18:57:11.992944 (+ 36us) write_op.cc:276] Timestamp: P: 1786474621852710 usec, L: 0
0811 18:57:11.993098 (+ 154us) write_op.cc:270] Start()
0811 18:57:11.993122 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621853943 usec, L: 0
0811 18:57:11.993280 (+ 158us) write_op.cc:270] Start()
0811 18:57:11.993303 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621874157 usec, L: 0
0811 18:57:11.993468 (+ 165us) write_op.cc:270] Start()
0811 18:57:11.993492 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621904587 usec, L: 0
0811 18:57:11.993666 (+ 174us) write_op.cc:270] Start()
0811 18:57:11.993689 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621908927 usec, L: 0
0811 18:57:11.993826 (+ 137us) write_op.cc:270] Start()
0811 18:57:11.993851 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621909657 usec, L: 0
0811 18:57:11.994005 (+ 154us) write_op.cc:270] Start()
0811 18:57:11.994028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622004715 usec, L: 0
0811 18:57:11.994181 (+ 153us) write_op.cc:270] Start()
0811 18:57:11.994207 (+ 26us) write_op.cc:276] Timestamp: P: 1786474622006470 usec, L: 0
0811 18:57:12.056002 (+ 61795us) write_op.cc:270] Start()
0811 18:57:12.059310 (+ 3308us) write_op.cc:276] Timestamp: P: 1786474622045673 usec, L: 0
0811 18:57:12.083380 (+ 24070us) write_op.cc:270] Start()
0811 18:57:12.086523 (+ 3143us) write_op.cc:276] Timestamp: P: 1786474622046391 usec, L: 0
0811 18:57:12.106318 (+ 19795us) write_op.cc:270] Start()
0811 18:57:12.110145 (+ 3827us) write_op.cc:276] Timestamp: P: 1786474622047787 usec, L: 0
0811 18:57:12.134923 (+ 24778us) write_op.cc:270] Start()
0811 18:57:12.140077 (+ 5154us) write_op.cc:276] Timestamp: P: 1786474622048501 usec, L: 0
0811 18:57:12.178586 (+ 38509us) write_op.cc:270] Start()
0811 18:57:12.178658 (+ 72us) write_op.cc:276] Timestamp: P: 1786474622049494 usec, L: 0
0811 18:57:12.179113 (+ 455us) write_op.cc:270] Start()
0811 18:57:12.179147 (+ 34us) write_op.cc:276] Timestamp: P: 1786474622067276 usec, L: 0
0811 18:57:12.179302 (+ 155us) write_op.cc:270] Start()
0811 18:57:12.179321 (+ 19us) write_op.cc:276] Timestamp: P: 1786474622081817 usec, L: 0
0811 18:57:12.179477 (+ 156us) write_op.cc:270] Start()
0811 18:57:12.179505 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622108942 usec, L: 0
0811 18:57:12.179654 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.179679 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622109557 usec, L: 0
0811 18:57:12.188784 (+ 9105us) write_op.cc:270] Start()
0811 18:57:12.189831 (+ 1047us) write_op.cc:276] Timestamp: P: 1786474622112463 usec, L: 0
0811 18:57:12.202322 (+ 12491us) write_op.cc:270] Start()
0811 18:57:12.205552 (+ 3230us) write_op.cc:276] Timestamp: P: 1786474622125688 usec, L: 0
0811 18:57:12.239304 (+ 33752us) write_op.cc:270] Start()
0811 18:57:12.242909 (+ 3605us) write_op.cc:276] Timestamp: P: 1786474622212794 usec, L: 0
0811 18:57:12.267796 (+ 24887us) write_op.cc:270] Start()
0811 18:57:12.271423 (+ 3627us) write_op.cc:276] Timestamp: P: 1786474622214897 usec, L: 0
0811 18:57:12.292568 (+ 21145us) write_op.cc:270] Start()
0811 18:57:12.294008 (+ 1440us) write_op.cc:276] Timestamp: P: 1786474622216185 usec, L: 0
0811 18:57:12.306470 (+ 12462us) write_op.cc:270] Start()
0811 18:57:12.309344 (+ 2874us) write_op.cc:276] Timestamp: P: 1786474622257586 usec, L: 0
0811 18:57:12.325469 (+ 16125us) write_op.cc:270] Start()
0811 18:57:12.326818 (+ 1349us) write_op.cc:276] Timestamp: P: 1786474622289720 usec, L: 0
0811 18:57:12.345826 (+ 19008us) write_op.cc:270] Start()
0811 18:57:12.349294 (+ 3468us) write_op.cc:276] Timestamp: P: 1786474622314198 usec, L: 0
0811 18:57:12.367562 (+ 18268us) write_op.cc:270] Start()
0811 18:57:12.371573 (+ 4011us) write_op.cc:276] Timestamp: P: 1786474622315614 usec, L: 0
0811 18:57:12.408088 (+ 36515us) write_op.cc:270] Start()
0811 18:57:12.412307 (+ 4219us) write_op.cc:276] Timestamp: P: 1786474622318020 usec, L: 0
0811 18:57:12.430256 (+ 17949us) write_op.cc:270] Start()
0811 18:57:12.433204 (+ 2948us) write_op.cc:276] Timestamp: P: 1786474622325784 usec, L: 0
0811 18:57:12.460123 (+ 26919us) write_op.cc:270] Start()
0811 18:57:12.463765 (+ 3642us) write_op.cc:276] Timestamp: P: 1786474622327031 usec, L: 0
0811 18:57:12.503004 (+ 39239us) write_op.cc:270] Start()
0811 18:57:12.507644 (+ 4640us) write_op.cc:276] Timestamp: P: 1786474622328122 usec, L: 0
0811 18:57:12.542953 (+ 35309us) write_op.cc:270] Start()
0811 18:57:12.543014 (+ 61us) write_op.cc:276] Timestamp: P: 1786474622353667 usec, L: 0
0811 18:57:12.543246 (+ 232us) write_op.cc:270] Start()
0811 18:57:12.543307 (+ 61us) write_op.cc:276] Timestamp: P: 1786474622355045 usec, L: 0
0811 18:57:12.543480 (+ 173us) write_op.cc:270] Start()
0811 18:57:12.543507 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622364171 usec, L: 0
0811 18:57:12.543691 (+ 184us) write_op.cc:270] Start()
0811 18:57:12.543719 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622364888 usec, L: 0
0811 18:57:12.543909 (+ 190us) write_op.cc:270] Start()
0811 18:57:12.543937 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622365415 usec, L: 0
0811 18:57:12.544136 (+ 199us) write_op.cc:270] Start()
0811 18:57:12.544163 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622366051 usec, L: 0
0811 18:57:12.544347 (+ 184us) write_op.cc:270] Start()
0811 18:57:12.544373 (+ 26us) write_op.cc:276] Timestamp: P: 1786474622366666 usec, L: 0
0811 18:57:12.544548 (+ 175us) write_op.cc:270] Start()
0811 18:57:12.544576 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622404720 usec, L: 0
0811 18:57:12.544747 (+ 171us) write_op.cc:270] Start()
0811 18:57:12.544775 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622412481 usec, L: 0
0811 18:57:12.544916 (+ 141us) write_op.cc:270] Start()
0811 18:57:12.544939 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622426342 usec, L: 0
0811 18:57:12.545101 (+ 162us) write_op.cc:270] Start()
0811 18:57:12.545126 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622437132 usec, L: 0
0811 18:57:12.545312 (+ 186us) write_op.cc:270] Start()
0811 18:57:12.545340 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622446932 usec, L: 0
0811 18:57:12.545509 (+ 169us) write_op.cc:270] Start()
0811 18:57:12.545537 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622447678 usec, L: 0
0811 18:57:12.545728 (+ 191us) write_op.cc:270] Start()
0811 18:57:12.545756 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622448255 usec, L: 0
0811 18:57:12.545910 (+ 154us) write_op.cc:270] Start()
0811 18:57:12.545937 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622464675 usec, L: 0
0811 18:57:12.546102 (+ 165us) write_op.cc:270] Start()
0811 18:57:12.546131 (+ 29us) write_op.cc:276] Timestamp: P: 1786474622489315 usec, L: 0
0811 18:57:12.546288 (+ 157us) write_op.cc:270] Start()
0811 18:57:12.546312 (+ 24us) write_op.cc:276] Timestamp: P: 1786474622547221 usec, L: 0
0811 18:57:12.630867 (+ 84555us) write_op.cc:270] Start()
0811 18:57:12.630927 (+ 60us) write_op.cc:276] Timestamp: P: 1786474622552886 usec, L: 0
0811 18:57:12.631193 (+ 266us) write_op.cc:270] Start()
0811 18:57:12.631226 (+ 33us) write_op.cc:276] Timestamp: P: 1786474622686514 usec, L: 0
0811 18:57:12.631437 (+ 211us) write_op.cc:270] Start()
0811 18:57:12.631464 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622687469 usec, L: 0
0811 18:57:12.631613 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.631668 (+ 55us) write_op.cc:276] Timestamp: P: 1786474622688362 usec, L: 0
0811 18:57:12.631863 (+ 195us) write_op.cc:270] Start()
0811 18:57:12.631887 (+ 24us) write_op.cc:276] Timestamp: P: 1786474622692612 usec, L: 0
0811 18:57:12.632051 (+ 164us) write_op.cc:270] Start()
0811 18:57:12.632073 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622694298 usec, L: 0
0811 18:57:12.632228 (+ 155us) write_op.cc:270] Start()
0811 18:57:12.632255 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622695082 usec, L: 0
0811 18:57:12.632437 (+ 182us) write_op.cc:270] Start()
0811 18:57:12.632458 (+ 21us) write_op.cc:276] Timestamp: P: 1786474622695781 usec, L: 0
0811 18:57:12.632593 (+ 135us) write_op.cc:270] Start()
0811 18:57:12.632614 (+ 21us) write_op.cc:276] Timestamp: P: 1786474622748237 usec, L: 0
0811 18:57:12.632782 (+ 168us) write_op.cc:270] Start()
0811 18:57:12.632802 (+ 20us) write_op.cc:276] Timestamp: P: 1786474622786588 usec, L: 0
0811 18:57:12.632960 (+ 158us) write_op.cc:270] Start()
0811 18:57:12.632983 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622787350 usec, L: 0
0811 18:57:12.633132 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.633157 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622803836 usec, L: 0
0811 18:57:12.633306 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.633328 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622804502 usec, L: 0
0811 18:57:12.633476 (+ 148us) write_op.cc:270] Start()
0811 18:57:12.633501 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622805136 usec, L: 0
0811 18:57:12.633645 (+ 144us) write_op.cc:270] Start()
0811 18:57:12.633672 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622823222 usec, L: 0
0811 18:57:12.633837 (+ 165us) write_op.cc:270] Start()
0811 18:57:12.633859 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622823974 usec, L: 0
0811 18:57:12.634005 (+ 146us) write_op.cc:270] Start()
0811 18:57:12.634028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622825294 usec, L: 0
0811 18:57:12.634173 (+ 145us) write_op.cc:270] Start()
0811 18:57:12.634193 (+ 20us) write_op.cc:276] Timestamp: P: 1786474622826372 usec, L: 0
0811 18:57:12.672180 (+ 37987us) write_op.cc:270] Start()
0811 18:57:12.672480 (+ 300us) write_op.cc:276] Timestamp: P: 1786474622827042 usec, L: 0
0811 18:57:12.679526 (+ 7046us) write_op.cc:270] Start()
0811 18:57:12.679784 (+ 258us) write_op.cc:276] Timestamp: P: 1786474622829576 usec, L: 0
0811 18:57:12.681637 (+ 1853us) write_op.cc:270] Start()
0811 18:57:12.682790 (+ 1153us) write_op.cc:276] Timestamp: P: 1786474622837820 usec, L: 0
0811 18:57:12.686993 (+ 4203us) write_op.cc:270] Start()
0811 18:57:12.687520 (+ 527us) write_op.cc:276] Timestamp: P: 1786474622838640 usec, L: 0
0811 18:57:12.740282 (+ 52762us) write_op.cc:270] Start()
0811 18:57:12.744388 (+ 4106us) write_op.cc:276] Timestamp: P: 1786474622871381 usec, L: 0
0811 18:57:12.761430 (+ 17042us) write_op.cc:270] Start()
0811 18:57:12.763737 (+ 2307us) write_op.cc:276] Timestamp: P: 1786474622883864 usec, L: 0
0811 18:57:12.818384 (+ 54647us) write_op.cc:270] Start()
0811 18:57:12.818556 (+ 172us) write_op.cc:276] Timestamp: P: 1786474622891993 usec, L: 0
0811 18:57:12.836150 (+ 17594us) write_op.cc:270] Start()
0811 18:57:12.836366 (+ 216us) write_op.cc:276] Timestamp: P: 1786474622911019 usec, L: 0
0811 18:57:12.839841 (+ 3475us) write_op.cc:270] Start()
0811 18:57:12.840824 (+ 983us) write_op.cc:276] Timestamp: P: 1786474622937463 usec, L: 0
0811 18:57:12.848971 (+ 8147us) write_op.cc:270] Start()
0811 18:57:12.852766 (+ 3795us) write_op.cc:276] Timestamp: P: 1786474622949955 usec, L: 0
0811 18:57:12.857047 (+ 4281us) write_op.cc:270] Start()
0811 18:57:12.857603 (+ 556us) write_op.cc:276] Timestamp: P: 1786474622976378 usec, L: 0
0811 18:57:12.877860 (+ 20257us) write_op.cc:270] Start()
0811 18:57:12.878306 (+ 446us) write_op.cc:276] Timestamp: P: 1786474622995113 usec, L: 0
0811 18:57:12.880231 (+ 1925us) write_op.cc:270] Start()
0811 18:57:12.880454 (+ 223us) write_op.cc:276] Timestamp: P: 1786474622998372 usec, L: 0
0811 18:57:12.882900 (+ 2446us) write_op.cc:270] Start()
0811 18:57:12.884419 (+ 1519us) write_op.cc:276] Timestamp: P: 1786474622999173 usec, L: 0
0811 18:57:12.886114 (+ 1695us) write_op.cc:270] Start()
0811 18:57:12.886351 (+ 237us) write_op.cc:276] Timestamp: P: 1786474623043663 usec, L: 0
0811 18:57:12.888347 (+ 1996us) write_op.cc:270] Start()
0811 18:57:12.888715 (+ 368us) write_op.cc:276] Timestamp: P: 1786474623044497 usec, L: 0
0811 18:57:12.889920 (+ 1205us) write_op.cc:270] Start()
0811 18:57:12.890121 (+ 201us) write_op.cc:276] Timestamp: P: 1786474623046123 usec, L: 0
0811 18:57:12.891415 (+ 1294us) write_op.cc:270] Start()
0811 18:57:12.891622 (+ 207us) write_op.cc:276] Timestamp: P: 1786474623048341 usec, L: 0
0811 18:57:12.893060 (+ 1438us) write_op.cc:270] Start()
0811 18:57:12.893090 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623055330 usec, L: 0
0811 18:57:12.899906 (+ 6816us) write_op.cc:270] Start()
0811 18:57:12.899956 (+ 50us) write_op.cc:276] Timestamp: P: 1786474623190642 usec, L: 0
0811 18:57:12.901612 (+ 1656us) write_op.cc:270] Start()
0811 18:57:12.901658 (+ 46us) write_op.cc:276] Timestamp: P: 1786474623235306 usec, L: 0
0811 18:57:12.909259 (+ 7601us) write_op.cc:270] Start()
0811 18:57:12.917736 (+ 8477us) write_op.cc:276] Timestamp: P: 1786474623256077 usec, L: 0
0811 18:57:12.937842 (+ 20106us) write_op.cc:270] Start()
0811 18:57:12.937900 (+ 58us) write_op.cc:276] Timestamp: P: 1786474623257645 usec, L: 0
0811 18:57:12.945400 (+ 7500us) write_op.cc:270] Start()
0811 18:57:12.945467 (+ 67us) write_op.cc:276] Timestamp: P: 1786474623259198 usec, L: 0
0811 18:57:12.952540 (+ 7073us) write_op.cc:270] Start()
0811 18:57:12.952623 (+ 83us) write_op.cc:276] Timestamp: P: 1786474623329019 usec, L: 0
0811 18:57:12.952943 (+ 320us) write_op.cc:270] Start()
0811 18:57:12.952972 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623331218 usec, L: 0
0811 18:57:12.953276 (+ 304us) write_op.cc:270] Start()
0811 18:57:12.953307 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623333719 usec, L: 0
0811 18:57:12.953524 (+ 217us) write_op.cc:270] Start()
0811 18:57:12.953543 (+ 19us) write_op.cc:276] Timestamp: P: 1786474623335405 usec, L: 0
0811 18:57:12.953679 (+ 136us) write_op.cc:270] Start()
0811 18:57:12.953706 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623336724 usec, L: 0
0811 18:57:12.957689 (+ 3983us) write_op.cc:270] Start()
0811 18:57:12.957745 (+ 56us) write_op.cc:276] Timestamp: P: 1786474623338424 usec, L: 0
0811 18:57:12.957987 (+ 242us) write_op.cc:270] Start()
0811 18:57:12.958017 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623343161 usec, L: 0
0811 18:57:12.958530 (+ 513us) write_op.cc:270] Start()
0811 18:57:12.958566 (+ 36us) write_op.cc:276] Timestamp: P: 1786474623347212 usec, L: 0
0811 18:57:12.958857 (+ 291us) write_op.cc:270] Start()
0811 18:57:12.958891 (+ 34us) write_op.cc:276] Timestamp: P: 1786474623349546 usec, L: 0
0811 18:57:12.959174 (+ 283us) write_op.cc:270] Start()
0811 18:57:12.959205 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623422449 usec, L: 0
0811 18:57:12.959522 (+ 317us) write_op.cc:270] Start()
0811 18:57:12.959554 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623430473 usec, L: 0
0811 18:57:12.960031 (+ 477us) write_op.cc:270] Start()
0811 18:57:12.960066 (+ 35us) write_op.cc:276] Timestamp: P: 1786474623432035 usec, L: 0
0811 18:57:12.960290 (+ 224us) write_op.cc:270] Start()
0811 18:57:12.960310 (+ 20us) write_op.cc:276] Timestamp: P: 1786474623435592 usec, L: 0
0811 18:57:12.995410 (+ 35100us) write_op.cc:270] Start()
0811 18:57:12.995610 (+ 200us) write_op.cc:276] Timestamp: P: 1786474623440107 usec, L: 0
0811 18:57:12.996055 (+ 445us) write_op.cc:270] Start()
0811 18:57:12.996107 (+ 52us) write_op.cc:276] Timestamp: P: 1786474623448895 usec, L: 0
0811 18:57:12.998732 (+ 2625us) write_op.cc:270] Start()
0811 18:57:12.998776 (+ 44us) write_op.cc:276] Timestamp: P: 1786474623487172 usec, L: 0
0811 18:57:12.999071 (+ 295us) write_op.cc:270] Start()
0811 18:57:12.999102 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623488870 usec, L: 0
0811 18:57:12.999275 (+ 173us) write_op.cc:270] Start()
0811 18:57:12.999302 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623497373 usec, L: 0
0811 18:57:12.999586 (+ 284us) write_op.cc:270] Start()
0811 18:57:12.999616 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623512725 usec, L: 0
0811 18:57:12.999849 (+ 233us) write_op.cc:270] Start()
0811 18:57:12.999879 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623526487 usec, L: 0
0811 18:57:13.000132 (+ 253us) write_op.cc:270] Start()
0811 18:57:13.000158 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623537613 usec, L: 0
0811 18:57:13.000382 (+ 224us) write_op.cc:270] Start()
0811 18:57:13.000428 (+ 46us) write_op.cc:276] Timestamp: P: 1786474623538331 usec, L: 0
0811 18:57:13.000684 (+ 256us) write_op.cc:270] Start()
0811 18:57:13.000715 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623549647 usec, L: 0
0811 18:57:13.000937 (+ 222us) write_op.cc:270] Start()
0811 18:57:13.000969 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623550546 usec, L: 0
0811 18:57:13.001189 (+ 220us) write_op.cc:270] Start()
0811 18:57:13.001218 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623607852 usec, L: 0
0811 18:57:13.001436 (+ 218us) write_op.cc:270] Start()
0811 18:57:13.001464 (+ 28us) write_op.cc:276] Timestamp: P: 1786474623735005 usec, L: 0
0811 18:57:13.001759 (+ 295us) write_op.cc:270] Start()
0811 18:57:13.001792 (+ 33us) write_op.cc:276] Timestamp: P: 1786474623746020 usec, L: 0
0811 18:57:13.002057 (+ 265us) write_op.cc:270] Start()
0811 18:57:13.002088 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623746835 usec, L: 0
0811 18:57:13.002311 (+ 223us) write_op.cc:270] Start()
0811 18:57:13.002368 (+ 57us) write_op.cc:276] Timestamp: P: 1786474623759665 usec, L: 0
0811 18:57:13.002621 (+ 253us) write_op.cc:270] Start()
0811 18:57:13.002711 (+ 90us) write_op.cc:276] Timestamp: P: 1786474623762433 usec, L: 0
0811 18:57:13.002901 (+ 190us) write_op.cc:270] Start()
0811 18:57:13.002930 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623770621 usec, L: 0
0811 18:57:13.003262 (+ 332us) write_op.cc:270] Start()
0811 18:57:13.003296 (+ 34us) write_op.cc:276] Timestamp: P: 1786474623773272 usec, L: 0
0811 18:57:13.003562 (+ 266us) write_op.cc:270] Start()
0811 18:57:13.003593 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623778446 usec, L: 0
0811 18:57:13.003774 (+ 181us) write_op.cc:270] Start()
0811 18:57:13.003800 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623786685 usec, L: 0
0811 18:57:13.004087 (+ 287us) write_op.cc:270] Start()
0811 18:57:13.004118 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623955702 usec, L: 0
0811 18:57:13.004437 (+ 319us) write_op.cc:270] Start()
0811 18:57:13.004476 (+ 39us) write_op.cc:276] Timestamp: P: 1786474623956496 usec, L: 0
0811 18:57:13.004695 (+ 219us) write_op.cc:270] Start()
0811 18:57:13.004724 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623960711 usec, L: 0
0811 18:57:13.004908 (+ 184us) write_op.cc:270] Start()
0811 18:57:13.004934 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623975063 usec, L: 0
0811 18:57:13.005085 (+ 151us) write_op.cc:270] Start()
0811 18:57:13.005112 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623989664 usec, L: 0
0811 18:57:13.022384 (+ 17272us) write_op.cc:270] Start()
0811 18:57:13.022436 (+ 52us) write_op.cc:276] Timestamp: P: 1786474623992708 usec, L: 0
0811 18:57:13.022684 (+ 248us) write_op.cc:270] Start()
0811 18:57:13.022709 (+ 25us) write_op.cc:276] Timestamp: P: 1786474623994235 usec, L: 0
0811 18:57:13.023014 (+ 305us) write_op.cc:270] Start()
0811 18:57:13.023046 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623997481 usec, L: 0
0811 18:57:13.023189 (+ 143us) write_op.cc:270] Start()
0811 18:57:13.023218 (+ 29us) write_op.cc:276] Timestamp: P: 1786474624059689 usec, L: 0
0811 18:57:13.023402 (+ 184us) write_op.cc:270] Start()
0811 18:57:13.023429 (+ 27us) write_op.cc:276] Timestamp: P: 1786474624243324 usec, L: 0
0811 18:57:13.023581 (+ 152us) write_op.cc:270] Start()
0811 18:57:13.023606 (+ 25us) write_op.cc:276] Timestamp: P: 1786474624256820 usec, L: 0
0811 18:57:13.023895 (+ 289us) write_op.cc:270] Start()
0811 18:57:13.023926 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624352114 usec, L: 0
0811 18:57:13.052012 (+ 28086us) write_op.cc:270] Start()
0811 18:57:13.052075 (+ 63us) write_op.cc:276] Timestamp: P: 1786474624402904 usec, L: 0
0811 18:57:13.052346 (+ 271us) write_op.cc:270] Start()
0811 18:57:13.052371 (+ 25us) write_op.cc:276] Timestamp: P: 1786474624474253 usec, L: 0
0811 18:57:13.052641 (+ 270us) write_op.cc:270] Start()
0811 18:57:13.052678 (+ 37us) write_op.cc:276] Timestamp: P: 1786474624498885 usec, L: 0
0811 18:57:13.052975 (+ 297us) write_op.cc:270] Start()
0811 18:57:13.053089 (+ 114us) write_op.cc:276] Timestamp: P: 1786474624499707 usec, L: 0
0811 18:57:13.053505 (+ 416us) write_op.cc:270] Start()
0811 18:57:13.053540 (+ 35us) write_op.cc:276] Timestamp: P: 1786474624500506 usec, L: 0
0811 18:57:13.053827 (+ 287us) write_op.cc:270] Start()
0811 18:57:13.053858 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624501269 usec, L: 0
0811 18:57:13.054062 (+ 204us) write_op.cc:270] Start()
0811 18:57:13.054095 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624539494 usec, L: 0
0811 18:57:13.054384 (+ 289us) write_op.cc:270] Start()
0811 18:57:13.054482 (+ 98us) write_op.cc:276] Timestamp: P: 1786474624540391 usec, L: 0
0811 18:57:13.054860 (+ 378us) write_op.cc:270] Start()
0811 18:57:13.054893 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624541239 usec, L: 0
0811 18:57:13.055116 (+ 223us) write_op.cc:270] Start()
0811 18:57:13.055149 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624600335 usec, L: 0
0811 18:57:13.055395 (+ 246us) write_op.cc:270] Start()
0811 18:57:13.055427 (+ 32us) write_op.cc:276] Timestamp: P: 1786474624630105 usec, L: 0
0811 18:57:13.055609 (+ 182us) write_op.cc:270] Start()
0811 18:57:13.055638 (+ 29us) write_op.cc:276] Timestamp: P: 1786474624657814 usec, L: 0
0811 18:57:13.055873 (+ 235us) write_op.cc:270] Start()
0811 18:57:13.055904 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624675500 usec, L: 0
0811 18:57:13.056170 (+ 266us) write_op.cc:270] Start()
0811 18:57:13.056208 (+ 38us) write_op.cc:276] Timestamp: P: 1786474624676763 usec, L: 0
0811 18:57:13.056416 (+ 208us) write_op.cc:270] Start()
0811 18:57:13.056447 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624677587 usec, L: 0
0811 18:57:13.056800 (+ 353us) write_op.cc:270] Start()
0811 18:57:13.056837 (+ 37us) write_op.cc:276] Timestamp: P: 1786474624966055 usec, L: 0
0811 18:57:13.057139 (+ 302us) write_op.cc:270] Start()
0811 18:57:13.057172 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624967959 usec, L: 0
0811 18:57:13.057407 (+ 235us) write_op.cc:270] Start()
0811 18:57:13.057441 (+ 34us) write_op.cc:276] Timestamp: P: 1786474624969041 usec, L: 0
0811 18:57:13.057680 (+ 239us) write_op.cc:270] Start()
0811 18:57:13.057711 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625027920 usec, L: 0
0811 18:57:13.058194 (+ 483us) write_op.cc:270] Start()
0811 18:57:13.058233 (+ 39us) write_op.cc:276] Timestamp: P: 1786474625046678 usec, L: 0
0811 18:57:13.058503 (+ 270us) write_op.cc:270] Start()
0811 18:57:13.058538 (+ 35us) write_op.cc:276] Timestamp: P: 1786474625049907 usec, L: 0
0811 18:57:13.058735 (+ 197us) write_op.cc:270] Start()
0811 18:57:13.058878 (+ 143us) write_op.cc:276] Timestamp: P: 1786474625051415 usec, L: 0
0811 18:57:13.059078 (+ 200us) write_op.cc:270] Start()
0811 18:57:13.059129 (+ 51us) write_op.cc:276] Timestamp: P: 1786474625054870 usec, L: 0
0811 18:57:13.059318 (+ 189us) write_op.cc:270] Start()
0811 18:57:13.059350 (+ 32us) write_op.cc:276] Timestamp: P: 1786474625083925 usec, L: 0
0811 18:57:13.059547 (+ 197us) write_op.cc:270] Start()
0811 18:57:13.059576 (+ 29us) write_op.cc:276] Timestamp: P: 1786474625108071 usec, L: 0
0811 18:57:13.059774 (+ 198us) write_op.cc:270] Start()
0811 18:57:13.059805 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625125339 usec, L: 0
0811 18:57:13.060061 (+ 256us) write_op.cc:270] Start()
0811 18:57:13.060092 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625139115 usec, L: 0
0811 18:57:13.060399 (+ 307us) write_op.cc:270] Start()
0811 18:57:13.060434 (+ 35us) write_op.cc:276] Timestamp: P: 1786474625142066 usec, L: 0
0811 18:57:13.060628 (+ 194us) write_op.cc:270] Start()
0811 18:57:13.060658 (+ 30us) write_op.cc:276] Timestamp: P: 1786474625144990 usec, L: 0
0811 18:57:13.060933 (+ 275us) write_op.cc:270] Start()
0811 18:57:13.060966 (+ 33us) write_op.cc:276] Timestamp: P: 1786474625149520 usec, L: 0
0811 18:57:13.061207 (+ 241us) write_op.cc:270] Start()
0811 18:57:13.061238 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625191592 usec, L: 0
0811 18:57:13.061496 (+ 258us) write_op.cc:270] Start()
0811 18:57:13.061522 (+ 26us) write_op.cc:276] Timestamp: P: 1786474625268767 usec, L: 0
0811 18:57:13.061818 (+ 296us) write_op.cc:270] Start()
0811 18:57:13.061855 (+ 37us) write_op.cc:276] Timestamp: P: 1786474625491202 usec, L: 0
0811 18:57:13.062153 (+ 298us) write_op.cc:270] Start()
0811 18:57:13.062186 (+ 33us) write_op.cc:276] Timestamp: P: 178647
I20260811 18:57:14.002063 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 190) took 6749 ms. Trace:
I20260811 18:57:14.027637 4162 raft_consensus.cc:348] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.045637 4169 rpcz_store.cc:276] 0811 18:57:07.250231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.250307 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:13.689743 (+6439436us) ts_tablet_manager.cc:1145] Deleting tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
0811 18:57:13.689757 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.797657 (+107900us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.906071 (+108414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.999806 (+ 93735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5380109}
I20260811 18:57:14.046185 4162 raft_consensus.cc:374] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.046880 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 207) took 2675 ms. Trace:
I20260811 18:57:14.046888 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (table default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]) successfully deleted
I20260811 18:57:14.047044 4169 rpcz_store.cc:276] 0811 18:57:11.371448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375371 (+ 3923us) service_pool.cc:227] Handling call
0811 18:57:14.046501 (+2671130us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:14.046517 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.046691 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2386499}
I20260811 18:57:14.046180 4163 raft_consensus.cc:374] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:14.047775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0190cbd859274fcfa09c6fab899c4843 already in progress: creating tablet
I20260811 18:57:14.048267 4162 raft_consensus.cc:729] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.049067 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 208) took 2675 ms. Trace:
I20260811 18:57:14.055322 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet c08436f33db043b1bc6395cd92102e59 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.055609 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:14.056332 3549 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 309) took 2166 ms. Trace:
I20260811 18:57:14.056463 3549 rpcz_store.cc:276] 0811 18:57:11.889901 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.472638 (+1582737us) service_pool.cc:227] Handling call
0811 18:57:13.540976 (+ 68338us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f412e6 00007f61ef234126 00007f61ef233e55 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:13.541815 (+ 839us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.541974 (+ 159us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.542045 (+ 71us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.843289 (+301244us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:14.055320 (+212031us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:14.055418 (+ 98us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:14.055458 (+ 40us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:14.056209 (+ 751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.980964 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.981078 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.981085 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.981087 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:13.981221 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.981231 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.981236 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.981357 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:13.981360 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.981404 (+ 44us) write_op.cc:270] Start()
0811 18:57:13.981432 (+ 28us) write_op.cc:276] Timestamp: P: 1786474633981368 usec, L: 0
0811 18:57:13.981435 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.981612 (+ 177us) log.cc:842] Serialized 725 byte log entry
0811 18:57:14.053225 (+ 71613us) op_driver.cc:444] REPLICATION: finished
0811 18:57:14.054581 (+ 1356us) write_op.cc:301] APPLY: starting
0811 18:57:14.054601 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:14.054670 (+ 69us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:14.054746 (+ 76us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:14.054918 (+ 172us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:14.054968 (+ 50us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:14.054980 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:14.055077 (+ 97us) log.cc:842] Serialized 29 byte log entry
0811 18:57:14.055148 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:14.055171 (+ 23us) write_op.cc:454] Released schema lock
0811 18:57:14.055181 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:14.055277 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:14.055284 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":156432768,"child_traces":[["op",{"apply.queue_time_us":1301,"apply.run_cpu_time_us":781,"apply.run_wall_time_us":780,"num_ops":2,"prepare.queue_time_us":136521,"prepare.run_cpu_time_us":779,"prepare.run_wall_time_us":11884,"replication_time_us":71765}]]}
I20260811 18:57:14.057389 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 405773849fb94800be61ad8a7a6055a4
I20260811 18:57:14.057691 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c87170e0a947438da8d0091a78e401c4" } new_table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" modify_external_catalogs: true
I20260811 18:57:14.058051 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 405773849fb94800be61ad8a7a6055a4 deleted and replaced by tablet 0aee6beb2e70421083841520733087f8
I20260811 18:57:14.058564 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cb87feb6db445d792ba8021cbe75510" } new_table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" modify_external_catalogs: true
I20260811 18:57:14.054224 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_021124838f7841c49f01816fb948d73c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.061638 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_021124838f7841c49f01816fb948d73c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.064527 4638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:14.068084 4638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:14.078564 4163 raft_consensus.cc:729] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.079010 4163 consensus_queue.cc:260] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.083388 4163 ts_tablet_manager.cc:1428] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.103s user 0.002s sys 0.002s
I20260811 18:57:14.083566 4169 rpcz_store.cc:276] 0811 18:57:11.371593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375616 (+ 4023us) service_pool.cc:227] Handling call
0811 18:57:14.047331 (+2671715us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:14.047343 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.047446 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2387237}
I20260811 18:57:14.086212 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d" } modify_external_catalogs: true
I20260811 18:57:14.086710 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d" } modify_external_catalogs: true
I20260811 18:57:14.089480 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 6ce8db230c7e41b6a4792273a0f07e6e from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:14.090432 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6ce8db230c7e41b6a4792273a0f07e6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ce8db230c7e41b6a4792273a0f07e6e already in progress: creating tablet
I20260811 18:57:14.060243 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.069248 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.081662 4556 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.095513 4556 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.095630 4162 consensus_queue.cc:260] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.096680 4162 ts_tablet_manager.cc:1428] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.128s user 0.000s sys 0.004s
I20260811 18:57:14.096927 4556 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.101701 4162 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.103099 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:14.104030 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 9bc5448a555944dfba8ccae09e25cbe9 deleted and replaced by tablet d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:14.104806 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f3fb616fae4249b4a043ddb879c7db57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:14.105059 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:14.105944 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3fb616fae4249b4a043ddb879c7db57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.108853 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_98051d104e9e4d9b976577088d6126a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.109674 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98051d104e9e4d9b976577088d6126a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.113193 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.113700 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.114888 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c08436f33db043b1bc6395cd92102e59
I20260811 18:57:14.115520 3547 catalog_manager.cc:6565] ReplaceTablet: tablet c08436f33db043b1bc6395cd92102e59 deleted and replaced by tablet 059b325df1db4fde9d4b26b559121316
I20260811 18:57:14.113711 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.091609 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:14.117043 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.119222 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d16e5449864d18a0a01e39d25ad8dd" } new_table_name: "default.Table_88ad2bea931c4ea0843ace52fcb146b3" modify_external_catalogs: true
I20260811 18:57:14.120790 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f640c60e81b4a369458b1497a269c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.121217 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.090320 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1a8b47adc7454c64a44013e919b3a485" } modify_external_catalogs: true
I20260811 18:57:14.121871 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1a8b47adc7454c64a44013e919b3a485" } modify_external_catalogs: true
I20260811 18:57:14.125041 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a5043728e7c842ea851193559d898d80" } new_table_name: "default.Table_06e3c71319514b82b5e73f430a685e0f" modify_external_catalogs: true
I20260811 18:57:14.127823 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.128259 3891 raft_consensus.cc:2454] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.129204 3676 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.130024 4576 raft_consensus.cc:2781] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.130182 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f640c60e81b4a369458b1497a269c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.130244 4576 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.130582 4576 raft_consensus.cc:3037] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.131678 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.132234 4163 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.135795 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.137131 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 209) took 2758 ms. Trace:
I20260811 18:57:14.137382 4169 rpcz_store.cc:276] 0811 18:57:11.371709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375789 (+ 4080us) service_pool.cc:227] Handling call
0811 18:57:14.083974 (+2708185us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:14.118444 (+ 34470us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.130507 (+ 12063us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2423826}
I20260811 18:57:14.138285 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 210) took 2766 ms. Trace:
I20260811 18:57:14.138458 4169 rpcz_store.cc:276] 0811 18:57:11.371814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375940 (+ 4126us) service_pool.cc:227] Handling call
0811 18:57:14.137817 (+2761877us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:14.137832 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.138143 (+ 311us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2477617}
I20260811 18:57:14.139111 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 220) took 1357 ms. Trace:
I20260811 18:57:14.140012 4169 rpcz_store.cc:276] 0811 18:57:12.781970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.785648 (+ 3678us) service_pool.cc:227] Handling call
0811 18:57:14.138781 (+1353133us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:14.138797 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.139021 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1351340}
I20260811 18:57:14.140717 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 218) took 1359 ms. Trace:
I20260811 18:57:14.140918 4169 rpcz_store.cc:276] 0811 18:57:12.781641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.786564 (+ 4923us) service_pool.cc:227] Handling call
0811 18:57:14.140300 (+1353736us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:14.140310 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.140505 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35,"tablet-delete.queue_time_us":1352864}
I20260811 18:57:14.141752 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 219) took 1359 ms. Trace:
W20260811 18:57:14.144634 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 460e03cc14444c83b6c08ca194ca5a8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 460e03cc14444c83b6c08ca194ca5a8d already in progress: creating tablet
W20260811 18:57:14.145123 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dde27d8cc3eb4d09879e717fc90305fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dde27d8cc3eb4d09879e717fc90305fd already in progress: creating tablet
W20260811 18:57:14.145437 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1a55a42d1e64f04ad7f433614a3a1aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1a55a42d1e64f04ad7f433614a3a1aa already in progress: creating tablet
W20260811 18:57:14.145774 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ed3346bd12d04c45a511308c427856e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed3346bd12d04c45a511308c427856e5 already in progress: creating tablet
I20260811 18:57:14.148324 4163 tablet_bootstrap.cc:654] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.142179 4169 rpcz_store.cc:276] 0811 18:57:12.781813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.785678 (+ 3865us) service_pool.cc:227] Handling call
0811 18:57:14.141342 (+1355664us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:14.141354 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.141490 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1351677}
I20260811 18:57:14.151463 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 226) took 1267 ms. Trace:
I20260811 18:57:14.151716 4169 rpcz_store.cc:276] 0811 18:57:12.883525 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.883592 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:14.150846 (+1267254us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:14.150862 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.151300 (+ 438us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1248187}
W20260811 18:57:14.153930 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 94943cfbe41a47b3a36c81957a3adaeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94943cfbe41a47b3a36c81957a3adaeb already in progress: creating tablet
W20260811 18:57:14.154470 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f61d3822fc2d4b569f223238a6464e0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f61d3822fc2d4b569f223238a6464e0a already in progress: creating tablet
W20260811 18:57:14.154924 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f335e50ae57547ed80b144129faaffbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f335e50ae57547ed80b144129faaffbb already in progress: creating tablet
W20260811 18:57:14.156575 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7251204325dd424192a6f58e18e71c34 has no consensus state
I20260811 18:57:14.157183 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44e9cbdec47545899467895fd869b389" } modify_external_catalogs: true
I20260811 18:57:14.157533 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44e9cbdec47545899467895fd869b389" } modify_external_catalogs: true
W20260811 18:57:14.159238 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbba1923ce0049448fe4e28f2dd258dc consensus state has no leader
I20260811 18:57:14.159881 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5345e1fe57284913bcf778715a827515" } new_table_name: "default.Table_2e738d6898cb440ca15ea7cca8f58b15" modify_external_catalogs: true
I20260811 18:57:14.161495 3550 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd] alter complete (version 1)
I20260811 18:57:14.163079 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1b41e9593787430fb7137bed80d6f078"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.164193 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b41e9593787430fb7137bed80d6f078 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.170094 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ef393c341bb54baf87176db5b5254ec7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef393c341bb54baf87176db5b5254ec7 already in progress: creating tablet
W20260811 18:57:14.170740 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because the tablet was not found. No further retry: Not found: Tablet not found: d106e0e1f4b845429b10f4e100f69a8b
W20260811 18:57:14.171175 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c already in progress: creating tablet
I20260811 18:57:14.173969 4729 consensus_queue.cc:1035] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:14.174511 4169 tablet_replica.cc:330] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.175050 4169 raft_consensus.cc:2229] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.175987 3891 raft_consensus.cc:1261] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260811 18:57:14.177224 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 490f34d7a403416fbab64e46973746de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 490f34d7a403416fbab64e46973746de already in progress: creating tablet
W20260811 18:57:14.178653 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 125757b8c1a6448b841c63339f392339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125757b8c1a6448b841c63339f392339 already in progress: creating tablet
I20260811 18:57:14.179186 4729 consensus_queue.cc:1035] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260811 18:57:14.173527 4162 tablet_bootstrap.cc:654] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.190479 4163 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.190907 4163 ts_tablet_manager.cc:1397] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.004s
I20260811 18:57:14.191689 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:14.192538 4163 raft_consensus.cc:348] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.193017 4163 raft_consensus.cc:374] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.193179 4163 raft_consensus.cc:729] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.196250 4163 consensus_queue.cc:260] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.198835 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.198469 4163 ts_tablet_manager.cc:1428] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:14.202958 4163 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.207110 4163 tablet_bootstrap.cc:654] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.211108 4576 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.213176 4576 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.216956 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.238520 4169 raft_consensus.cc:2258] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.239652 4169 ts_tablet_manager.cc:1905] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.241153 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.241576 3888 raft_consensus.cc:3037] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.246127 4162 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.246594 4162 ts_tablet_manager.cc:1397] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.145s user 0.009s sys 0.000s
I20260811 18:57:14.248323 4162 raft_consensus.cc:348] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.248987 4162 raft_consensus.cc:374] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.250488 4162 raft_consensus.cc:729] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.266387 4658 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.266702 4658 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.268030 4658 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:14.264408 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 43526.5us
I20260811 18:57:14.269130 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.275563 3744 raft_consensus.cc:2454] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.276484 3824 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.276793 4163 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.277025 4658 raft_consensus.cc:2781] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.277364 4163 ts_tablet_manager.cc:1397] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.010s
I20260811 18:57:14.277794 4658 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.278021 4658 raft_consensus.cc:3037] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:14.278913 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23f41a2638e74ba19c1c527b9fc796df consensus state has no leader
I20260811 18:57:14.280632 4739 raft_consensus.cc:482] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.281013 4739 raft_consensus.cc:504] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.283249 4739 leader_election.cc:288] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.283289 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.284137 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdc11a6352eb4aac9ff657cf245aceee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.285270 4162 consensus_queue.cc:260] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.286157 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c70d9db064e45af91fd9901d85e211f consensus state has no leader
I20260811 18:57:14.286360 4162 ts_tablet_manager.cc:1428] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.039s user 0.001s sys 0.002s
I20260811 18:57:14.286867 4162 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.287636 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6d87a15f366a4e118c84e294c3b9926e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.288818 3676 leader_election.cc:341] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 2852f9e5f67c4025836018af553388a5
I20260811 18:57:14.288992 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd862d7d75f24a05aa54d761e710cd50" } modify_external_catalogs: true
I20260811 18:57:14.289295 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd862d7d75f24a05aa54d761e710cd50" } modify_external_catalogs: true
I20260811 18:57:14.290139 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_858da7e74d4141a19ced6fdc29c33d89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.291023 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_858da7e74d4141a19ced6fdc29c33d89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.294792 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d87a15f366a4e118c84e294c3b9926e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.298964 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6f36ab6bb774257b5c2692c64eff63d consensus state has no leader
I20260811 18:57:14.299612 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e746326e8bff404aa5b3bc1fac099489"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.300535 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e746326e8bff404aa5b3bc1fac099489 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.306668 4162 tablet_bootstrap.cc:654] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.324653 4169 ts_tablet_manager.cc:1918] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.327580 4167 ts_tablet_manager.cc:1918] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.332760 4167 log.cc:1196] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
I20260811 18:57:14.333642 4167 ts_tablet_manager.cc:1939] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:14.328423 4658 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.336668 4658 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.337692 4169 log.cc:1196] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:14.338109 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:14.338562 3744 raft_consensus.cc:3037] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.339172 4169 ts_tablet_manager.cc:1939] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.342064 4163 raft_consensus.cc:348] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.342542 4163 raft_consensus.cc:374] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.342720 4163 raft_consensus.cc:729] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.343040 4163 consensus_queue.cc:260] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.344162 4163 ts_tablet_manager.cc:1428] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.066s user 0.003s sys 0.000s
I20260811 18:57:14.344658 4163 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.348503 4163 tablet_bootstrap.cc:654] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.358245 4162 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.358500 4162 ts_tablet_manager.cc:1397] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.002s
I20260811 18:57:14.359221 4739 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.359579 4739 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.360391 4739 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.361632 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.362684 3891 raft_consensus.cc:2454] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.360550 4162 raft_consensus.cc:348] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.363386 3676 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.363440 4162 raft_consensus.cc:374] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.363688 4162 raft_consensus.cc:729] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.363910 4739 raft_consensus.cc:2781] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.364087 4739 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.364061 4162 consensus_queue.cc:260] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.364507 4739 raft_consensus.cc:3037] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.366189 4162 ts_tablet_manager.cc:1428] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:14.366746 4162 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.368081 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:14.369165 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_994d2d4469b54597a9aa33c59dca9d84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.369988 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_994d2d4469b54597a9aa33c59dca9d84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.373051 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.375102 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b07958bac6344da1bbf3d7d35f1ac551"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.376061 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b07958bac6344da1bbf3d7d35f1ac551 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.379679 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.387741 4162 tablet_bootstrap.cc:654] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.391525 4658 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.391824 4658 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.392771 4658 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.394115 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:14.395164 3824 leader_election.cc:341] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:14.400990 4163 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.401284 4163 ts_tablet_manager.cc:1397] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.057s user 0.015s sys 0.001s
I20260811 18:57:14.402413 4163 raft_consensus.cc:348] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.403062 4163 raft_consensus.cc:374] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.403291 4163 raft_consensus.cc:729] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.403771 4163 consensus_queue.cc:260] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.408244 4163 ts_tablet_manager.cc:1428] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:14.410610 4163 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.414026 4163 tablet_bootstrap.cc:654] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.418205 3888 raft_consensus.cc:2454] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.419806 4169 tablet_replica.cc:330] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.420199 4169 raft_consensus.cc:2229] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.420634 4169 raft_consensus.cc:2258] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.421659 4169 ts_tablet_manager.cc:1905] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.426789 3744 raft_consensus.cc:2454] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:14.427402 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a0ba169c32ef48b984e649b211ee2af1 (table default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:14.427543 3824 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:14.427793 3824 raft_consensus.cc:2670] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:14.427989 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a902ad70a25546be926ff8f7bf2ed64f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a902ad70a25546be926ff8f7bf2ed64f already in progress: creating tablet
W20260811 18:57:14.428294 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 867259e37fe44c33a35823ec2d896754 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 867259e37fe44c33a35823ec2d896754 already in progress: creating tablet
W20260811 18:57:14.428843 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 405773849fb94800be61ad8a7a6055a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 405773849fb94800be61ad8a7a6055a4 already in progress: creating tablet
I20260811 18:57:14.433521 3676 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.433717 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 176) took 7466 ms. Trace:
I20260811 18:57:14.434047 4576 raft_consensus.cc:2781] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.435012 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:14.435946 4576 raft_consensus.cc:686] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.436350 4576 consensus_queue.cc:237] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.434077 4167 rpcz_store.cc:276] 0811 18:57:06.966943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.967502 (+ 559us) service_pool.cc:227] Handling call
0811 18:57:13.996346 (+7028844us) ts_tablet_manager.cc:1145] Deleting tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
0811 18:57:13.996369 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.207991 (+211622us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:14.327310 (+119319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:14.433566 (+106256us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7028515}
I20260811 18:57:14.437835 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:14.438325 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 198) took 6039 ms. Trace:
I20260811 18:57:14.438681 4167 rpcz_store.cc:276] 0811 18:57:08.399130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.399224 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:14.437961 (+6038737us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:14.437975 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.438193 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6020530}
I20260811 18:57:14.439425 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 199) took 6024 ms. Trace:
I20260811 18:57:14.439726 4167 rpcz_store.cc:276] 0811 18:57:08.414579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.414663 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:14.439074 (+6024411us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:14.439085 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.439259 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5991913}
I20260811 18:57:14.440969 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (table default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]) successfully deleted
W20260811 18:57:14.441370 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0190cbd859274fcfa09c6fab899c4843 already in progress: creating tablet
I20260811 18:57:14.441895 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 200) took 5995 ms. Trace:
W20260811 18:57:14.441895 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6ce8db230c7e41b6a4792273a0f07e6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ce8db230c7e41b6a4792273a0f07e6e already in progress: creating tablet
I20260811 18:57:14.442255 4167 rpcz_store.cc:276] 0811 18:57:08.446364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.446549 (+ 185us) service_pool.cc:227] Handling call
0811 18:57:14.440011 (+5993462us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:14.440021 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.441710 (+ 1689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5963396}
I20260811 18:57:14.443230 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.443403 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 201) took 5992 ms. Trace:
I20260811 18:57:14.443591 4167 rpcz_store.cc:276] 0811 18:57:08.451041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.451112 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:14.443083 (+5991971us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:14.443097 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.443310 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":553,"tablet-delete.queue_time_us":5966469}
W20260811 18:57:14.443910 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 460e03cc14444c83b6c08ca194ca5a8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 460e03cc14444c83b6c08ca194ca5a8d already in progress: creating tablet
W20260811 18:57:14.445024 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dde27d8cc3eb4d09879e717fc90305fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dde27d8cc3eb4d09879e717fc90305fd already in progress: creating tablet
I20260811 18:57:14.445751 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 211) took 2762 ms. Trace:
I20260811 18:57:14.446125 4167 rpcz_store.cc:276] 0811 18:57:11.682595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682898 (+ 303us) service_pool.cc:227] Handling call
0811 18:57:14.445021 (+2762123us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:14.445034 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.445153 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761830}
I20260811 18:57:14.446455 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.447355 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 210) took 2764 ms. Trace:
I20260811 18:57:14.447983 4167 rpcz_store.cc:276] 0811 18:57:11.682450 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682805 (+ 355us) service_pool.cc:227] Handling call
0811 18:57:14.447168 (+2764363us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:14.447178 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.447293 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1656622}
W20260811 18:57:14.448906 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1a55a42d1e64f04ad7f433614a3a1aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1a55a42d1e64f04ad7f433614a3a1aa already in progress: creating tablet
I20260811 18:57:14.449204 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 209) took 2766 ms. Trace:
I20260811 18:57:14.449536 4167 rpcz_store.cc:276] 0811 18:57:11.682208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682674 (+ 466us) service_pool.cc:227] Handling call
0811 18:57:14.448937 (+2766263us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:14.448949 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.449089 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1658311}
W20260811 18:57:14.449767 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 94943cfbe41a47b3a36c81957a3adaeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94943cfbe41a47b3a36c81957a3adaeb already in progress: creating tablet
W20260811 18:57:14.451191 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ed3346bd12d04c45a511308c427856e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed3346bd12d04c45a511308c427856e5 already in progress: creating tablet
I20260811 18:57:14.455888 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 58c515c153df4317b3fb427983ee15e0 (DEFAULT_TABLE table=default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456067 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1374a80d15444cb0faff339275aa8d (DEFAULT_TABLE table=default.Table_9292c8eb06fb4a0d8e33b6796398b740 [id=e0ef202b868f47269e4f2a63cb5a390a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456451 3861 tablet_service.cc:1467] Processing CreateTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 (DEFAULT_TABLE table=default.Table_cec1a44bf54f48019648c9338236fcc0 [id=5600dcaaf83c4f978c3d47f79e01568c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456986 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 40311cd1f327461e94eb443d83a16480 (DEFAULT_TABLE table=default.Table_4dfa50c8631f4b70b6b743837d5a9032 [id=01ec228d80ec40c8a1778d09edcd61b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.461556 3710 tablet_service.cc:1467] Processing CreateTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 (DEFAULT_TABLE table=default.Table_cec1a44bf54f48019648c9338236fcc0 [id=5600dcaaf83c4f978c3d47f79e01568c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.462311 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d4c26da86b49b8a37f2289f4ef6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.468864 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 217) took 1665 ms. Trace:
I20260811 18:57:14.471963 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 58c515c153df4317b3fb427983ee15e0 (DEFAULT_TABLE table=default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.472681 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c515c153df4317b3fb427983ee15e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.477416 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1374a80d15444cb0faff339275aa8d (DEFAULT_TABLE table=default.Table_9292c8eb06fb4a0d8e33b6796398b740 [id=e0ef202b868f47269e4f2a63cb5a390a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.478400 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1374a80d15444cb0faff339275aa8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.479609 4163 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.480031 4163 ts_tablet_manager.cc:1397] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.004s
I20260811 18:57:14.469154 4167 rpcz_store.cc:276] 0811 18:57:12.803641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.803976 (+ 335us) service_pool.cc:227] Handling call
0811 18:57:14.468452 (+1664476us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:14.468468 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.468755 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1664165}
I20260811 18:57:14.481886 4163 raft_consensus.cc:348] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.482456 4163 raft_consensus.cc:374] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.482745 4163 raft_consensus.cc:729] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.483498 4163 consensus_queue.cc:260] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.484460 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (DEFAULT_TABLE table=default.Table_77c2e3d67a494888be8b3405d4a06cf2 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.485064 4163 ts_tablet_manager.cc:1428] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260811 18:57:14.486851 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (DEFAULT_TABLE table=default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.486852 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 45fde25bcb40493aafb083e161aa6932 (DEFAULT_TABLE table=default.Table_575c5aca059048d68252cca6583a1f67 [id=4bc5d2d6894e40ebab164e5ffd6ac600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.486907 4163 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.490526 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3441d374bdb04681b2bd464f6ed910e2" } modify_external_catalogs: true
I20260811 18:57:14.491117 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3441d374bdb04681b2bd464f6ed910e2" } modify_external_catalogs: true
I20260811 18:57:14.494781 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6317dbacc890428caf5adac65ad7895d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:14.496156 4163 tablet_bootstrap.cc:654] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:14.496451 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6317dbacc890428caf5adac65ad7895d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.509548 4740 raft_consensus.cc:482] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.509869 4740 raft_consensus.cc:504] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.511914 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807cee88c18340dbb23c51cad54ab56d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.512475 3888 raft_consensus.cc:2454] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.513873 3676 leader_election.cc:302] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.519372 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267303174d744009a1803a363c1e431f" } new_table_name: "default.Table_2f38c7848c3140fbac5000b908fe7eae" modify_external_catalogs: true
I20260811 18:57:14.522282 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.522948 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.523453 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.524570 4740 leader_election.cc:288] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.525323 4729 raft_consensus.cc:2781] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.525534 4729 raft_consensus.cc:482] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.527379 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1374a80d15444cb0faff339275aa8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.528067 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d4c26da86b49b8a37f2289f4ef6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.528891 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3263da75f0d84528b7c8cc55dcaeaa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.529568 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.530232 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fde25bcb40493aafb083e161aa6932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.531581 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40311cd1f327461e94eb443d83a16480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.537883 4729 raft_consensus.cc:3037] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.538915 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 40311cd1f327461e94eb443d83a16480 (DEFAULT_TABLE table=default.Table_4dfa50c8631f4b70b6b743837d5a9032 [id=01ec228d80ec40c8a1778d09edcd61b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.540172 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (DEFAULT_TABLE table=default.Table_77c2e3d67a494888be8b3405d4a06cf2 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.540414 4739 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.540835 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (DEFAULT_TABLE table=default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.541962 4739 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.545497 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c515c153df4317b3fb427983ee15e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:14.547870 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f61d3822fc2d4b569f223238a6464e0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f61d3822fc2d4b569f223238a6464e0a already in progress: creating tablet
I20260811 18:57:14.547641 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.550441 4162 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.550386 3888 raft_consensus.cc:3037] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.559473 4163 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.562475 4162 ts_tablet_manager.cc:1397] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.188s user 0.008s sys 0.024s
I20260811 18:57:14.562772 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40311cd1f327461e94eb443d83a16480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.564850 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 218) took 1644 ms. Trace:
I20260811 18:57:14.588819 4163 ts_tablet_manager.cc:1397] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.006s
I20260811 18:57:14.593355 4163 raft_consensus.cc:348] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.595299 4163 raft_consensus.cc:374] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.595528 4163 raft_consensus.cc:729] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.596906 4163 consensus_queue.cc:260] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.598273 4163 ts_tablet_manager.cc:1428] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260811 18:57:14.598860 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.599399 4163 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.603276 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.603308 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 45fde25bcb40493aafb083e161aa6932 (DEFAULT_TABLE table=default.Table_575c5aca059048d68252cca6583a1f67 [id=4bc5d2d6894e40ebab164e5ffd6ac600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.564057 4162 raft_consensus.cc:348] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.624786 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68084.7us
I20260811 18:57:14.626539 4658 raft_consensus.cc:482] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.626811 4658 raft_consensus.cc:504] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.627628 4658 leader_election.cc:288] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.628855 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7251204325dd424192a6f58e18e71c34 (DEFAULT_TABLE table=default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.629367 4658 consensus_queue.cc:1035] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:14.629688 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.631393 4163 tablet_bootstrap.cc:654] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.639753 4169 ts_tablet_manager.cc:1918] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:14.640121 4169 log.cc:1196] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/60274115fe164d85badcf2e72d640f54
I20260811 18:57:14.607082 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3263da75f0d84528b7c8cc55dcaeaa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.641050 4169 ts_tablet_manager.cc:1939] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.642127 4162 raft_consensus.cc:374] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.642315 4162 raft_consensus.cc:729] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.642558 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6432abd4fc5c4f9ca95971ca4795e1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.642916 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7251204325dd424192a6f58e18e71c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.642777 4162 consensus_queue.cc:260] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.643565 4162 ts_tablet_manager.cc:1428] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.081s user 0.000s sys 0.003s
I20260811 18:57:14.643985 4162 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.610565 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7251204325dd424192a6f58e18e71c34 (DEFAULT_TABLE table=default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.644974 4729 raft_consensus.cc:504] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.647147 4729 leader_election.cc:288] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.648236 4658 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.648427 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807cee88c18340dbb23c51cad54ab56d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.649047 3891 raft_consensus.cc:3037] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.650941 3888 raft_consensus.cc:2454] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.652063 4658 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.652944 3676 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.653651 4576 raft_consensus.cc:2781] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.653980 4576 raft_consensus.cc:686] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.655087 4576 consensus_queue.cc:237] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.608532 4167 rpcz_store.cc:276] 0811 18:57:12.919809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.920000 (+ 191us) service_pool.cc:227] Handling call
0811 18:57:14.480345 (+1560345us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:14.480359 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.564569 (+ 84210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1342147}
I20260811 18:57:14.660842 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:14.664582 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ef393c341bb54baf87176db5b5254ec7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef393c341bb54baf87176db5b5254ec7 already in progress: creating tablet
I20260811 18:57:14.664665 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cf2375b4c64dbfae4e20003545da60" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.665225 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6432abd4fc5c4f9ca95971ca4795e1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.670487 4658 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.673517 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 219) took 1744 ms. Trace:
I20260811 18:57:14.675731 3744 raft_consensus.cc:2454] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.679462 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fde25bcb40493aafb083e161aa6932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.679551 4167 rpcz_store.cc:276] 0811 18:57:12.920626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.920709 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:14.664452 (+1743743us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:14.664478 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.665183 (+ 705us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1524526}
I20260811 18:57:14.680342 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 220) took 1269 ms. Trace:
W20260811 18:57:14.681063 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f335e50ae57547ed80b144129faaffbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f335e50ae57547ed80b144129faaffbb already in progress: creating tablet
I20260811 18:57:14.681699 4167 rpcz_store.cc:276] 0811 18:57:13.410590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.411165 (+ 575us) service_pool.cc:227] Handling call
0811 18:57:14.680043 (+1268878us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:14.680058 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.680217 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268527}
W20260811 18:57:14.682592 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 494c09652d9e408880ca606ae0b30773 because the tablet was not found. No further retry: Not found: Tablet not found: 494c09652d9e408880ca606ae0b30773
I20260811 18:57:14.683828 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7251204325dd424192a6f58e18e71c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.683848 4167 tablet_replica.cc:330] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:14.684314 3824 leader_election.cc:341] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:14.685207 3824 leader_election.cc:302] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.685832 4658 raft_consensus.cc:2781] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.685988 4658 raft_consensus.cc:482] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.687242 4167 raft_consensus.cc:2229] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.687434 4658 raft_consensus.cc:3037] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.687711 4167 raft_consensus.cc:2258] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.689836 4167 ts_tablet_manager.cc:1905] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:14.690739 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 125757b8c1a6448b841c63339f392339 (table default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 125757b8c1a6448b841c63339f392339 consensus state has no leader
I20260811 18:57:14.704293 4756 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:14.704892 4756 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:14.707348 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 339 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:57:14.708520 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 339 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:14.723606 3891 raft_consensus.cc:2454] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.731932 4169 tablet_replica.cc:330] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.732283 4169 raft_consensus.cc:2229] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:14.732703 4169 pending_rounds.cc:62] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:14.732843 4169 pending_rounds.cc:68] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400100693155840 op_type: NO_OP noop_request { }
I20260811 18:57:14.733148 4169 raft_consensus.cc:2866] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:14.733346 4169 raft_consensus.cc:2258] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.734015 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 60274115fe164d85badcf2e72d640f54 (table default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]) successfully deleted
I20260811 18:57:14.734490 4169 ts_tablet_manager.cc:1905] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:14.734826 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9bc5448a555944dfba8ccae09e25cbe9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bc5448a555944dfba8ccae09e25cbe9 already in progress: creating tablet
I20260811 18:57:14.735411 4739 raft_consensus.cc:482] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.738466 3676 leader_election.cc:302] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.738554 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
W20260811 18:57:14.740767 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 54461.4us
I20260811 18:57:14.741855 4163 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.742309 4163 ts_tablet_manager.cc:1397] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.143s user 0.005s sys 0.008s
I20260811 18:57:14.744073 4163 raft_consensus.cc:348] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.744447 4163 raft_consensus.cc:374] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.744664 4163 raft_consensus.cc:729] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.745772 4729 raft_consensus.cc:2781] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.745254 4739 raft_consensus.cc:504] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.749279 4756 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:14.752321 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet fbba1923ce0049448fe4e28f2dd258dc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.753898 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508f0b39cd654aa991a2928a4b067762" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.754208 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 151df87806694f46be5013592c68c01a
I20260811 18:57:14.754308 3891 raft_consensus.cc:2454] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.745280 4163 consensus_queue.cc:260] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.760242 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.763062 4163 ts_tablet_manager.cc:1428] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.000s sys 0.003s
I20260811 18:57:14.771055 4163 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.778507 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 2f9fa42654cf4ec18220e5fb173ddef7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.779376 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0859cafbc76b4eae9fe4453842a95178" } modify_external_catalogs: true
I20260811 18:57:14.780402 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0859cafbc76b4eae9fe4453842a95178" } modify_external_catalogs: true
W20260811 18:57:14.780236 4381 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.124:44143, user_credentials={real_user=slave}} blocked reactor thread for 70946.5us
I20260811 18:57:14.786280 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_53cd8000542843c984903cd25e451bdf" } new_table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" modify_external_catalogs: true
I20260811 18:57:14.790138 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cdd3a653fc1b405cbfd89c5d98728e45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.791041 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cdd3a653fc1b405cbfd89c5d98728e45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.791046 4756 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:14.794299 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.795280 4163 tablet_bootstrap.cc:654] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.796810 4739 leader_election.cc:288] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.797367 4704 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:14.797686 4729 raft_consensus.cc:686] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.798054 4704 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: loading cluster ID for follower catalog manager: success
I20260811 18:57:14.799016 4729 consensus_queue.cc:237] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.801538 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.809217 3676 leader_election.cc:302] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.810732 4162 tablet_bootstrap.cc:654] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.818806 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4687b95728fc485fa0e842bc447eb83f
I20260811 18:57:14.829703 4169 ts_tablet_manager.cc:1918] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.830484 4169 log.cc:1196] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:14.831396 4169 ts_tablet_manager.cc:1939] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.831866 4763 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.832314 4763 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.833662 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.833858 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.834120 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.834833 4763 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.835325 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d62ce6ce67e045c3a28f6c71419187b5" } new_table_name: "default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3" modify_external_catalogs: true
I20260811 18:57:14.839375 4704 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: acquiring CA information for follower catalog manager: success
W20260811 18:57:14.840747 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 7215ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:14.842674 4658 raft_consensus.cc:504] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.848261 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53475.2us
I20260811 18:57:14.851058 4658 leader_election.cc:288] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:14.856086 3676 consensus_peers.cc:487] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0ba169c32ef48b984e649b211ee2af1. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:14.857503 4739 raft_consensus.cc:2781] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.859315 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cf2375b4c64dbfae4e20003545da60" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:14.859601 3745 raft_consensus.cc:3037] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.863816 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.862087 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33979de5b48c4674a46c8da3b7151753" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.865533 4704 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:14.866279 4739 raft_consensus.cc:482] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.867288 3744 raft_consensus.cc:2454] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.868175 3824 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.871337 4739 raft_consensus.cc:3037] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.872457 4658 raft_consensus.cc:2781] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.872737 4658 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.872912 4658 raft_consensus.cc:3037] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.895556 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
W20260811 18:57:14.899354 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c08436f33db043b1bc6395cd92102e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c08436f33db043b1bc6395cd92102e59 already in progress: creating tablet
W20260811 18:57:14.906484 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 02808605ee6c4634af47fc3fd57e9cd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02808605ee6c4634af47fc3fd57e9cd8 already in progress: creating tablet
W20260811 18:57:14.910087 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8c07150e31994bdf9a00ac2e7a954691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c07150e31994bdf9a00ac2e7a954691 already in progress: creating tablet
I20260811 18:57:14.911738 4169 tablet_replica.cc:330] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.912189 4169 raft_consensus.cc:2229] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.912660 4169 raft_consensus.cc:2258] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.913969 4169 ts_tablet_manager.cc:1905] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.929283 3745 raft_consensus.cc:2454] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:14.930330 3824 leader_election.cc:302] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.930963 4763 raft_consensus.cc:2781] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.931341 4763 raft_consensus.cc:686] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:14.931890 4763 consensus_queue.cc:237] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.940135 4162 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.940423 4162 ts_tablet_manager.cc:1397] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.297s user 0.000s sys 0.011s
W20260811 18:57:14.981016 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bd39dcc0bed4623a06f00a56b740dba has no consensus state
I20260811 18:57:14.983256 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ff1be026ca404eada53f60f257ea367c
I20260811 18:57:14.985773 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.987586 4770 tablet.cc:1722] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:14.989581 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b41e9593787430fb7137bed80d6f078" } new_table_name: "default.Table_b9bdd685e23249119bbeac4997cbbf9c" modify_external_catalogs: true
I20260811 18:57:14.993271 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c37e3fb34b3f446d854006b05dbe5a94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.994095 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c37e3fb34b3f446d854006b05dbe5a94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.998102 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:14.999747 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59cd4e34981541759a761fe10bf57a2c" } new_table_name: "default.Table_2bb9c598e2de422b86a8304bbece96db" modify_external_catalogs: true
I20260811 18:57:15.010043 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.943213 4162 raft_consensus.cc:348] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.015707 4162 raft_consensus.cc:374] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.016063 4162 raft_consensus.cc:729] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.016567 4162 consensus_queue.cc:260] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.017505 4162 ts_tablet_manager.cc:1428] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.077s user 0.003s sys 0.002s
I20260811 18:57:15.018118 4162 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.031839 4658 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.032785 4768 tablet.cc:1722] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.033599 4658 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.041458 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 9007f2c1f860449cbfda3986b307656f
I20260811 18:57:15.051326 4163 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.051864 4163 ts_tablet_manager.cc:1397] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.281s user 0.027s sys 0.064s
I20260811 18:57:15.053409 4163 raft_consensus.cc:348] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.053687 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet ff1be026ca404eada53f60f257ea367c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.054081 4163 raft_consensus.cc:374] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.054320 4163 raft_consensus.cc:729] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.054740 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a311fc9dc1724795a63f4df79f8ebcbc" } modify_external_catalogs: true
I20260811 18:57:15.055140 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a311fc9dc1724795a63f4df79f8ebcbc" } modify_external_catalogs: true
I20260811 18:57:15.055167 4163 consensus_queue.cc:260] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.057701 4163 ts_tablet_manager.cc:1428] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260811 18:57:15.058290 4163 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.065850 4163 tablet_bootstrap.cc:654] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:15.071933 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e51a08e6dbf04e3fa58689ee0cd45db3 consensus state has no leader
W20260811 18:57:15.072274 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 8041ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:15.081316 4167 ts_tablet_manager.cc:1918] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:15.089502 4163 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.089821 4163 ts_tablet_manager.cc:1397] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.011s
I20260811 18:57:15.094084 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5409ed17441242119ff946163da07a21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.096004 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5409ed17441242119ff946163da07a21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.097157 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.096990 4163 raft_consensus.cc:348] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.097347 4163 raft_consensus.cc:374] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.097536 4163 raft_consensus.cc:729] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.097977 4163 consensus_queue.cc:260] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.102160 4163 ts_tablet_manager.cc:1428] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20260811 18:57:15.104463 4163 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.079878 4739 raft_consensus.cc:504] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.108778 4729 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.109357 4163 tablet_bootstrap.cc:654] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.112577 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508f0b39cd654aa991a2928a4b067762" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.112903 3891 raft_consensus.cc:3037] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.135679 3891 raft_consensus.cc:2454] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.136343 4658 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.136672 4658 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.137583 4658 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.140025 3676 leader_election.cc:302] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.140658 4576 raft_consensus.cc:2781] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.141047 4167 log.cc:1196] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/60274115fe164d85badcf2e72d640f54
I20260811 18:57:15.141395 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33979de5b48c4674a46c8da3b7151753" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:15.142366 4739 leader_election.cc:288] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.142704 4576 raft_consensus.cc:686] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.143129 4576 consensus_queue.cc:237] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.145352 3740 raft_consensus.cc:3037] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.130249 4729 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.155591 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.156824 3891 raft_consensus.cc:2454] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.165807 3676 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.166419 4729 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.167591 4729 raft_consensus.cc:2781] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.167168 4167 ts_tablet_manager.cc:1939] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:15.170625 4729 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:15.171249 4729 raft_consensus.cc:3037] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:15.182710 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 76094.1us
I20260811 18:57:15.188937 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fbba1923ce0049448fe4e28f2dd258dc
I20260811 18:57:15.189574 3549 catalog_manager.cc:6565] ReplaceTablet: tablet fbba1923ce0049448fe4e28f2dd258dc deleted and replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:15.195484 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.198758 4163 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.199092 4163 ts_tablet_manager.cc:1397] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.095s user 0.015s sys 0.027s
I20260811 18:57:15.199980 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.200523 4163 raft_consensus.cc:348] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.200919 4163 raft_consensus.cc:374] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.201100 4163 raft_consensus.cc:729] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.201582 4163 consensus_queue.cc:260] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.206022 4163 ts_tablet_manager.cc:1428] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 18:57:15.206609 4163 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.208272 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.208591 4739 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.209043 4739 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.209928 4739 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.210899 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.211297 3888 raft_consensus.cc:2454] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.211891 4162 tablet_bootstrap.cc:654] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.211920 3676 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.212635 4739 raft_consensus.cc:2781] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.212817 4739 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:15.213002 4739 raft_consensus.cc:3037] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.213647 4163 tablet_bootstrap.cc:654] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.224627 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.225486 4163 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.225662 4163 ts_tablet_manager.cc:1397] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.003s
I20260811 18:57:15.226583 4163 raft_consensus.cc:348] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.226958 4163 raft_consensus.cc:374] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.227115 4163 raft_consensus.cc:729] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.227409 4163 consensus_queue.cc:260] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.230765 4163 ts_tablet_manager.cc:1428] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:57:15.231217 4163 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.234150 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f640c60e81b4a369458b1497a269c9c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.237787 4163 tablet_bootstrap.cc:654] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:15.242750 3676 consensus_peers.cc:487] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:15.255019 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 221) took 1843 ms. Trace:
I20260811 18:57:15.255188 4167 rpcz_store.cc:276] 0811 18:57:13.411019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.415153 (+ 4134us) service_pool.cc:227] Handling call
0811 18:57:14.682014 (+1266861us) ts_tablet_manager.cc:1145] Deleting tablet 60274115fe164d85badcf2e72d640f54
0811 18:57:14.682027 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.931971 (+249944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.031658 (+ 99687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.254922 (+223264us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1258213}
I20260811 18:57:15.255796 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 222) took 1783 ms. Trace:
I20260811 18:57:15.255910 4167 rpcz_store.cc:276] 0811 18:57:13.472609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.472686 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:15.255575 (+1782889us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:15.255589 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.255734 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1782491}
I20260811 18:57:15.256345 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 223) took 1692 ms. Trace:
I20260811 18:57:15.256459 4167 rpcz_store.cc:276] 0811 18:57:13.564094 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.564196 (+ 102us) service_pool.cc:227] Handling call
0811 18:57:15.256180 (+1691984us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:15.256192 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.256289 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1691456}
I20260811 18:57:15.256920 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 224) took 1669 ms. Trace:
I20260811 18:57:15.256950 4163 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.257036 4167 rpcz_store.cc:276] 0811 18:57:13.587553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.587626 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:15.256732 (+1669106us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:15.256744 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.256868 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1668714}
I20260811 18:57:15.257208 4163 ts_tablet_manager.cc:1397] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.000s
I20260811 18:57:15.257382 4167 tablet_replica.cc:330] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:15.257651 4167 raft_consensus.cc:2229] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:15.258265 4163 raft_consensus.cc:348] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.258643 4163 raft_consensus.cc:374] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.258733 4163 raft_consensus.cc:729] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.259123 4163 consensus_queue.cc:260] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.261822 4163 ts_tablet_manager.cc:1428] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:15.262316 4163 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.263171 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 60274115fe164d85badcf2e72d640f54 (table default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]) successfully deleted
W20260811 18:57:15.263504 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c already in progress: creating tablet
W20260811 18:57:15.263736 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 490f34d7a403416fbab64e46973746de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 490f34d7a403416fbab64e46973746de already in progress: creating tablet
W20260811 18:57:15.264008 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 125757b8c1a6448b841c63339f392339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125757b8c1a6448b841c63339f392339 already in progress: creating tablet
I20260811 18:57:15.265262 4167 raft_consensus.cc:2258] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.266079 4167 ts_tablet_manager.cc:1905] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.266777 4163 tablet_bootstrap.cc:654] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.270197 3740 raft_consensus.cc:2454] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:15.270977 3824 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.271376 4658 raft_consensus.cc:2781] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.271631 4658 raft_consensus.cc:686] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:15.271986 4658 consensus_queue.cc:237] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.282989 4162 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.283253 4162 ts_tablet_manager.cc:1397] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.265s user 0.005s sys 0.010s
I20260811 18:57:15.284157 4162 raft_consensus.cc:348] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.284488 4162 raft_consensus.cc:374] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.284587 4162 raft_consensus.cc:729] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.284945 4162 consensus_queue.cc:260] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.285552 4162 ts_tablet_manager.cc:1428] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:15.286065 4162 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.286499 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet b0b199f4c9e14b468cb26cbd61fc8586 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.288072 4162 tablet_bootstrap.cc:654] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.288923 4658 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.289192 4658 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.297816 4658 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.299924 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:15.300658 3824 leader_election.cc:341] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:15.302894 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6070ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:15.313436 4163 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.313295 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.313704 4163 ts_tablet_manager.cc:1397] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.005s
W20260811 18:57:15.314193 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a303b89a5ef54a7ea92f2bef6298d619 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.315037 4163 raft_consensus.cc:348] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.315317 4162 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.315367 4163 raft_consensus.cc:374] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.315585 4163 raft_consensus.cc:729] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.315593 4162 ts_tablet_manager.cc:1397] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260811 18:57:15.315976 4163 consensus_queue.cc:260] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.317094 4162 raft_consensus.cc:348] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.317512 4162 raft_consensus.cc:374] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.317725 4162 raft_consensus.cc:729] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.319545 4163 ts_tablet_manager.cc:1428] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:15.320020 4163 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.322670 4162 consensus_queue.cc:260] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.330422 4162 ts_tablet_manager.cc:1428] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.002s sys 0.002s
I20260811 18:57:15.330997 4162 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.330778 4729 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.331889 4729 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.332717 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.333029 3745 raft_consensus.cc:2379] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
I20260811 18:57:15.333061 3891 raft_consensus.cc:3037] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.337256 3891 raft_consensus.cc:2454] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.346547 4163 tablet_bootstrap.cc:654] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.347111 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.347769 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.349640 3676 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.354107 4576 raft_consensus.cc:2781] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.354313 4576 raft_consensus.cc:686] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.354705 4576 consensus_queue.cc:237] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.355779 4163 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.356011 4163 ts_tablet_manager.cc:1397] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.007s
I20260811 18:57:15.357311 4163 raft_consensus.cc:348] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.357666 4163 raft_consensus.cc:374] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.357758 4163 raft_consensus.cc:729] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.358075 4163 consensus_queue.cc:260] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.361004 4739 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.361683 4163 ts_tablet_manager.cc:1428] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260811 18:57:15.362025 4163 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.362414 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.362630 3891 raft_consensus.cc:3037] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.363121 4739 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.363901 4163 tablet_bootstrap.cc:654] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.365648 4162 tablet_bootstrap.cc:654] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.368017 3891 raft_consensus.cc:2454] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.368517 3676 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.368943 4739 raft_consensus.cc:2781] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.369321 4739 raft_consensus.cc:686] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.369887 4739 consensus_queue.cc:237] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.382799 4163 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.383031 4163 ts_tablet_manager.cc:1397] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.001s
I20260811 18:57:15.384140 4163 raft_consensus.cc:348] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.384400 4163 raft_consensus.cc:374] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.384517 4163 raft_consensus.cc:729] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.384783 4163 consensus_queue.cc:260] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.389657 4163 ts_tablet_manager.cc:1428] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 18:57:15.390017 4163 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.393390 4163 tablet_bootstrap.cc:654] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.401964 4169 ts_tablet_manager.cc:1918] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.402280 4169 log.cc:1196] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:15.412606 4162 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.417853 4162 ts_tablet_manager.cc:1397] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.087s user 0.004s sys 0.004s
W20260811 18:57:15.419087 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73f05e1d5cb947fe9dbcc2e876a23788 has no consensus state
I20260811 18:57:15.421239 4162 raft_consensus.cc:348] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.423362 4162 raft_consensus.cc:374] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.423514 4162 raft_consensus.cc:729] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.424160 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.424989 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b48fafe85a504185911a1e7edb4fcbfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:15.431068 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 866ed614511649459b22a9fce55220b2 consensus state has no leader
I20260811 18:57:15.432998 4167 ts_tablet_manager.cc:1918] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.435113 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.435817 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 8336ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
W20260811 18:57:15.435827 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.447005 3554 catalog_manager.cc:5495] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:15.447556 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:15.456494 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0e1efffade0f486d8d627e56e0be6ff6 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]). Expected version 2 got 0
I20260811 18:57:15.458994 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (DEFAULT_TABLE table=default.Table_b9bdd685e23249119bbeac4997cbbf9c [id=0ea510ab7a724cadb94d2cfbc5fde4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.459403 4658 raft_consensus.cc:482] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.459563 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f05e1d5cb947fe9dbcc2e876a23788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.459622 4658 raft_consensus.cc:504] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.460351 4658 leader_election.cc:288] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.461732 4169 ts_tablet_manager.cc:1939] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:15.462133 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 247) took 1083 ms. Trace:
I20260811 18:57:15.462260 4169 rpcz_store.cc:276] 0811 18:57:14.378801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.379400 (+ 599us) service_pool.cc:227] Handling call
0811 18:57:14.911507 (+532107us) ts_tablet_manager.cc:1145] Deleting tablet e7b3a14979e84b1e8ec6ed3f49b93b00
0811 18:57:14.911526 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.243329 (+331803us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.401707 (+158378us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.462083 (+ 60376us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":531610}
I20260811 18:57:15.463114 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_005d29915a3b4449acfc665e917be061"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.463884 4169 tablet_replica.cc:330] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:15.463924 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_005d29915a3b4449acfc665e917be061 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.464148 4169 raft_consensus.cc:2229] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:15.464413 4169 pending_rounds.cc:62] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:15.464496 4169 pending_rounds.cc:68] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400104684687360 op_type: NO_OP noop_request { }
I20260811 18:57:15.464632 4169 raft_consensus.cc:2866] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:15.464761 4169 raft_consensus.cc:2258] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.465481 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]) successfully deleted
I20260811 18:57:15.463177 4163 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.466743 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a122fe2820f4d23b12edb970a94e958"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.467089 4163 ts_tablet_manager.cc:1397] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.012s
W20260811 18:57:15.467480 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a122fe2820f4d23b12edb970a94e958 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.467922 4658 raft_consensus.cc:482] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.468055 4658 raft_consensus.cc:504] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.468827 4658 leader_election.cc:288] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.469218 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" } modify_external_catalogs: true
I20260811 18:57:15.469556 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" } modify_external_catalogs: true
I20260811 18:57:15.469789 4163 raft_consensus.cc:348] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.470093 4163 raft_consensus.cc:374] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.470247 4163 raft_consensus.cc:729] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.472847 4163 consensus_queue.cc:260] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.477246 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13fbac538d16421e921aa37f54683662" } modify_external_catalogs: true
I20260811 18:57:15.477602 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13fbac538d16421e921aa37f54683662" } modify_external_catalogs: true
I20260811 18:57:15.478901 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (DEFAULT_TABLE table=default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.479661 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45aff3e8cb0c4f7191e4522c6ef7d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.480172 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e738d6898cb440ca15ea7cca8f58b15" } new_table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" modify_external_catalogs: true
I20260811 18:57:15.484982 4169 ts_tablet_manager.cc:1905] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.461215 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a300a7aca2814da6b999eb4d865fea66" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.473461 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (DEFAULT_TABLE table=default.Table_b9bdd685e23249119bbeac4997cbbf9c [id=0ea510ab7a724cadb94d2cfbc5fde4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.500849 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f05e1d5cb947fe9dbcc2e876a23788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.476207 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (DEFAULT_TABLE table=default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.484750 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce8db230c7e41b6a4792273a0f07e6e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.491309 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 4f3c4536fc7440db83851fc8d7b3de10 (DEFAULT_TABLE table=default.Table_e75a55498d4c4072968ee03aaf48b831 [id=bd974444a6c546959eb6442a80543187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.493860 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 61cea7b0b85448f5854e8fd99ea5e792 (DEFAULT_TABLE table=default.Table_efa46b4a159b4805b21635ef65051022 [id=20ac4cbacf8f4ec8a8a41810f1da3355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.496317 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 3c96a264e017471ea114ca0c6a9500ef (DEFAULT_TABLE table=default.Table_98051d104e9e4d9b976577088d6126a2 [id=f5844f91d9ef446bbd4d34bdea7cba8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.499119 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.509636 4167 log.cc:1196] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:15.510620 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45aff3e8cb0c4f7191e4522c6ef7d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.511535 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61cea7b0b85448f5854e8fd99ea5e792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.512413 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f3c4536fc7440db83851fc8d7b3de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.514984 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968fa8b8912b40a8bdba77db1d39c5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.515893 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c96a264e017471ea114ca0c6a9500ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.520220 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3c96a264e017471ea114ca0c6a9500ef (DEFAULT_TABLE table=default.Table_98051d104e9e4d9b976577088d6126a2 [id=f5844f91d9ef446bbd4d34bdea7cba8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.521083 4163 ts_tablet_manager.cc:1428] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.041s user 0.003s sys 0.000s
W20260811 18:57:15.522327 3824 leader_election.cc:341] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:15.523008 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 4f3c4536fc7440db83851fc8d7b3de10 (DEFAULT_TABLE table=default.Table_e75a55498d4c4072968ee03aaf48b831 [id=bd974444a6c546959eb6442a80543187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.525804 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 61cea7b0b85448f5854e8fd99ea5e792 (DEFAULT_TABLE table=default.Table_efa46b4a159b4805b21635ef65051022 [id=20ac4cbacf8f4ec8a8a41810f1da3355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.526218 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.530076 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c96a264e017471ea114ca0c6a9500ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.531720 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0aee6beb2e70421083841520733087f8 (DEFAULT_TABLE table=default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.533450 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61cea7b0b85448f5854e8fd99ea5e792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.534245 3862 tablet_service.cc:1467] Processing CreateTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 (DEFAULT_TABLE table=default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.536696 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968fa8b8912b40a8bdba77db1d39c5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.538470 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aee6beb2e70421083841520733087f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.545753 4163 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.545651 3857 tablet_service.cc:1467] Processing CreateTablet for tablet b57ed7814a1e440cba362613cbab32f8 (DEFAULT_TABLE table=default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.547295 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3f120758c6405dafd2ea0bf54c81d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.548861 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57ed7814a1e440cba362613cbab32f8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:15.550277 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6246ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:15.552578 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f3c4536fc7440db83851fc8d7b3de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.560801 4770 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.564872 4768 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.572599 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 0aee6beb2e70421083841520733087f8 (DEFAULT_TABLE table=default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:15.577533 3554 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 42) took 976 ms (client timeout 1000 ms). Trace:
W20260811 18:57:15.577886 3554 rpcz_store.cc:269] 0811 18:57:14.600868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.643635 (+ 42767us) service_pool.cc:227] Handling call
0811 18:57:14.782024 (+138389us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f3ad8f 00007f61ef22f946 00007f61ef22f675 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:15.577383 (+795359us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.542318 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.548372 (+ 6054us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.548380 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:15.548383 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:15.548495 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.548509 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.548515 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:15.548661 (+ 146us) tablet.cc:699] Row locks acquired
0811 18:57:15.548666 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:15.548732 (+ 66us) write_op.cc:270] Start()
0811 18:57:15.548783 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635548691 usec, L: 0
0811 18:57:15.548789 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.549073 (+ 284us) log.cc:842] Serialized 527 byte log entry
0811 18:57:15.574469 (+ 25396us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.574713 (+ 244us) write_op.cc:301] APPLY: starting
0811 18:57:15.574728 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.574763 (+ 35us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.574813 (+ 50us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.574911 (+ 98us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.574950 (+ 39us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:15.574957 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:15.575049 (+ 92us) log.cc:842] Serialized 23 byte log entry
0811 18:57:15.575147 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.575180 (+ 33us) write_op.cc:454] Released schema lock
0811 18:57:15.575371 (+ 191us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.575458 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.575465 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":316642176,"child_traces":[["op",{"apply.queue_time_us":190,"apply.run_cpu_time_us":825,"apply.run_wall_time_us":825,"mutex_wait_us":1851,"num_ops":1,"prepare.queue_time_us":94063,"prepare.run_cpu_time_us":1294,"prepare.run_wall_time_us":35757,"raft.queue_time_us":15481,"raft.run_cpu_time_us":778,"raft.run_wall_time_us":779,"replication_time_us":25647}]]}
I20260811 18:57:15.586459 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd120e0ab3b34418960cefc5eb459cdb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.587258 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd120e0ab3b34418960cefc5eb459cdb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.598248 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b (DEFAULT_TABLE table=default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.598431 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:15.599031 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 2f9fa42654cf4ec18220e5fb173ddef7 deleted and replaced by tablet bb56d23ccabc4b02961249312257d9c4
I20260811 18:57:15.599340 3555 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 366) took 1122 ms. Trace:
I20260811 18:57:15.599799 3555 rpcz_store.cc:276] 0811 18:57:14.476954 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.778410 (+301456us) service_pool.cc:227] Handling call
0811 18:57:15.599149 (+820739us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.578678 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.578858 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.578865 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:15.578867 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:15.579045 (+ 178us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.579072 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.579078 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:15.579203 (+ 125us) tablet.cc:699] Row locks acquired
0811 18:57:15.579207 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:15.579265 (+ 58us) write_op.cc:270] Start()
0811 18:57:15.579327 (+ 62us) write_op.cc:276] Timestamp: P: 1786474635579230 usec, L: 0
0811 18:57:15.579333 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.585883 (+ 6550us) log.cc:842] Serialized 622 byte log entry
0811 18:57:15.596753 (+ 10870us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.596909 (+ 156us) write_op.cc:301] APPLY: starting
0811 18:57:15.596928 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.596992 (+ 64us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.597137 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.597292 (+ 155us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.597372 (+ 80us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:15.597384 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:15.597504 (+ 120us) log.cc:842] Serialized 29 byte log entry
0811 18:57:15.597595 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.597621 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:15.597631 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.597701 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.597708 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":867,"apply.run_wall_time_us":866,"num_ops":2,"prepare.queue_time_us":128890,"prepare.run_cpu_time_us":1187,"prepare.run_wall_time_us":16420,"raft.queue_time_us":554,"raft.run_cpu_time_us":938,"raft.run_wall_time_us":938,"replication_time_us":17339}]]}
I20260811 18:57:15.601220 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c038c8ee6e694c6f99d5cb717ba90ca9" } modify_external_catalogs: true
I20260811 18:57:15.601536 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c038c8ee6e694c6f99d5cb717ba90ca9" } modify_external_catalogs: true
I20260811 18:57:15.614852 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:15.615085 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d32bb4eca047e4ba786c04b8bebf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.615641 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 372) took 1033 ms. Trace:
I20260811 18:57:15.615903 3550 rpcz_store.cc:276] 0811 18:57:14.582816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.054364 (+471548us) service_pool.cc:227] Handling call
0811 18:57:15.055369 (+ 1005us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:15.055520 (+ 151us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:15.055595 (+ 75us) catalog_manager.cc:2761] Locking tablets
0811 18:57:15.450766 (+395171us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:15.614438 (+163672us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:15.614539 (+ 101us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:15.614631 (+ 92us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:15.615493 (+ 862us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.595172 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.595306 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.595312 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:15.595314 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:15.595462 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.595476 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.595482 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:15.595660 (+ 178us) tablet.cc:699] Row locks acquired
0811 18:57:15.595665 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:15.595717 (+ 52us) write_op.cc:270] Start()
0811 18:57:15.605463 (+ 9746us) write_op.cc:276] Timestamp: P: 1786474635595679 usec, L: 0
0811 18:57:15.605470 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.605706 (+ 236us) log.cc:842] Serialized 725 byte log entry
0811 18:57:15.612811 (+ 7105us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.613014 (+ 203us) write_op.cc:301] APPLY: starting
0811 18:57:15.613031 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.613076 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.613118 (+ 42us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.613189 (+ 71us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.613236 (+ 47us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:15.613242 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:15.614120 (+ 878us) log.cc:842] Serialized 29 byte log entry
0811 18:57:15.614204 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.614235 (+ 31us) write_op.cc:454] Released schema lock
0811 18:57:15.614243 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.614304 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.614307 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":152,"apply.run_cpu_time_us":631,"apply.run_wall_time_us":1368,"num_ops":2,"prepare.queue_time_us":143777,"raft.queue_time_us":2185,"raft.run_cpu_time_us":650,"raft.run_wall_time_us":651,"replication_time_us":7314}]]}
I20260811 18:57:15.619822 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91346cfabb664d0ebfc876dacc125c50" } modify_external_catalogs: true
I20260811 18:57:15.620389 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91346cfabb664d0ebfc876dacc125c50" } modify_external_catalogs: true
I20260811 18:57:15.631266 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 68a7051bc539468aafba30fbaf0669d4 (DEFAULT_TABLE table=default.Table_021124838f7841c49f01816fb948d73c [id=4a5d04ba5f3942a3942909ebd2bf5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.633623 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:15.634295 3553 catalog_manager.cc:6565] ReplaceTablet: tablet b0b199f4c9e14b468cb26cbd61fc8586 deleted and replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:15.637380 3865 tablet_service.cc:1467] Processing CreateTablet for tablet d00d943b778741b4aeec076ba46a9c40 (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.637948 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 with delete_type TABLET_DATA_DELETED (replaced by tablet bb56d23ccabc4b02961249312257d9c4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.639417 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a7051bc539468aafba30fbaf0669d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.640995 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_88ad2bea931c4ea0843ace52fcb146b3" } new_table_name: "default.Table_cbe771adb7364096802df77fa867acb0" modify_external_catalogs: true
I20260811 18:57:15.646188 4658 raft_consensus.cc:482] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.647890 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00d943b778741b4aeec076ba46a9c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.651384 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45] alter complete (version 2)
I20260811 18:57:15.652891 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 80cf2375b4c64dbfae4e20003545da60 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.659003 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2d444511ad44775b701b9ed7527e251"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.660531 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2d444511ad44775b701b9ed7527e251 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.664865 3710 tablet_service.cc:1467] Processing CreateTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 (DEFAULT_TABLE table=default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.663108 4576 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.669095 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aee6beb2e70421083841520733087f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.669348 4162 consensus_queue.cc:260] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.670281 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_220d78aeb25b4b64b594e0839928c35e" } new_table_name: "default.Table_ae88bb870eb143408638ffe3715fbf48" modify_external_catalogs: true
I20260811 18:57:15.653435 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd39dcc0bed4623a06f00a56b740dba (DEFAULT_TABLE table=default.Table_2f38c7848c3140fbac5000b908fe7eae [id=f312f8bcd459400b92a33532675d3991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.690121 4658 raft_consensus.cc:504] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.690943 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6bf41429faeb4785ad747d207e4641f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.691761 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bf41429faeb4785ad747d207e4641f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.697466 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd39dcc0bed4623a06f00a56b740dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.702185 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:15.703562 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.703970 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:15.705854 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_fce143fe6d0849a0a3af8939df804986 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:15.707434 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.707705 4163 tablet_bootstrap.cc:654] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.708560 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2f38c7848c3140fbac5000b908fe7eae" } new_table_name: "default.Table_28ed0eb238674836950a5639ebcbc6b9" modify_external_catalogs: true
I20260811 18:57:15.711599 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.715760 4658 leader_election.cc:288] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.719605 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:15.721724 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cec1a44bf54f48019648c9338236fcc0" } new_table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" modify_external_catalogs: true
I20260811 18:57:15.726428 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 059b325df1db4fde9d4b26b559121316 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.727077 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b325df1db4fde9d4b26b559121316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.743011 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.746801 3722 tablet_service.cc:1467] Processing CreateTablet for tablet b57ed7814a1e440cba362613cbab32f8 (DEFAULT_TABLE table=default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.747776 4729 consensus_queue.cc:1035] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.129s
I20260811 18:57:15.762185 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b (DEFAULT_TABLE table=default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.762996 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3f120758c6405dafd2ea0bf54c81d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.778739 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.779232 4162 ts_tablet_manager.cc:1428] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.342s user 0.003s sys 0.000s
W20260811 18:57:15.781948 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6f36ab6bb774257b5c2692c64eff63d consensus state has no leader
I20260811 18:57:15.782800 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7b9dec45633841ac830523612293aabf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.783895 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b9dec45633841ac830523612293aabf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.784636 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:15.786123 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:15.787206 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3e16ca699564a61a49c4cb989ad865f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.788069 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3e16ca699564a61a49c4cb989ad865f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.789809 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:15.790756 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_133a44e94783442e8c27d4730310f996"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.791425 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_133a44e94783442e8c27d4730310f996 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.795080 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57ed7814a1e440cba362613cbab32f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.796926 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.789825 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e8859199c3440ebb2b9664b65e0fb71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.800547 4167 ts_tablet_manager.cc:1939] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:15.805369 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.798955 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.807847 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d32bb4eca047e4ba786c04b8bebf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.809309 3740 raft_consensus.cc:2454] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:15.809964 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde27d8cc3eb4d09879e717fc90305fd" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.810649 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.810949 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 68a7051bc539468aafba30fbaf0669d4 (DEFAULT_TABLE table=default.Table_021124838f7841c49f01816fb948d73c [id=4a5d04ba5f3942a3942909ebd2bf5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.811609 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a7051bc539468aafba30fbaf0669d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:15.814765 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23f41a2638e74ba19c1c527b9fc796df consensus state has no leader
I20260811 18:57:15.818182 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:15.818683 4162 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.829174 3824 leader_election.cc:302] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:15.829936 3824 leader_election.cc:341] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:15.830193 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.833873 4658 raft_consensus.cc:2781] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.834360 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d00d943b778741b4aeec076ba46a9c40 (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.834980 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00d943b778741b4aeec076ba46a9c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.836705 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 with delete_type TABLET_DATA_DELETED (replaced by tablet bb56d23ccabc4b02961249312257d9c4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.836896 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 225) took 2193 ms. Trace:
I20260811 18:57:15.840718 4658 raft_consensus.cc:482] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:15.842557 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e8859199c3440ebb2b9664b65e0fb71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.848526 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.849548 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.850132 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3370
W20260811 18:57:15.850804 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71df3738a5cf4b12820dda131ced62e1 consensus state has no leader
I20260811 18:57:15.852005 3436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:15.852015 4167 rpcz_store.cc:276] 0811 18:57:13.608351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.608473 (+ 122us) service_pool.cc:227] Handling call
0811 18:57:15.257284 (+1648811us) ts_tablet_manager.cc:1145] Deleting tablet a0ba169c32ef48b984e649b211ee2af1
0811 18:57:15.257307 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.335251 (+ 77944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.432821 (+ 97570us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.801029 (+368208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1648348}
I20260811 18:57:15.852775 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 226) took 2228 ms. Trace:
I20260811 18:57:15.856858 4167 rpcz_store.cc:276] 0811 18:57:13.623956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.624038 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:15.852378 (+2228340us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:15.852393 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.852591 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2227921}
I20260811 18:57:15.857453 4167 tablet_replica.cc:330] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:15.858065 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e1efffade0f486d8d627e56e0be6ff6 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:15.858551 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f7d4c26da86b49b8a37f2289f4ef6614 consensus state has no leader
I20260811 18:57:15.860270 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1" } modify_external_catalogs: true
I20260811 18:57:15.860574 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1" } modify_external_catalogs: true
W20260811 18:57:15.861207 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bd39dcc0bed4623a06f00a56b740dba consensus state has no leader
I20260811 18:57:15.862002 3552 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:49260 (request call id 1) took 1936 ms. Trace:
I20260811 18:57:15.862218 3552 rpcz_store.cc:276] 0811 18:57:13.925736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.861423 (+1935687us) service_pool.cc:227] Handling call
0811 18:57:15.861628 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:15.862906 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8426ccbe530446fcbaa162a03cbb1892 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.872663 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4981573f7a434b5da77a06d251ea684c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.873124 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:15.873612 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4981573f7a434b5da77a06d251ea684c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.877954 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878362 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878690 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878931 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.881908 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a0ba169c32ef48b984e649b211ee2af1 (table default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:15.887876 4163 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.888243 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b35ea225e3114cdc9ed09c78af2f272d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.889127 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b35ea225e3114cdc9ed09c78af2f272d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.890074 4658 raft_consensus.cc:3037] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.908994 4658 raft_consensus.cc:504] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.911847 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" } modify_external_catalogs: true
I20260811 18:57:15.912096 4163 ts_tablet_manager.cc:1397] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.366s user 0.002s sys 0.008s
I20260811 18:57:15.912228 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" } modify_external_catalogs: true
I20260811 18:57:15.836540 4576 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.921749 4658 leader_election.cc:288] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.922492 4658 raft_consensus.cc:482] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.922902 4658 raft_consensus.cc:504] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.924512 4658 leader_election.cc:288] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.924588 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b9dec45633841ac830523612293aabf" } modify_external_catalogs: true
I20260811 18:57:15.924875 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b9dec45633841ac830523612293aabf" } modify_external_catalogs: true
I20260811 18:57:15.924957 4658 raft_consensus.cc:482] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.925170 4658 raft_consensus.cc:504] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.926136 4658 leader_election.cc:288] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.927659 4162 tablet_bootstrap.cc:654] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.933044 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.933532 3888 raft_consensus.cc:2454] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.939993 4800 raft_consensus.cc:482] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.940514 4800 raft_consensus.cc:504] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.942162 4800 leader_election.cc:288] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.949836 4163 raft_consensus.cc:348] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.950331 4163 raft_consensus.cc:374] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.950632 4163 raft_consensus.cc:729] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.953994 4163 consensus_queue.cc:260] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.955219 4163 ts_tablet_manager.cc:1428] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.043s user 0.003s sys 0.000s
I20260811 18:57:15.960588 4169 ts_tablet_manager.cc:1918] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.960868 4169 log.cc:1196] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/80cf2375b4c64dbfae4e20003545da60
I20260811 18:57:15.961946 4169 ts_tablet_manager.cc:1939] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:15.962718 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 248) took 1516 ms. Trace:
I20260811 18:57:15.963003 4169 rpcz_store.cc:276] 0811 18:57:14.446125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.446200 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:15.463816 (+1017616us) ts_tablet_manager.cc:1145] Deleting tablet 80cf2375b4c64dbfae4e20003545da60
0811 18:57:15.463829 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.744741 (+280912us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.948833 (+204092us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.962654 (+ 13821us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1017208}
I20260811 18:57:15.963747 4163 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.963726 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 80cf2375b4c64dbfae4e20003545da60 (table default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]) successfully deleted
I20260811 18:57:15.975049 4169 tablet_replica.cc:330] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:15.975553 4169 raft_consensus.cc:2229] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:15.975911 4169 raft_consensus.cc:2258] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.976817 4169 ts_tablet_manager.cc:1905] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.978128 4167 raft_consensus.cc:2229] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:15.978477 4576 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.980146 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd39dcc0bed4623a06f00a56b740dba (DEFAULT_TABLE table=default.Table_2f38c7848c3140fbac5000b908fe7eae [id=f312f8bcd459400b92a33532675d3991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.980304 4163 tablet_bootstrap.cc:654] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.980795 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd39dcc0bed4623a06f00a56b740dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.987360 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet e51a08e6dbf04e3fa58689ee0cd45db3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.978415 4740 consensus_queue.cc:1035] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.070s
I20260811 18:57:15.993281 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 059b325df1db4fde9d4b26b559121316 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.993819 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b325df1db4fde9d4b26b559121316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.999471 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c6377231db8495796c02e71f7f71991"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.000353 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c6377231db8495796c02e71f7f71991 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.003286 4163 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.003536 4163 ts_tablet_manager.cc:1397] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.006s
I20260811 18:57:16.005201 4163 raft_consensus.cc:348] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.005666 4163 raft_consensus.cc:374] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.016821 3553 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137] alter complete (version 2)
I20260811 18:57:16.019117 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" } modify_external_catalogs: true
I20260811 18:57:16.019440 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" } modify_external_catalogs: true
I20260811 18:57:16.020850 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" } modify_external_catalogs: true
I20260811 18:57:16.021136 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" } modify_external_catalogs: true
I20260811 18:57:16.023365 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0f6df7a291456d80538e2b76417873 (DEFAULT_TABLE table=default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.024147 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0f6df7a291456d80538e2b76417873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.018224 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_994d2d4469b54597a9aa33c59dca9d84" } new_table_name: "default.Table_51c66177bc494cd0a293e2959ff854e5" modify_external_catalogs: true
I20260811 18:57:16.026541 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e (DEFAULT_TABLE table=default.Table_994d2d4469b54597a9aa33c59dca9d84 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.026786 4739 consensus_queue.cc:1035] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.108s
I20260811 18:57:16.027352 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e544b4b1592c4d51a2ed0d6461edc75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.028558 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 (DEFAULT_TABLE table=default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.034554 3865 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c8d542ae2489ba466ffadd518db12 (DEFAULT_TABLE table=default.Table_5409ed17441242119ff946163da07a21 [id=97d17499bd684f0abf98e0582fa06f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.035338 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 (DEFAULT_TABLE table=default.Table_e746326e8bff404aa5b3bc1fac099489 [id=bebb3efbbc56416597abdf4a0b3952ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.036051 3856 tablet_service.cc:1467] Processing CreateTablet for tablet a847581412494e34b2c76e605ff11772 (DEFAULT_TABLE table=default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.036641 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 4814a3562ef14fb0bec79be8c3cff4d2 (DEFAULT_TABLE table=default.Table_858da7e74d4141a19ced6fdc29c33d89 [id=613069b1ba8145baa7e3614a2c5b59be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.037061 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 (DEFAULT_TABLE table=default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.038177 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.039359 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 (DEFAULT_TABLE table=default.Table_c37e3fb34b3f446d854006b05dbe5a94 [id=5fc5ade3f38c438d9b501127ac866b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.039497 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d (DEFAULT_TABLE table=default.Table_b07958bac6344da1bbf3d7d35f1ac551 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.040148 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7057b8a7bae249caaed304b5c66e6871 (DEFAULT_TABLE table=default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.041949 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5d495354c0994fb4a74c376078e41a26 (DEFAULT_TABLE table=default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.044173 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:16.045454 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 490f34d7a403416fbab64e46973746de from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.046118 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 799ba9a044e145ec89d0025c03f391ea from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.046635 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce94d37efd2245bf8cc71fdd90bd9735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047010 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7057b8a7bae249caaed304b5c66e6871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047389 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4814a3562ef14fb0bec79be8c3cff4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047499 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.047763 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d495354c0994fb4a74c376078e41a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047923 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.048143 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ceca57cb245f4a3a6cb4b36f1306d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.049048 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b404e99e9414c67b3cd5aff12203a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.049296 4167 raft_consensus.cc:2258] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.050314 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cccebfc54484a9b9abffcce4e459fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.058468 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cce97bfdb1d41e685e091c8c5cc6946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.058406 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce8db230c7e41b6a4792273a0f07e6e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:16.063184 3552 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 22071cd82e0645939e21f0be945e0eef
I20260811 18:57:16.064849 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0b3ec38f9b55489ba00ed76f9e4456f5
W20260811 18:57:16.064539 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 52402.4us
I20260811 18:57:16.066149 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f97f37fa7c483cacaeed54f93e7a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.069151 4163 raft_consensus.cc:729] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.069744 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_77c2e3d67a494888be8b3405d4a06cf2" } new_table_name: "default.Table_655dd0bc17cc45d3a344bb5c02ff7f80" modify_external_catalogs: true
I20260811 18:57:16.070389 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:16.071002 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0e1efffade0f486d8d627e56e0be6ff6 deleted and replaced by tablet 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:16.075160 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c8d542ae2489ba466ffadd518db12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.076524 3870 tablet_service.cc:1467] Processing CreateTablet for tablet faf0594d55844ae49c99247ecd8b810f (DEFAULT_TABLE table=default.Table_6d87a15f366a4e118c84e294c3b9926e [id=5a23d06641cb447fb634032bd768498b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.078080 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9fa42654cf4ec18220e5fb173ddef7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.080222 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847581412494e34b2c76e605ff11772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.082827 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.082736 4163 consensus_queue.cc:260] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.085798 4169 ts_tablet_manager.cc:1918] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:16.092288 3714 consensus_meta.cc:287] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent flushing consensus metadata: real 0.572s user 0.002s sys 0.003s
W20260811 18:57:16.098001 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 7892ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:16.103914 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1efffade0f486d8d627e56e0be6ff6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.107035 4167 ts_tablet_manager.cc:1905] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.111280 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae200f3814e431a8288c30fe94bffcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.112900 3706 tablet_service.cc:1467] Processing CreateTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 (DEFAULT_TABLE table=default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.115787 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 4814a3562ef14fb0bec79be8c3cff4d2 (DEFAULT_TABLE table=default.Table_858da7e74d4141a19ced6fdc29c33d89 [id=613069b1ba8145baa7e3614a2c5b59be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.116480 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0f6df7a291456d80538e2b76417873 (DEFAULT_TABLE table=default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.116737 3745 raft_consensus.cc:3037] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.083921 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.141305 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf0594d55844ae49c99247ecd8b810f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.142432 4169 log.cc:1196] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/807cee88c18340dbb23c51cad54ab56d
I20260811 18:57:16.143404 4169 ts_tablet_manager.cc:1939] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.143437 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0f6df7a291456d80538e2b76417873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.144353 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 with delete_type TABLET_DATA_DELETED (replaced by tablet 27cda53b7ca54346a7a22bf26f5db990) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.147993 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 256) took 1624 ms. Trace:
I20260811 18:57:16.148334 4169 rpcz_store.cc:276] 0811 18:57:14.521479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.522038 (+ 559us) service_pool.cc:227] Handling call
0811 18:57:15.974882 (+1452844us) ts_tablet_manager.cc:1145] Deleting tablet 807cee88c18340dbb23c51cad54ab56d
0811 18:57:15.974901 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.991220 (+ 16319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.085503 (+ 94283us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.145873 (+ 60370us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1452314}
I20260811 18:57:16.149242 4169 tablet_replica.cc:330] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.149734 4169 raft_consensus.cc:2229] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.150251 4169 raft_consensus.cc:2258] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.157544 4163 ts_tablet_manager.cc:1428] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.138s user 0.004s sys 0.001s
I20260811 18:57:16.160624 4739 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:16.166893 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 238706us
I20260811 18:57:16.168709 4169 ts_tablet_manager.cc:1905] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.182003 4797 raft_consensus.cc:482] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.182432 4797 raft_consensus.cc:504] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.184228 4163 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:16.189023 4797 leader_election.cc:288] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.163792 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 (DEFAULT_TABLE table=default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.175261 3718 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c8d542ae2489ba466ffadd518db12 (DEFAULT_TABLE table=default.Table_5409ed17441242119ff946163da07a21 [id=97d17499bd684f0abf98e0582fa06f5d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.161844 3678 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.269s (SENT)
I20260811 18:57:16.175555 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 (DEFAULT_TABLE table=default.Table_e746326e8bff404aa5b3bc1fac099489 [id=bebb3efbbc56416597abdf4a0b3952ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.197286 4739 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.174170 3705 tablet_service.cc:1467] Processing CreateTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e (DEFAULT_TABLE table=default.Table_994d2d4469b54597a9aa33c59dca9d84 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.201033 4739 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.203037 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce94d37efd2245bf8cc71fdd90bd9735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.212296 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_575c5aca059048d68252cca6583a1f67" } new_table_name: "default.Table_1b2c3c3b4a2948bbba5b4b556faf1015" modify_external_catalogs: true
I20260811 18:57:16.214608 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f97f37fa7c483cacaeed54f93e7a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.216109 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a847581412494e34b2c76e605ff11772 (DEFAULT_TABLE table=default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.219858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 (DEFAULT_TABLE table=default.Table_c37e3fb34b3f446d854006b05dbe5a94 [id=5fc5ade3f38c438d9b501127ac866b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.224730 4797 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.230674 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.227612 4799 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.235260 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e544b4b1592c4d51a2ed0d6461edc75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.232570 4799 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.240844 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 7057b8a7bae249caaed304b5c66e6871 (DEFAULT_TABLE table=default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.242141 4797 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.243325 4797 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.245826 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847581412494e34b2c76e605ff11772. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.251487 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44221.3us
#0 0x967ea3 in base::subtle::NoBarrier_Load(int const volatile*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/atomicops-internals-x86.h:200:10
I20260811 18:57:16.255136 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7057b8a7bae249caaed304b5c66e6871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.256587 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cccebfc54484a9b9abffcce4e459fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.258010 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cce97bfdb1d41e685e091c8c5cc6946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.261217 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5befe5c7d84b401e8323659cf14a3a17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.262097 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5befe5c7d84b401e8323659cf14a3a17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.268129 3745 raft_consensus.cc:2454] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:16.269307 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f62a01a774b4295a6dc3352d22d1ba4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.270452 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f62a01a774b4295a6dc3352d22d1ba4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.276307 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c8d542ae2489ba466ffadd518db12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.279762 4799 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.296530 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4814a3562ef14fb0bec79be8c3cff4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.300856 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:16.304724 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a81334e7da3741a9b24f8978312181b9
I20260811 18:57:16.305757 3891 raft_consensus.cc:2454] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:16.308293 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:16.308539 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d18dc31cde54577b8bcc329191b74ef" } new_table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" modify_external_catalogs: true
I20260811 18:57:16.309333 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_005d29915a3b4449acfc665e917be061" } new_table_name: "default.Table_94cadddacfc545b6af75e68ed02f34c1" modify_external_catalogs: true
W20260811 18:57:16.311510 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45fde25bcb40493aafb083e161aa6932 consensus state has no leader
I20260811 18:57:16.312011 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e51a08e6dbf04e3fa58689ee0cd45db3
I20260811 18:57:16.312876 3550 catalog_manager.cc:6565] ReplaceTablet: tablet e51a08e6dbf04e3fa58689ee0cd45db3 deleted and replaced by tablet 02799b864ad242f08fd89798c0fef4bf
I20260811 18:57:16.316053 4163 tablet_bootstrap.cc:654] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.316495 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.317281 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2058b5fb0d08491a9f2fa354276fc21b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.318223 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e9433d18ac7b46958942fc1f4b458354"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.318775 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
W20260811 18:57:16.318918 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9433d18ac7b46958942fc1f4b458354 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.319288 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7fc31b7c8d824f879807f105af51df48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.319895 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7fc31b7c8d824f879807f105af51df48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:16.321349 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:16.322000 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.322497 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5054093f72194ee8a604b8aa7fb6cf67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.323422 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:16.327219 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 799ba9a044e145ec89d0025c03f391ea deleted and replaced by tablet e1af33fd98c6486f975368ec90493db7
I20260811 18:57:16.332072 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef1abae384a9404089b0f6600e935a5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.332970 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef1abae384a9404089b0f6600e935a5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.335225 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dd7b9197d5d74b599220994b06e7830a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.335991 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd7b9197d5d74b599220994b06e7830a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.338006 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28b2e1b607a34ab2badc82e3c333a386"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.339003 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28b2e1b607a34ab2badc82e3c333a386 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:16.362550 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 266964us
I20260811 18:57:16.363533 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 807cee88c18340dbb23c51cad54ab56d (table default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]) successfully deleted
W20260811 18:57:16.365342 3824 leader_election.cc:341] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:16.373780 4803 log.cc:924] Time spent T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
I20260811 18:57:16.376866 3724 tablet_service.cc:1467] Processing CreateTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d (DEFAULT_TABLE table=default.Table_b07958bac6344da1bbf3d7d35f1ac551 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.388599 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b404e99e9414c67b3cd5aff12203a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.360265 3676 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:16.397364 3824 leader_election.cc:341] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:16.397998 3824 leader_election.cc:341] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.398448 3824 leader_election.cc:302] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:16.398816 4799 raft_consensus.cc:2781] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.399084 4799 raft_consensus.cc:686] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:16.399509 4799 consensus_queue.cc:237] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.400733 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf285cdf5235451bb1531cb294017101" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.401392 4739 raft_consensus.cc:2781] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.401582 4739 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.401762 4739 raft_consensus.cc:3037] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.410136 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ceca57cb245f4a3a6cb4b36f1306d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.411087 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.414824 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 117124us
I20260811 18:57:16.418111 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 02799b864ad242f08fd89798c0fef4bf
I20260811 18:57:16.422991 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f3fb616fae4249b4a043ddb879c7db57" } modify_external_catalogs: true
I20260811 18:57:16.423950 3861 tablet_service.cc:1467] Processing CreateTablet for tablet bb56d23ccabc4b02961249312257d9c4 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.425530 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45c439603b1140e998b5d9199026da49" } modify_external_catalogs: true
I20260811 18:57:16.425869 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45c439603b1140e998b5d9199026da49" } modify_external_catalogs: true
I20260811 18:57:16.424608 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f3fb616fae4249b4a043ddb879c7db57" } modify_external_catalogs: true
I20260811 18:57:16.428740 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet a81334e7da3741a9b24f8978312181b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.429047 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.428987 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:16.429910 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 473040a90e6941c38beb697316d2a27b has no consensus state
I20260811 18:57:16.430106 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d779d44f532f404cad704a689e72f1b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.431141 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b2718cd4a98742dfa8014a8a38c69c45 consensus state has no leader
W20260811 18:57:16.431806 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d779d44f532f404cad704a689e72f1b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.434716 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.437373 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (replaced by tablet 02799b864ad242f08fd89798c0fef4bf) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.438186 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.439306 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb56d23ccabc4b02961249312257d9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.414671 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5d495354c0994fb4a74c376078e41a26 (DEFAULT_TABLE table=default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.451838 3824 leader_election.cc:341] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.451839 4797 consensus_queue.cc:1035] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:16.458257 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.458932 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 with delete_type TABLET_DATA_DELETED (replaced by tablet 27cda53b7ca54346a7a22bf26f5db990) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:16.468437 3676 consensus_peers.cc:487] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807cee88c18340dbb23c51cad54ab56d. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:16.481539 4167 ts_tablet_manager.cc:1918] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.510494 4797 raft_consensus.cc:482] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.511442 4797 raft_consensus.cc:504] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.512590 4797 leader_election.cc:288] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.512748 4169 ts_tablet_manager.cc:1918] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.513032 4169 log.cc:1196] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/508f0b39cd654aa991a2928a4b067762
I20260811 18:57:16.513856 4169 ts_tablet_manager.cc:1939] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.514420 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 257) took 1992 ms. Trace:
I20260811 18:57:16.514581 4169 rpcz_store.cc:276] 0811 18:57:14.521734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.522645 (+ 911us) service_pool.cc:227] Handling call
0811 18:57:16.148838 (+1626193us) ts_tablet_manager.cc:1145] Deleting tablet 508f0b39cd654aa991a2928a4b067762
0811 18:57:16.148858 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.396710 (+247852us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.444751 (+ 48041us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.514281 (+ 69530us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1625678}
I20260811 18:57:16.515121 4169 tablet_replica.cc:330] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.515453 4169 raft_consensus.cc:2229] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.515688 4169 raft_consensus.cc:2258] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.516577 4169 ts_tablet_manager.cc:1905] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.521870 4163 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.522228 4163 ts_tablet_manager.cc:1397] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.364s user 0.005s sys 0.007s
I20260811 18:57:16.524169 4163 raft_consensus.cc:348] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.525245 4163 raft_consensus.cc:374] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.526067 4163 raft_consensus.cc:729] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.526538 4163 consensus_queue.cc:260] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.552806 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet c26851245cab4a2d80e94d4fe278fd50 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.555778 4163 ts_tablet_manager.cc:1428] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:16.485510 4739 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.485831 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d495354c0994fb4a74c376078e41a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.492626 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.492839 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet a81334e7da3741a9b24f8978312181b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.495676 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.506567 3716 tablet_service.cc:1467] Processing CreateTablet for tablet faf0594d55844ae49c99247ecd8b810f (DEFAULT_TABLE table=default.Table_6d87a15f366a4e118c84e294c3b9926e [id=5a23d06641cb447fb634032bd768498b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.509341 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:16.520027 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53202.5us
I20260811 18:57:16.527338 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (replaced by tablet 02799b864ad242f08fd89798c0fef4bf) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.527565 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.530315 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bb56d23ccabc4b02961249312257d9c4 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.538254 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.573338 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" } modify_external_catalogs: true
I20260811 18:57:16.573722 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_133a44e94783442e8c27d4730310f996" } modify_external_catalogs: true
I20260811 18:57:16.573808 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" } modify_external_catalogs: true
I20260811 18:57:16.574119 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_133a44e94783442e8c27d4730310f996" } modify_external_catalogs: true
I20260811 18:57:16.574906 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.575557 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b717724905d74a6a865cd6eb35c3a4d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.577358 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 508f0b39cd654aa991a2928a4b067762 (table default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]) successfully deleted
I20260811 18:57:16.573390 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9292c8eb06fb4a0d8e33b6796398b740" } new_table_name: "default.Table_647ae82e63464976abb7f26c9526a349" modify_external_catalogs: true
I20260811 18:57:16.586555 3676 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:16.601809 3676 consensus_peers.cc:487] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:16.613153 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.566743 4163 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:16.590197 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
W20260811 18:57:16.577457 3824 leader_election.cc:341] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.566256 4169 ts_tablet_manager.cc:1918] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.622728 4729 consensus_queue.cc:1035] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260811 18:57:16.624408 4739 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.624818 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb56d23ccabc4b02961249312257d9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.628427 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf0594d55844ae49c99247ecd8b810f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.632085 3890 raft_consensus.cc:3037] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:16.635375 3824 leader_election.cc:341] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.641379 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f8eb041f3d704762868a048faec0e0fc
I20260811 18:57:16.643877 4169 log.cc:1196] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ed3346bd12d04c45a511308c427856e5
I20260811 18:57:16.644385 4163 tablet_bootstrap.cc:654] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.644815 3890 raft_consensus.cc:2454] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:16.647977 4576 raft_consensus.cc:2781] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.647969 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8" } new_table_name: "default.Table_12d05bfe0d2348f0a14ef6b8ecf8414a" modify_external_catalogs: true
I20260811 18:57:16.648172 4576 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.649474 4167 log.cc:1196] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2852f9e5f67c4025836018af553388a5
I20260811 18:57:16.650022 4576 raft_consensus.cc:3037] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.650259 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd7b9197d5d74b599220994b06e7830a" } new_table_name: "default.Table_a15aed3f339749a7ab05c1774bc21112" modify_external_catalogs: true
I20260811 18:57:16.652334 4169 ts_tablet_manager.cc:1939] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.652145 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9f42592ab70540e59f42f7e97293a058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:16.652841 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 258) took 2130 ms. Trace:
I20260811 18:57:16.653007 4169 rpcz_store.cc:276] 0811 18:57:14.521960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.523214 (+ 1254us) service_pool.cc:227] Handling call
0811 18:57:16.515035 (+1991821us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:16.515051 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.559161 (+ 44110us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.566111 (+ 6950us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.652774 (+ 86663us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1991378}
W20260811 18:57:16.653137 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f42592ab70540e59f42f7e97293a058 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.653600 4169 tablet_replica.cc:330] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.653988 4169 raft_consensus.cc:2229] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.654678 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a30bb384136f4a169a5751655437d5a3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.656500 4162 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:16.656805 4162 ts_tablet_manager.cc:1397] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.877s user 0.012s sys 0.020s
W20260811 18:57:16.657828 3824 leader_election.cc:341] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.658957 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ed3346bd12d04c45a511308c427856e5 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
I20260811 18:57:16.659417 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:16.660856 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e1374a80d15444cb0faff339275aa8d consensus state has no leader
I20260811 18:57:16.661491 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cce97bfdb1d41e685e091c8c5cc6946 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.667182 4162 raft_consensus.cc:348] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.667932 4162 raft_consensus.cc:374] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.670575 4576 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.675948 4576 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.676529 4162 raft_consensus.cc:729] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.677043 4162 consensus_queue.cc:260] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.680051 4167 ts_tablet_manager.cc:1939] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:16.684051 4169 raft_consensus.cc:2258] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.689421 4169 ts_tablet_manager.cc:1905] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.690826 4162 ts_tablet_manager.cc:1428] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260811 18:57:16.692134 4163 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.692625 4163 ts_tablet_manager.cc:1397] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.136s user 0.008s sys 0.004s
W20260811 18:57:16.706261 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_655dd0bc17cc45d3a344bb5c02ff7f80 [id=f51f38f3582f44db99007a265b665fd4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:16.708043 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.710078 4162 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:16.713802 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 97780.7us
I20260811 18:57:16.694177 4163 raft_consensus.cc:348] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.714766 4163 raft_consensus.cc:374] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.714962 4163 raft_consensus.cc:729] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.715477 4163 consensus_queue.cc:260] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.716989 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 228) took 2991 ms. Trace:
I20260811 18:57:16.717715 4163 ts_tablet_manager.cc:1428] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.004s sys 0.000s
I20260811 18:57:16.718243 4163 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:16.730176 3676 consensus_peers.cc:487] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 508f0b39cd654aa991a2928a4b067762. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:16.738080 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a9efd2f19d74a68a8536fd057377738 consensus state has no leader
I20260811 18:57:16.738751 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" } modify_external_catalogs: true
I20260811 18:57:16.739086 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" } modify_external_catalogs: true
I20260811 18:57:16.743220 4163 tablet_bootstrap.cc:654] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.745519 4167 rpcz_store.cc:276] 0811 18:57:13.689015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.689089 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:15.857307 (+2168218us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:15.857324 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.434471 (+577147us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.481370 (+ 46899us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.680880 (+199510us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2167725}
I20260811 18:57:16.749091 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9f5c2bff2c124718a66c2f87c60ccbd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:16.749502 4169 ts_tablet_manager.cc:1918] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.749750 4169 log.cc:1196] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/151df87806694f46be5013592c68c01a
W20260811 18:57:16.749938 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f5c2bff2c124718a66c2f87c60ccbd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.751722 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2852f9e5f67c4025836018af553388a5 (table default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]) successfully deleted
W20260811 18:57:16.752960 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffa8a4a1c0cb4834be501bdd7d468472 has no consensus state
I20260811 18:57:16.753724 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b654b59f369e429097b918236d8ddd08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.754809 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b654b59f369e429097b918236d8ddd08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.761240 4162 tablet_bootstrap.cc:654] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.764739 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:16.765727 4169 ts_tablet_manager.cc:1939] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.765352 3554 catalog_manager.cc:5495] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:16.766125 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:16.766912 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 262) took 2006 ms. Trace:
I20260811 18:57:16.767135 4169 rpcz_store.cc:276] 0811 18:57:14.759982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.760065 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:16.653383 (+1893318us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:16.653396 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.703050 (+ 49654us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.749303 (+ 46253us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.766861 (+ 17558us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1892920}
I20260811 18:57:16.768282 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 151df87806694f46be5013592c68c01a (table default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]) successfully deleted
I20260811 18:57:16.771859 4163 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.772176 4163 ts_tablet_manager.cc:1397] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.054s user 0.006s sys 0.000s
I20260811 18:57:16.773095 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 2c29af3615e74722a809d3e5ede9b9a1 (DEFAULT_TABLE table=default.Table_9c6377231db8495796c02e71f7f71991 [id=dd46ddba2d9843a981969eb5db1d4cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.773329 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:16.773849 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c29af3615e74722a809d3e5ede9b9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.774096 4163 raft_consensus.cc:348] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.774513 4163 raft_consensus.cc:374] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.774778 4163 raft_consensus.cc:729] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.775252 4163 consensus_queue.cc:260] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.777853 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (DEFAULT_TABLE table=default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.778596 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4de110bd3348f0b0dae21c9f2b6a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.783131 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 452ee450778e4d33adfa7e2745b5d970 (DEFAULT_TABLE table=default.Table_d3e16ca699564a61a49c4cb989ad865f [id=48f45fcc06e0418185841260a06cb98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.783846 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452ee450778e4d33adfa7e2745b5d970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.786734 4169 tablet_replica.cc:330] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.787227 4169 raft_consensus.cc:2229] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:16.788592 3676 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:16.789373 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 0bfb54175c834517b7af291c942333cd (DEFAULT_TABLE table=default.Table_6bf41429faeb4785ad747d207e4641f8 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.790208 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bfb54175c834517b7af291c942333cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.793938 4729 raft_consensus.cc:2781] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.794258 4729 raft_consensus.cc:686] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:16.797688 4729 consensus_queue.cc:237] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:16.799795 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 58323.2us
I20260811 18:57:16.801889 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 230) took 2913 ms. Trace:
I20260811 18:57:16.801913 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e23d79bc476d496eaa713465bcfb1abf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.802780 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e23d79bc476d496eaa713465bcfb1abf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.803128 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (DEFAULT_TABLE table=default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.804598 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 0bfb54175c834517b7af291c942333cd (DEFAULT_TABLE table=default.Table_6bf41429faeb4785ad747d207e4641f8 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.807047 3722 tablet_service.cc:1467] Processing CreateTablet for tablet eb82243777a9494b96959ced0c24de5a (DEFAULT_TABLE table=default.Table_fd120e0ab3b34418960cefc5eb459cdb [id=3dc6eeef579941eaaa83be543d1c01d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.808159 3860 tablet_service.cc:1467] Processing CreateTablet for tablet d73a98bd35bd444195fd32fe6c67a04f (DEFAULT_TABLE table=default.Table_6a122fe2820f4d23b12edb970a94e958 [id=356820f47d4942d0a4cce16978925421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.808773 3713 tablet_service.cc:1467] Processing CreateTablet for tablet d73a98bd35bd444195fd32fe6c67a04f (DEFAULT_TABLE table=default.Table_6a122fe2820f4d23b12edb970a94e958 [id=356820f47d4942d0a4cce16978925421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.809031 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a98bd35bd444195fd32fe6c67a04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.810204 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_241ec7c366e84652a6d9c0123d2149ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.811141 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_241ec7c366e84652a6d9c0123d2149ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.811115 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 (DEFAULT_TABLE table=default.Table_b2d444511ad44775b701b9ed7527e251 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.811794 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c365f79d4a4201ac323f16bddf3e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.812639 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa (DEFAULT_TABLE table=default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.813078 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa (DEFAULT_TABLE table=default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.813711 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 (DEFAULT_TABLE table=default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.814774 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 (DEFAULT_TABLE table=default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.812186 3863 tablet_service.cc:1467] Processing CreateTablet for tablet eb82243777a9494b96959ced0c24de5a (DEFAULT_TABLE table=default.Table_fd120e0ab3b34418960cefc5eb459cdb [id=3dc6eeef579941eaaa83be543d1c01d1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.816428 4812 log.cc:924] Time spent T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260811 18:57:16.822003 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 452ee450778e4d33adfa7e2745b5d970 (DEFAULT_TABLE table=default.Table_d3e16ca699564a61a49c4cb989ad865f [id=48f45fcc06e0418185841260a06cb98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.822288 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:16.822674 3888 raft_consensus.cc:3037] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.822917 4167 rpcz_store.cc:276] 0811 18:57:13.852271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.852339 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:16.746736 (+2894397us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:16.746749 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.765231 (+ 18482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2893716}
W20260811 18:57:16.824663 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 867259e37fe44c33a35823ec2d896754 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 867259e37fe44c33a35823ec2d896754 already in progress: creating tablet
I20260811 18:57:16.824841 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4de110bd3348f0b0dae21c9f2b6a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.829545 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb82243777a9494b96959ced0c24de5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.831524 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19243ab3d6744aaa87a133f1462cd2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.832860 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 19243ab3d6744aaa87a133f1462cd2fa
I20260811 18:57:16.833977 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27cda53b7ca54346a7a22bf26f5db990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.833974 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_704e71bf00da49609103fb3fde6afb2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.834748 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_704e71bf00da49609103fb3fde6afb2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.835880 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a98bd35bd444195fd32fe6c67a04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.838873 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27cda53b7ca54346a7a22bf26f5db990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.838850 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 (DEFAULT_TABLE table=default.Table_b2d444511ad44775b701b9ed7527e251 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.839932 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.843851 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452ee450778e4d33adfa7e2745b5d970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.850556 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e18242f762d8407db0e2f8716890feb4 (DEFAULT_TABLE table=default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.850239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 229) took 2989 ms. Trace:
I20260811 18:57:16.851974 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.852300 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c365f79d4a4201ac323f16bddf3e62. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.853190 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.620s (SENT) [suppressed 9 similar messages]
I20260811 18:57:16.865577 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19243ab3d6744aaa87a133f1462cd2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.867250 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 473040a90e6941c38beb697316d2a27b (DEFAULT_TABLE table=default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.865331 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c0b355a0d3984012be4e9406998fd4a0 (DEFAULT_TABLE table=default.Table_7e8859199c3440ebb2b9664b65e0fb71 [id=aa5ff7c0d063424992f9dc22c89aa620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.868180 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 2599dbd65a814601928373ab9910e3ca (DEFAULT_TABLE table=default.Table_b35ea225e3114cdc9ed09c78af2f272d [id=3a1d54898d9b4f918ab8ea53a0f1bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.876839 4169 pending_rounds.cc:62] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:16.882097 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18242f762d8407db0e2f8716890feb4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.895574 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6026ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:16.900357 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2599dbd65a814601928373ab9910e3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.914697 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b355a0d3984012be4e9406998fd4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.877903 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 2c29af3615e74722a809d3e5ede9b9a1 (DEFAULT_TABLE table=default.Table_9c6377231db8495796c02e71f7f71991 [id=dd46ddba2d9843a981969eb5db1d4cea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.915635 3361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3370: Timed out: CheckLeaks RPC to 127.3.72.126:34821 timed out after 1.000s (SENT)
I20260811 18:57:16.915947 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3370
I20260811 18:57:16.869776 4167 rpcz_store.cc:276] 0811 18:57:13.851791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.852598 (+ 807us) service_pool.cc:227] Handling call
0811 18:57:16.835322 (+2982724us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:16.835339 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.841095 (+ 5756us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2982157}
I20260811 18:57:16.917130 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473040a90e6941c38beb697316d2a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.918853 3723 tablet_service.cc:1467] Processing CreateTablet for tablet e18242f762d8407db0e2f8716890feb4 (DEFAULT_TABLE table=default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.920295 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:16.920491 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb82243777a9494b96959ced0c24de5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.921787 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:16.922808 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:16.923664 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 8cce97bfdb1d41e685e091c8c5cc6946 deleted and replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa
I20260811 18:57:16.926376 4169 pending_rounds.cc:68] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106077646848 op_type: NO_OP noop_request { }
I20260811 18:57:16.926738 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c0b355a0d3984012be4e9406998fd4a0 (DEFAULT_TABLE table=default.Table_7e8859199c3440ebb2b9664b65e0fb71 [id=aa5ff7c0d063424992f9dc22c89aa620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.927363 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 2599dbd65a814601928373ab9910e3ca (DEFAULT_TABLE table=default.Table_b35ea225e3114cdc9ed09c78af2f272d [id=3a1d54898d9b4f918ab8ea53a0f1bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.929191 4167 tablet_replica.cc:330] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:16.929527 4167 raft_consensus.cc:2229] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.929883 4167 raft_consensus.cc:2258] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.930706 4167 ts_tablet_manager.cc:1905] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.935238 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 473040a90e6941c38beb697316d2a27b (DEFAULT_TABLE table=default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.936170 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bfb54175c834517b7af291c942333cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.936776 4169 raft_consensus.cc:2866] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260811 18:57:16.939486 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a902ad70a25546be926ff8f7bf2ed64f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a902ad70a25546be926ff8f7bf2ed64f already in progress: creating tablet
I20260811 18:57:16.942937 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.943414 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.943672 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.951963 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_021124838f7841c49f01816fb948d73c" } new_table_name: "default.Table_27d36ce508354815add475acc3f872eb" modify_external_catalogs: true
I20260811 18:57:16.953199 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:16.953888 3549 catalog_manager.cc:6565] ReplaceTablet: tablet c26851245cab4a2d80e94d4fe278fd50 deleted and replaced by tablet 4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:16.961086 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:16.962442 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c6e0d1eeb63493bbea4cd78dd085a3c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.965147 4169 raft_consensus.cc:2258] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.965246 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 968fa8b8912b40a8bdba77db1d39c5c0 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.969695 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.970023 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2599dbd65a814601928373ab9910e3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.974848 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:16.977389 4818 log.cc:924] Time spent T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260811 18:57:16.977720 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 987b9ac6859444e3a86c56af786cc84e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.982048 4169 ts_tablet_manager.cc:1905] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:16.982422 3678 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.013s (SENT)
I20260811 18:57:16.982672 3678 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:16.986173 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 50458.7us
I20260811 18:57:16.987360 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c29af3615e74722a809d3e5ede9b9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.988545 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18242f762d8407db0e2f8716890feb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.989219 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b355a0d3984012be4e9406998fd4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.990090 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473040a90e6941c38beb697316d2a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.991477 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3cc27811ad29451daaebbf31fb3ad198"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.992157 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cc27811ad29451daaebbf31fb3ad198 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.992540 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f" } modify_external_catalogs: true
I20260811 18:57:16.993521 3888 raft_consensus.cc:2454] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:16.993988 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f" } modify_external_catalogs: true
I20260811 18:57:16.998663 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5befe5c7d84b401e8323659cf14a3a17" } new_table_name: "default.Table_c86ddee4faa543c79b02e248ac801fb3" modify_external_catalogs: true
I20260811 18:57:17.000433 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.001080 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d490ceda4c745668e9deb77c4a07fc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.010648 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:17.017896 4163 ts_tablet_manager.cc:1428] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.246s user 0.005s sys 0.000s
I20260811 18:57:17.018985 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b227bc7305f4b2dae28f04e7001553b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.022143 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6533d74b53c54546bdd082863b38257c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.022460 4163 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:17.022951 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6533d74b53c54546bdd082863b38257c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.024914 4163 tablet_bootstrap.cc:654] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.024923 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5f4d9f42e99c454e88964d343262e534"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.026041 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f4d9f42e99c454e88964d343262e534 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.029412 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68a7051bc539468aafba30fbaf0669d4 consensus state has no leader
I20260811 18:57:17.030081 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_457be7b012ba40deab01ec6480b82ede" } new_table_name: "default.Table_3c807c78cf1249abbf878e819573553d" modify_external_catalogs: true
I20260811 18:57:17.046979 4815 raft_consensus.cc:2726] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:17.065057 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:17.050594 3824 leader_election.cc:334] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.017s (SENT)
I20260811 18:57:17.068308 3824 leader_election.cc:302] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:17.068620 3824 raft_consensus.cc:2670] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:17.083979 4169 ts_tablet_manager.cc:1918] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.086759 4162 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.090675 4162 ts_tablet_manager.cc:1397] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.393s user 0.010s sys 0.002s
I20260811 18:57:17.091281 4169 log.cc:1196] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/33979de5b48c4674a46c8da3b7151753
I20260811 18:57:17.092073 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.078861 4163 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.092720 4163 ts_tablet_manager.cc:1397] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.005s sys 0.000s
I20260811 18:57:17.094066 4163 raft_consensus.cc:348] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.094867 4163 raft_consensus.cc:374] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.095870 4163 raft_consensus.cc:729] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.096522 4169 ts_tablet_manager.cc:1939] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.096622 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b227bc7305f4b2dae28f04e7001553b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.097187 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.097419 4163 consensus_queue.cc:260] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.105831 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 263) took 2312 ms. Trace:
I20260811 18:57:17.106745 4811 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.106958 4811 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.107745 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.107960 4811 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.105330 4163 ts_tablet_manager.cc:1428] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260811 18:57:17.109196 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.110363 3676 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.109295 4163 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.106417 4169 rpcz_store.cc:276] 0811 18:57:14.793877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.793950 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:16.786633 (+1992683us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:16.786648 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.063263 (+276615us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.083771 (+ 20508us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.103008 (+ 19237us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14349,"tablet-delete.queue_time_us":1992042}
I20260811 18:57:17.112529 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 33979de5b48c4674a46c8da3b7151753 (table default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]) successfully deleted
I20260811 18:57:17.112883 4169 tablet_replica.cc:330] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.113886 4169 raft_consensus.cc:2229] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.115831 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.116180 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.116426 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.117376 3824 leader_election.cc:341] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.118753 4169 raft_consensus.cc:2258] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.122947 4162 raft_consensus.cc:348] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.125545 4815 raft_consensus.cc:2781] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.126389 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.127413 4162 raft_consensus.cc:374] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.127590 4162 raft_consensus.cc:729] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.130231 4162 consensus_queue.cc:260] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.131098 4162 ts_tablet_manager.cc:1428] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
I20260811 18:57:17.131872 4815 raft_consensus.cc:686] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.133136 4162 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.132321 4815 consensus_queue.cc:237] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.145714 4829 tablet.cc:1722] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:17.148610 4169 ts_tablet_manager.cc:1905] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.155660 4163 tablet_bootstrap.cc:654] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.156774 4167 ts_tablet_manager.cc:1918] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.176676 4167 log.cc:1196] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:17.177388 4162 tablet_bootstrap.cc:654] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.177742 4163 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.178392 4163 ts_tablet_manager.cc:1397] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.000s
I20260811 18:57:17.181421 4163 raft_consensus.cc:348] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.184401 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 has no consensus state
I20260811 18:57:17.186439 4163 raft_consensus.cc:374] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.187003 4163 raft_consensus.cc:729] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.187604 4163 consensus_queue.cc:260] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.188642 4167 ts_tablet_manager.cc:1939] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.189534 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 233) took 3332 ms. Trace:
I20260811 18:57:17.191222 4167 rpcz_store.cc:276] 0811 18:57:13.857415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.860602 (+ 3187us) service_pool.cc:227] Handling call
0811 18:57:16.916222 (+3055620us) ts_tablet_manager.cc:1145] Deleting tablet 8af3e74889b64c408c9ed9c54916f890
0811 18:57:16.916234 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.082066 (+165832us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.139745 (+ 57679us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.189486 (+ 49741us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3055186}
I20260811 18:57:17.191849 4167 tablet_replica.cc:330] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:17.192533 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8af3e74889b64c408c9ed9c54916f890 (table default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]) successfully deleted
I20260811 18:57:17.193099 4167 raft_consensus.cc:2229] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:17.193574 4167 pending_rounds.cc:62] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:17.195363 4163 ts_tablet_manager.cc:1428] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260811 18:57:17.199059 4163 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.199108 4167 pending_rounds.cc:68] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400104139776000 op_type: NO_OP noop_request { }
I20260811 18:57:17.199554 4167 raft_consensus.cc:2866] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:17.203006 4167 raft_consensus.cc:2258] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.203881 4167 ts_tablet_manager.cc:1905] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.214500 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f5c2bff2c124718a66c2f87c60ccbd3" } new_table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" modify_external_catalogs: true
I20260811 18:57:17.216441 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.216744 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_66613580530e4ae8989a795dd5d5ef69" } new_table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" modify_external_catalogs: true
W20260811 18:57:17.217166 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.219020 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55c1a7d44717415ca76a2d0b7e94394e consensus state has no leader
I20260811 18:57:17.219591 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4927a23edf074354bdd4cf94e4b74eff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.220117 4163 tablet_bootstrap.cc:654] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:17.220292 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4927a23edf074354bdd4cf94e4b74eff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.230631 4169 ts_tablet_manager.cc:1918] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.231786 4169 log.cc:1196] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9f8c0444ec2242299482736a73551f80
W20260811 18:57:17.232758 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 42405aec21e3405486b6c658438b5fb2 has no consensus state
I20260811 18:57:17.233359 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.234020 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a80fc1f336804bea9495ec0bce4bd47a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.234431 4163 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.234593 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_2bb9c598e2de422b86a8304bbece96db [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 1 got 0
I20260811 18:57:17.234810 4163 ts_tablet_manager.cc:1397] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
I20260811 18:57:17.235877 4163 raft_consensus.cc:348] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.236315 4163 raft_consensus.cc:374] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.236398 4163 raft_consensus.cc:729] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.236315 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_46eb50be61c143f2930a20efd2cd56b8" } modify_external_catalogs: true
I20260811 18:57:17.236560 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_46eb50be61c143f2930a20efd2cd56b8" } modify_external_catalogs: true
I20260811 18:57:17.240893 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:17.241160 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
W20260811 18:57:17.241952 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a443fe209556489abb47f4faa5d696e5 consensus state has no leader
I20260811 18:57:17.242385 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a108c0f554824b94acc46ae9330d1115" } modify_external_catalogs: true
I20260811 18:57:17.242589 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a108c0f554824b94acc46ae9330d1115" } modify_external_catalogs: true
I20260811 18:57:17.242707 4163 consensus_queue.cc:260] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.236688 4169 ts_tablet_manager.cc:1939] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.238497 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.238705 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.243922 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 265) took 2410 ms. Trace:
I20260811 18:57:17.244237 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:17.244118 4169 rpcz_store.cc:276] 0811 18:57:14.833448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833521 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:17.112572 (+2279051us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:17.112585 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.153756 (+ 41171us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.230436 (+ 76680us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.243853 (+ 13417us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2278745}
I20260811 18:57:17.244664 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9f8c0444ec2242299482736a73551f80 (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:17.244900 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 264) took 2411 ms. Trace:
I20260811 18:57:17.245033 4169 rpcz_store.cc:276] 0811 18:57:14.833276 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833733 (+ 457us) service_pool.cc:227] Handling call
0811 18:57:17.244670 (+2410937us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:17.244686 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.244829 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2410655}
I20260811 18:57:17.245239 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:17.246861 3864 tablet_service.cc:1467] Processing CreateTablet for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (DEFAULT_TABLE table=default.Table_b654b59f369e429097b918236d8ddd08 [id=1323bd99594a4382ba121cc11e79de31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.247479 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.251286 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 42405aec21e3405486b6c658438b5fb2 (DEFAULT_TABLE table=default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.251806 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42405aec21e3405486b6c658438b5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.254792 3862 tablet_service.cc:1467] Processing CreateTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 (DEFAULT_TABLE table=default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.255741 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8f5b87062349c4b19ec99fa0175f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.258652 3861 tablet_service.cc:1467] Processing CreateTablet for tablet f75dc515d7f3473b8e43f20255a70e3e (DEFAULT_TABLE table=default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.261547 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f75dc515d7f3473b8e43f20255a70e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.264439 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc07c28ccfd48ee9c33e49442c6047b (DEFAULT_TABLE table=default.Table_d779d44f532f404cad704a689e72f1b3 [id=cabc6cc20754488eb152c0d4f686dce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.264973 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc07c28ccfd48ee9c33e49442c6047b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.268030 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b591f65f304846aab2374c6ada0dd0ec" } modify_external_catalogs: true
I20260811 18:57:17.268342 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b591f65f304846aab2374c6ada0dd0ec" } modify_external_catalogs: true
I20260811 18:57:17.270187 3716 tablet_service.cc:1467] Processing CreateTablet for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (DEFAULT_TABLE table=default.Table_b654b59f369e429097b918236d8ddd08 [id=1323bd99594a4382ba121cc11e79de31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.270799 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.272143 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef613c8dc0524f318790523ba7652317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.272945 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef613c8dc0524f318790523ba7652317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.277019 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 42405aec21e3405486b6c658438b5fb2 (DEFAULT_TABLE table=default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.277539 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42405aec21e3405486b6c658438b5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.280438 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d37e81d376e14907b13746cca831a96b (DEFAULT_TABLE table=default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.281710 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37e81d376e14907b13746cca831a96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.282807 4162 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.283082 4162 ts_tablet_manager.cc:1397] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.152s user 0.004s sys 0.003s
I20260811 18:57:17.284245 4162 raft_consensus.cc:348] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.286705 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a811d4488d0e4541808a5e1aa44d663b (DEFAULT_TABLE table=default.Table_7fc31b7c8d824f879807f105af51df48 [id=a71b71af19a44182aea7e742062558ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.288004 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a811d4488d0e4541808a5e1aa44d663b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.291543 3871 tablet_service.cc:1467] Processing CreateTablet for tablet d73a43558dbb48889ab205108f4ce6be (DEFAULT_TABLE table=default.Table_e9433d18ac7b46958942fc1f4b458354 [id=d818dd73be204b458c789ff1d10f7b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.292455 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a43558dbb48889ab205108f4ce6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.298440 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (DEFAULT_TABLE table=default.Table_c86ddee4faa543c79b02e248ac801fb3 [id=7a869d9623bc4e07938b2d26688723a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.298882 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.300468 3860 tablet_service.cc:1467] Processing CreateTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.300882 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa8a4a1c0cb4834be501bdd7d468472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.309458 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.310956 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8304ac7709ca4392baec4cfa78ef5b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.285526 3710 tablet_service.cc:1467] Processing CreateTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 (DEFAULT_TABLE table=default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.314837 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8f5b87062349c4b19ec99fa0175f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.285928 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f75dc515d7f3473b8e43f20255a70e3e (DEFAULT_TABLE table=default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.318887 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f75dc515d7f3473b8e43f20255a70e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.288858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet d37e81d376e14907b13746cca831a96b (DEFAULT_TABLE table=default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.322700 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37e81d376e14907b13746cca831a96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.288802 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc07c28ccfd48ee9c33e49442c6047b (DEFAULT_TABLE table=default.Table_d779d44f532f404cad704a689e72f1b3 [id=cabc6cc20754488eb152c0d4f686dce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.326682 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc07c28ccfd48ee9c33e49442c6047b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.290843 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a811d4488d0e4541808a5e1aa44d663b (DEFAULT_TABLE table=default.Table_7fc31b7c8d824f879807f105af51df48 [id=a71b71af19a44182aea7e742062558ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.330935 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a811d4488d0e4541808a5e1aa44d663b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.334203 3870 tablet_service.cc:1467] Processing CreateTablet for tablet e1af33fd98c6486f975368ec90493db7 (DEFAULT_TABLE table=default.Table_34c1bb6042d54870bb236a2204ce71ed [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.334769 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1af33fd98c6486f975368ec90493db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.339053 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 38e24f8d4522466d81605ea37fa7064a (DEFAULT_TABLE table=default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.339640 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e24f8d4522466d81605ea37fa7064a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.353374 4162 raft_consensus.cc:374] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.353554 4162 raft_consensus.cc:729] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.353840 4162 consensus_queue.cc:260] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.356462 4162 ts_tablet_manager.cc:1428] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.073s user 0.002s sys 0.000s
I20260811 18:57:17.356819 4162 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.295095 4815 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.359694 4815 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.361088 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:17.361356 3888 raft_consensus.cc:2454] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:17.361824 3676 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.362473 4576 raft_consensus.cc:2781] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:17.362625 4576 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:17.362731 4576 raft_consensus.cc:3037] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.365971 4576 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.361522 4815 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.367650 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:17.367918 3888 raft_consensus.cc:3037] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.371268 3888 raft_consensus.cc:2454] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:17.371795 3676 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.294827 4828 tablet.cc:1722] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:17.296110 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (DEFAULT_TABLE table=default.Table_c86ddee4faa543c79b02e248ac801fb3 [id=7a869d9623bc4e07938b2d26688723a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.372954 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.373303 3861 tablet_service.cc:1467] Processing CreateTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 (DEFAULT_TABLE table=default.Table_ef1abae384a9404089b0f6600e935a5f [id=73a97b63e5db407a94ff739f295cc859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.373836 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4dc2df4c6464d71bb13fe25e0046758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.300797 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.376993 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8304ac7709ca4392baec4cfa78ef5b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.377282 3860 tablet_service.cc:1467] Processing CreateTablet for tablet edd393d9c4d24aab8a170e1329400656 (DEFAULT_TABLE table=default.Table_28b2e1b607a34ab2badc82e3c333a386 [id=2699d23349ac49f0b9a0b92dbab4e3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.377851 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd393d9c4d24aab8a170e1329400656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.381289 4834 raft_consensus.cc:2781] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.382160 4169 tablet_replica.cc:330] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.382483 4169 raft_consensus.cc:2229] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.306644 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.307274 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 (DEFAULT_TABLE table=default.Table_ef1abae384a9404089b0f6600e935a5f [id=73a97b63e5db407a94ff739f295cc859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.388381 4811 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.388607 4811 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.389472 4811 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.390733 4811 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.390919 4811 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.391202 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4687b95728fc485fa0e842bc447eb83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4687b95728fc485fa0e842bc447eb83f already in progress: creating tablet
I20260811 18:57:17.391515 4811 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.392371 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4dc2df4c6464d71bb13fe25e0046758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.393671 4811 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.393843 4811 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.394662 4811 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.398754 4169 raft_consensus.cc:2258] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.399091 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa8a4a1c0cb4834be501bdd7d468472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.400015 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.403847 3824 leader_election.cc:341] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.403198 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.404759 4837 consensus_queue.cc:1035] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:17.405395 4576 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.405364 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.399639 4163 ts_tablet_manager.cc:1428] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.165s user 0.000s sys 0.003s
I20260811 18:57:17.406525 4163 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.412838 4163 tablet_bootstrap.cc:654] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.312175 3723 tablet_service.cc:1467] Processing CreateTablet for tablet d73a43558dbb48889ab205108f4ce6be (DEFAULT_TABLE table=default.Table_e9433d18ac7b46958942fc1f4b458354 [id=d818dd73be204b458c789ff1d10f7b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.419502 4834 raft_consensus.cc:686] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.426565 4834 consensus_queue.cc:237] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.441962 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a43558dbb48889ab205108f4ce6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.407384 4169 ts_tablet_manager.cc:1905] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.426561 3824 leader_election.cc:341] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:17.449270 3824 leader_election.cc:341] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.439671 3706 tablet_service.cc:1467] Processing CreateTablet for tablet edd393d9c4d24aab8a170e1329400656 (DEFAULT_TABLE table=default.Table_28b2e1b607a34ab2badc82e3c333a386 [id=2699d23349ac49f0b9a0b92dbab4e3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.454698 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 38e24f8d4522466d81605ea37fa7064a (DEFAULT_TABLE table=default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.461164 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e1af33fd98c6486f975368ec90493db7 (DEFAULT_TABLE table=default.Table_34c1bb6042d54870bb236a2204ce71ed [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.478238 4163 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.480417 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd393d9c4d24aab8a170e1329400656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.481036 4163 ts_tablet_manager.cc:1397] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.001s
I20260811 18:57:17.484153 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1af33fd98c6486f975368ec90493db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.485958 4163 raft_consensus.cc:348] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.489372 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e24f8d4522466d81605ea37fa7064a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.493955 4169 ts_tablet_manager.cc:1918] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.505352 4163 raft_consensus.cc:374] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.505528 4163 raft_consensus.cc:729] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.505607 4169 log.cc:1196] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:17.505860 4163 consensus_queue.cc:260] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.506520 4169 ts_tablet_manager.cc:1939] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.506734 4163 ts_tablet_manager.cc:1428] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260811 18:57:17.507102 4163 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.508490 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:17.508790 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 266) took 2675 ms. Trace:
I20260811 18:57:17.509109 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 0c6e0d1eeb63493bbea4cd78dd085a3c deleted and replaced by tablet b203df4124364737839bf4c1ae072988
I20260811 18:57:17.509017 4169 rpcz_store.cc:276] 0811 18:57:14.833664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833906 (+ 242us) service_pool.cc:227] Handling call
0811 18:57:17.382031 (+2548125us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:17.382053 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.453737 (+ 71684us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.493765 (+ 40028us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.508739 (+ 14974us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15932,"tablet-delete.queue_time_us":2547734}
I20260811 18:57:17.509757 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 267) took 2506 ms. Trace:
I20260811 18:57:17.509898 4169 rpcz_store.cc:276] 0811 18:57:15.003529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.003605 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:17.509562 (+2505957us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:17.509578 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.509721 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2499269}
I20260811 18:57:17.510425 4169 tablet_replica.cc:330] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.510697 4169 raft_consensus.cc:2229] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.511020 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b1ce1042ae974f4dbac964bd7ea16a94 (table default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]) successfully deleted
W20260811 18:57:17.511279 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ff1be026ca404eada53f60f257ea367c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff1be026ca404eada53f60f257ea367c already in progress: creating tablet
I20260811 18:57:17.511299 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.511469 4169 raft_consensus.cc:2258] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.513587 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c18120da2d245f495c52a670047760d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.513669 4169 ts_tablet_manager.cc:1905] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.508924 4163 tablet_bootstrap.cc:654] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.530763 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:17.531634 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ba1e7c81d1bf4209b7089639928742f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.532305 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba1e7c81d1bf4209b7089639928742f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.534405 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.535074 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.537292 4169 ts_tablet_manager.cc:1918] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.537911 4169 log.cc:1196] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:17.538509 4169 ts_tablet_manager.cc:1939] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.539394 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 268) took 2442 ms. Trace:
I20260811 18:57:17.540277 4169 rpcz_store.cc:276] 0811 18:57:15.096785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.096854 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:17.510327 (+2413473us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:17.510370 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.518234 (+ 7864us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.537149 (+ 18915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.539320 (+ 2171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2412993}
W20260811 18:57:17.540405 3823 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:17.541034 3540 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:17.541244 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 9 similar messages]
I20260811 18:57:17.541302 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a300a7aca2814da6b999eb4d865fea66 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:17.541519 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260811 18:57:17.541569 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 269) took 2346 ms. Trace:
I20260811 18:57:17.541674 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:17.541684 4169 rpcz_store.cc:276] 0811 18:57:15.195062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.195145 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:17.541389 (+2346244us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:17.541403 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.541514 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2345657}
W20260811 18:57:17.542163 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fbba1923ce0049448fe4e28f2dd258dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbba1923ce0049448fe4e28f2dd258dc already in progress: creating tablet
I20260811 18:57:17.542802 4169 tablet_replica.cc:330] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:17.549517 3677 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:17.555217 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.556661 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.556949 4162 tablet_bootstrap.cc:654] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.560159 4163 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.560523 4163 ts_tablet_manager.cc:1397] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.053s user 0.005s sys 0.002s
I20260811 18:57:17.561740 4163 raft_consensus.cc:348] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.562259 4163 raft_consensus.cc:374] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.562445 4163 raft_consensus.cc:729] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.562848 4163 consensus_queue.cc:260] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.563426 4169 raft_consensus.cc:2229] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.566402 4169 raft_consensus.cc:2258] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.567199 4163 ts_tablet_manager.cc:1428] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:17.570449 4163 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.571231 4169 ts_tablet_manager.cc:1905] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.578006 4848 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.553023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.3.72.65:56675)
0811 18:57:17.577456 (+ 24433us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.577652 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":24254,"threads_started":1}
W20260811 18:57:17.589577 4849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.560625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.0.0.1:39266)
0811 18:57:17.588826 (+ 28201us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.589079 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":27985,"threads_started":1}
I20260811 18:57:17.598001 4163 tablet_bootstrap.cc:654] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:17.598505 4846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.543633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.3.72.66:51551)
0811 18:57:17.597611 (+ 53978us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.597885 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53795,"threads_started":1}
I20260811 18:57:17.600116 4167 ts_tablet_manager.cc:1918] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.600383 4167 log.cc:1196] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/807cee88c18340dbb23c51cad54ab56d
I20260811 18:57:17.602214 4167 ts_tablet_manager.cc:1939] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.603032 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 235) took 3649 ms. Trace:
I20260811 18:57:17.605762 4167 rpcz_store.cc:276] 0811 18:57:13.953621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.954017 (+ 396us) service_pool.cc:227] Handling call
0811 18:57:17.191770 (+3237753us) ts_tablet_manager.cc:1145] Deleting tablet 807cee88c18340dbb23c51cad54ab56d
0811 18:57:17.191784 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.455348 (+263564us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.598331 (+142983us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.602982 (+ 4651us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3237160}
I20260811 18:57:17.611608 4169 ts_tablet_manager.cc:1918] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:17.611771 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:17.612999 4163 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.613301 4163 ts_tablet_manager.cc:1397] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.003s
I20260811 18:57:17.613664 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 807cee88c18340dbb23c51cad54ab56d (table default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]) successfully deleted
I20260811 18:57:17.614270 4163 raft_consensus.cc:348] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.614717 4163 raft_consensus.cc:374] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.614884 4163 raft_consensus.cc:729] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.615218 4163 consensus_queue.cc:260] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.622715 4163 ts_tablet_manager.cc:1428] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:17.627497 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 237) took 3567 ms. Trace:
I20260811 18:57:17.629243 4811 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.629480 4811 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.630479 4811 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.634554 4163 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.636646 4163 tablet_bootstrap.cc:654] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.637943 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:17.638609 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:17.639076 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 968fa8b8912b40a8bdba77db1d39c5c0 deleted and replaced by tablet f287040b879642448367abe2b1c73cd9
I20260811 18:57:17.640048 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:17.640477 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 987b9ac6859444e3a86c56af786cc84e deleted and replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104
I20260811 18:57:17.640695 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.640774 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet d2d7d986900244ddbc05dbe73a708dcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.636878 4169 log.cc:1196] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/dde27d8cc3eb4d09879e717fc90305fd
I20260811 18:57:17.642307 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 987b9ac6859444e3a86c56af786cc84e with delete_type TABLET_DATA_DELETED (replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.642686 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.642961 4811 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.643177 4811 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.644061 4811 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.644886 4169 ts_tablet_manager.cc:1939] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.645633 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 270) took 2421 ms. Trace:
I20260811 18:57:17.645810 4169 rpcz_store.cc:276] 0811 18:57:15.224444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.224503 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:17.542714 (+2318211us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:17.542723 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.585119 (+ 42396us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.611413 (+ 26294us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.645588 (+ 34175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2275,"tablet-delete.queue_time_us":2317907}
I20260811 18:57:17.647040 4169 tablet_replica.cc:330] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.647380 4169 raft_consensus.cc:2229] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.647622 4169 raft_consensus.cc:2258] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.648427 4169 ts_tablet_manager.cc:1905] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.631584 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.633876 4739 consensus_queue.cc:1035] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:17.657620 3546 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 2dcfbf6f46e34163a230c0c82df7ecfa
I20260811 18:57:17.658651 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:17.660106 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.663522 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9c18120da2d245f495c52a670047760d
I20260811 18:57:17.664021 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 9c18120da2d245f495c52a670047760d deleted and replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba
I20260811 18:57:17.665032 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c18120da2d245f495c52a670047760d with delete_type TABLET_DATA_DELETED (replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.665721 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b2d444511ad44775b701b9ed7527e251" } new_table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" modify_external_catalogs: true
I20260811 18:57:17.666075 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5120347318f64721a3d6ef8cdc5564cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.666663 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8" } modify_external_catalogs: true
I20260811 18:57:17.666859 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8" } modify_external_catalogs: true
I20260811 18:57:17.668159 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e2b67d7e59584a2bb877f79bdafed3a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.669376 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4efc43ae63254c67aac88d7bd9b6c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.669750 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d779d44f532f404cad704a689e72f1b3" } new_table_name: "default.Table_7de254c88a674097b5df7ea987172a9e" modify_external_catalogs: true
W20260811 18:57:17.671533 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.672057 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2b67d7e59584a2bb877f79bdafed3a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.673887 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5120347318f64721a3d6ef8cdc5564cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.682821 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24955fa6843f46dc883be60173049a16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.683917 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24955fa6843f46dc883be60173049a16 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.687554 4163 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.687769 4163 ts_tablet_manager.cc:1397] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.003s
I20260811 18:57:17.689112 4163 raft_consensus.cc:348] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.689465 4163 raft_consensus.cc:374] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.689560 4163 raft_consensus.cc:729] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.689882 4163 consensus_queue.cc:260] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.690140 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62c365f79d4a4201ac323f16bddf3e62 consensus state has no leader
I20260811 18:57:17.690616 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b07958bac6344da1bbf3d7d35f1ac551" } new_table_name: "default.Table_d9b7a2cfa85347bf88241286c4bf0083" modify_external_catalogs: true
I20260811 18:57:17.690915 4163 ts_tablet_manager.cc:1428] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
#1 0x7fc7c79f9671 in kudu::rw_semaphore::unlock() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/rw_semaphore.h:148:5
I20260811 18:57:17.709129 4167 rpcz_store.cc:276] 0811 18:57:14.059853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.059948 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:17.613322 (+3553374us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:17.613343 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.627364 (+ 14021us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3518185}
W20260811 18:57:17.709659 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:17.709728 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 236) took 3650 ms. Trace:
I20260811 18:57:17.709867 4167 rpcz_store.cc:276] 0811 18:57:14.059261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.069044 (+ 9783us) service_pool.cc:227] Handling call
0811 18:57:17.709537 (+3640493us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:17.709552 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.709683 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3614280}
I20260811 18:57:17.710393 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 238) took 3605 ms. Trace:
I20260811 18:57:17.710216 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7c4738e833404ad7b1947bb598f05842"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.710517 4167 rpcz_store.cc:276] 0811 18:57:14.104581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.104676 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:17.710099 (+3605423us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:17.710110 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.710356 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3604791}
W20260811 18:57:17.710934 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c4738e833404ad7b1947bb598f05842 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.712168 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 239) took 3595 ms. Trace:
I20260811 18:57:17.712184 4163 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.712307 4167 rpcz_store.cc:276] 0811 18:57:14.116675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.116785 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:17.712012 (+3595227us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:17.712023 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.712126 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3594737}
I20260811 18:57:17.712563 4167 tablet_replica.cc:330] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:17.712786 4167 raft_consensus.cc:2229] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.712962 4167 raft_consensus.cc:2258] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.713557 4167 ts_tablet_manager.cc:1905] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.714628 4163 tablet_bootstrap.cc:654] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.721661 4169 ts_tablet_manager.cc:1918] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.721896 4169 log.cc:1196] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:17.722477 4169 ts_tablet_manager.cc:1939] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.722945 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 282) took 2116 ms. Trace:
I20260811 18:57:17.723112 4169 rpcz_store.cc:276] 0811 18:57:15.606882 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.606945 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:17.646952 (+2040007us) ts_tablet_manager.cc:1145] Deleting tablet 2f9fa42654cf4ec18220e5fb173ddef7
0811 18:57:17.646969 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.717105 (+ 70136us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.721487 (+ 4382us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.722871 (+ 1384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2008832}
I20260811 18:57:17.723733 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2f9fa42654cf4ec18220e5fb173ddef7 (table default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]) successfully deleted
I20260811 18:57:17.727128 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 287) took 2053 ms. Trace:
I20260811 18:57:17.727312 4169 rpcz_store.cc:276] 0811 18:57:15.673272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.673489 (+ 217us) service_pool.cc:227] Handling call
0811 18:57:17.726825 (+2053336us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:17.726845 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.727037 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2023052}
I20260811 18:57:17.733035 4169 tablet_replica.cc:330] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.733368 4169 raft_consensus.cc:2229] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:17.733857 4169 raft_consensus.cc:2258] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:17.736295 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62c365f79d4a4201ac323f16bddf3e62 consensus state has no leader
I20260811 18:57:17.736512 4169 ts_tablet_manager.cc:1905] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.737748 3824 leader_election.cc:341] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.739167 4163 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.739347 4163 ts_tablet_manager.cc:1397] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.006s
I20260811 18:57:17.739845 4836 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.740278 4163 raft_consensus.cc:348] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.740018 4836 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.740561 4163 raft_consensus.cc:374] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.740692 4163 raft_consensus.cc:729] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.740998 4163 consensus_queue.cc:260] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.741461 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.741866 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:17.742167 3891 raft_consensus.cc:2454] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:17.742462 3824 leader_election.cc:341] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.742858 3676 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.744616 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.743551 4836 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.745906 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c18120da2d245f495c52a670047760d with delete_type TABLET_DATA_DELETED (replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.746163 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 987b9ac6859444e3a86c56af786cc84e with delete_type TABLET_DATA_DELETED (replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.746376 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:17.746907 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 405773849fb94800be61ad8a7a6055a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 405773849fb94800be61ad8a7a6055a4 already in progress: creating tablet
W20260811 18:57:17.747283 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 02808605ee6c4634af47fc3fd57e9cd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02808605ee6c4634af47fc3fd57e9cd8 already in progress: creating tablet
W20260811 18:57:17.747541 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9bc5448a555944dfba8ccae09e25cbe9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bc5448a555944dfba8ccae09e25cbe9 already in progress: creating tablet
W20260811 18:57:17.747853 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c08436f33db043b1bc6395cd92102e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c08436f33db043b1bc6395cd92102e59 already in progress: creating tablet
I20260811 18:57:17.748301 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.748373 4811 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.748587 4811 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.749480 4811 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.749675 4834 raft_consensus.cc:2781] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:17.749862 4834 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:17.749993 4834 raft_consensus.cc:3037] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.750029 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.750838 3824 leader_election.cc:341] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.754258 4834 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.758585 4834 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.759725 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:17.760085 3891 raft_consensus.cc:3037] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.762885 4162 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.763101 4162 ts_tablet_manager.cc:1397] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.406s user 0.003s sys 0.007s
I20260811 18:57:17.763994 4162 raft_consensus.cc:348] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.764323 4162 raft_consensus.cc:374] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.764418 4162 raft_consensus.cc:729] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.764680 4162 consensus_queue.cc:260] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.765097 3891 raft_consensus.cc:2454] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:17.765336 4162 ts_tablet_manager.cc:1428] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:17.765770 3676 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.759491 4163 ts_tablet_manager.cc:1428] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.001s sys 0.002s
I20260811 18:57:17.766301 4836 raft_consensus.cc:2781] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.766633 4836 raft_consensus.cc:686] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.767367 4836 consensus_queue.cc:237] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.772644 4163 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.775126 4163 tablet_bootstrap.cc:654] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.780707 4162 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.788740 4162 tablet_bootstrap.cc:654] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.797640 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12d05bfe0d2348f0a14ef6b8ecf8414a" } new_table_name: "default.Table_5781967ec2ee41a5ab982d9d7394c274" modify_external_catalogs: true
I20260811 18:57:17.800720 4169 ts_tablet_manager.cc:1918] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.800926 4169 log.cc:1196] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:17.801226 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056e34af1fcd4642abd908808acf8a67" } new_table_name: "default.Table_42b47e0e7e404c8c976e7472833fd243" modify_external_catalogs: true
I20260811 18:57:17.801486 4169 ts_tablet_manager.cc:1939] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.802167 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 288) took 2110 ms. Trace:
I20260811 18:57:17.802317 4169 rpcz_store.cc:276] 0811 18:57:15.692252 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.692325 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:17.732912 (+2040587us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:17.732932 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.783044 (+ 50112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.800518 (+ 17474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.802114 (+ 1596us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2028732}
I20260811 18:57:17.802876 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6ce8db230c7e41b6a4792273a0f07e6e (table default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]) successfully deleted
I20260811 18:57:17.805891 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 289) took 2098 ms. Trace:
I20260811 18:57:17.806080 4169 rpcz_store.cc:276] 0811 18:57:15.707111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.707175 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:17.805291 (+2098116us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:17.805310 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.805834 (+ 524us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097608}
W20260811 18:57:17.806728 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 already in progress: creating tablet
W20260811 18:57:17.808802 3824 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.183s (SENT)
I20260811 18:57:17.809406 4169 tablet_replica.cc:330] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.809731 4169 raft_consensus.cc:2229] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.810009 4169 raft_consensus.cc:2258] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.819176 4169 ts_tablet_manager.cc:1905] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.823889 4163 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.824146 4163 ts_tablet_manager.cc:1397] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.000s
I20260811 18:57:17.825385 4163 raft_consensus.cc:348] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.825719 4163 raft_consensus.cc:374] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.825835 4163 raft_consensus.cc:729] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.826181 4163 consensus_queue.cc:260] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.833021 4163 ts_tablet_manager.cc:1428] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:57:17.832902 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3e695533c42647128860db601dcc0b5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.833310 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1b071ee4d3e847099b98398e52574f8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.833607 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e695533c42647128860db601dcc0b5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.833879 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b071ee4d3e847099b98398e52574f8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.837958 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c99ceca57cb245f4a3a6cb4b36f1306d consensus state has no leader
I20260811 18:57:17.838361 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_241ec7c366e84652a6d9c0123d2149ac" } modify_external_catalogs: true
I20260811 18:57:17.838711 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_241ec7c366e84652a6d9c0123d2149ac" } modify_external_catalogs: true
I20260811 18:57:17.838925 4163 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.841444 4163 tablet_bootstrap.cc:654] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.863399 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 38e24f8d4522466d81605ea37fa7064a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.867286 3860 tablet_service.cc:1467] Processing CreateTablet for tablet b20872d80c9545e5822bfc82d496ce97 (DEFAULT_TABLE table=default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.867442 3555 catalog_manager.cc:5495] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:17.867950 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20872d80c9545e5822bfc82d496ce97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.867966 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:17.868697 3550 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4] alter complete (version 1)
I20260811 18:57:17.873369 3861 tablet_service.cc:1467] Processing CreateTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.873942 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14f8717bcd94d6d88295fd8be130ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.875392 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.876044 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.877990 3709 tablet_service.cc:1467] Processing CreateTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.878574 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14f8717bcd94d6d88295fd8be130ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.881398 4167 ts_tablet_manager.cc:1918] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.881587 4167 log.cc:1196] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:17.882138 4167 ts_tablet_manager.cc:1939] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.884609 3723 tablet_service.cc:1467] Processing CreateTablet for tablet b20872d80c9545e5822bfc82d496ce97 (DEFAULT_TABLE table=default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.885171 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20872d80c9545e5822bfc82d496ce97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.886536 4162 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.886708 4162 ts_tablet_manager.cc:1397] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.002s
I20260811 18:57:17.887591 4162 raft_consensus.cc:348] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.887853 4162 raft_consensus.cc:374] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.887960 4162 raft_consensus.cc:729] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.888260 4162 consensus_queue.cc:260] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.888685 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.888900 4162 ts_tablet_manager.cc:1428] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:17.889061 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.889232 4162 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.889873 3865 tablet_service.cc:1467] Processing CreateTablet for tablet cd1fc828f73642c88171d6ac827de90e (DEFAULT_TABLE table=default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.890389 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1fc828f73642c88171d6ac827de90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.894672 3862 tablet_service.cc:1467] Processing CreateTablet for tablet a0116997a1c64e658045c319f1c45861 (DEFAULT_TABLE table=default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.895289 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0116997a1c64e658045c319f1c45861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.898638 4822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:17.899127 4822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
W20260811 18:57:17.900777 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a9efd2f19d74a68a8536fd057377738 consensus state has no leader
I20260811 18:57:17.901180 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a82335ab20e412c93a5ac0938a96012" } modify_external_catalogs: true
I20260811 18:57:17.901257 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a (DEFAULT_TABLE table=default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.901399 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a82335ab20e412c93a5ac0938a96012" } modify_external_catalogs: true
I20260811 18:57:17.901850 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf7720e31d84ae2ab592da1e7a3132a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.906956 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 330801031cc74eb2a5bc2caf53a18ead (DEFAULT_TABLE table=default.Table_e23d79bc476d496eaa713465bcfb1abf [id=54b88a6b23fe45e8b0f235baf8f7234f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.907586 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330801031cc74eb2a5bc2caf53a18ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.909198 4169 ts_tablet_manager.cc:1918] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.909361 4169 log.cc:1196] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:17.909861 4169 ts_tablet_manager.cc:1939] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.910374 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 285) took 2263 ms. Trace:
I20260811 18:57:17.910494 4169 rpcz_store.cc:276] 0811 18:57:15.646846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.649334 (+ 2488us) service_pool.cc:227] Handling call
0811 18:57:17.809318 (+2159984us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:17.809337 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.859685 (+ 50348us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.909027 (+ 49342us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.910288 (+ 1261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097471}
I20260811 18:57:17.910868 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe24bb6edb944008ff9cc73faa7374c (DEFAULT_TABLE table=default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b [id=c52d624d402741e8a390e01b9b5056b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.911229 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 290) took 2182 ms. Trace:
I20260811 18:57:17.911341 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe24bb6edb944008ff9cc73faa7374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.914820 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 4a64df7c295c4d9baded1b63a9980581 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.915359 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a64df7c295c4d9baded1b63a9980581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.916683 4729 raft_consensus.cc:482] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.916888 4729 raft_consensus.cc:504] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.917701 4729 leader_election.cc:288] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.918849 3872 tablet_service.cc:1467] Processing CreateTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 (DEFAULT_TABLE table=default.Table_3cc27811ad29451daaebbf31fb3ad198 [id=17fdd095e10d41fe8033aad4d1034af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.919374 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfea0cd619024204b9bff1e2dd62b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.922148 4163 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.922317 4163 ts_tablet_manager.cc:1397] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.003s
I20260811 18:57:17.922854 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 (DEFAULT_TABLE table=default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.923416 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec4bba841dd446aa3db751358ce7f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.923470 4163 raft_consensus.cc:348] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:17.923789 4163 raft_consensus.cc:374] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.923909 4163 raft_consensus.cc:729] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.924227 4163 consensus_queue.cc:260] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:17.924861 4163 ts_tablet_manager.cc:1428] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:17.925285 4163 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.911350 4169 rpcz_store.cc:276] 0811 18:57:15.728729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.729246 (+ 517us) service_pool.cc:227] Handling call
0811 18:57:17.910881 (+2181635us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:17.910898 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.911042 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2167388}
I20260811 18:57:17.927215 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 293) took 2132 ms. Trace:
I20260811 18:57:17.927313 4169 rpcz_store.cc:276] 0811 18:57:15.795173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795241 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:17.926974 (+2131733us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:17.926987 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.927148 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2129871}
I20260811 18:57:17.927646 4169 tablet_replica.cc:330] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.927878 4169 raft_consensus.cc:2229] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.928113 4169 raft_consensus.cc:2258] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.928720 4169 ts_tablet_manager.cc:1905] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.943271 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8426ccbe530446fcbaa162a03cbb1892 (table default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]) successfully deleted
W20260811 18:57:17.943646 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45aff3e8cb0c4f7191e4522c6ef7d156 already in progress: creating tablet
W20260811 18:57:17.943964 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 36724a83bf9844b9a27a88ca36a9bf50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36724a83bf9844b9a27a88ca36a9bf50 already in progress: creating tablet
W20260811 18:57:17.949676 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 122b7898fea44ef78f269b1505d8ddc4 consensus state has no leader
I20260811 18:57:17.951823 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8af423b4350249d6bae77e7e3c7d9fa4" } modify_external_catalogs: true
I20260811 18:57:17.952109 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8af423b4350249d6bae77e7e3c7d9fa4" } modify_external_catalogs: true
I20260811 18:57:17.953423 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f42592ab70540e59f42f7e97293a058" } modify_external_catalogs: true
I20260811 18:57:17.953629 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f42592ab70540e59f42f7e97293a058" } modify_external_catalogs: true
I20260811 18:57:17.949302 4163 tablet_bootstrap.cc:654] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.976004 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.976848 4169 ts_tablet_manager.cc:1918] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.976995 4169 log.cc:1196] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5ae200f3814e431a8288c30fe94bffcf
I20260811 18:57:17.977480 4169 ts_tablet_manager.cc:1939] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.977833 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 294) took 2180 ms. Trace:
I20260811 18:57:17.977952 4169 rpcz_store.cc:276] 0811 18:57:15.797057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.797117 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:17.927562 (+2130445us) ts_tablet_manager.cc:1145] Deleting tablet 5ae200f3814e431a8288c30fe94bffcf
0811 18:57:17.927570 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.953769 (+ 26199us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.976686 (+ 22917us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.977796 (+ 1110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2121989}
W20260811 18:57:17.978492 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b53eaa03dd34a22a14fa65da3866fe4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.985392 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ebf7720e31d84ae2ab592da1e7a3132a
I20260811 18:57:17.987109 4169 tablet_replica.cc:330] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.987389 4169 raft_consensus.cc:2229] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.987356 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5ae200f3814e431a8288c30fe94bffcf (table default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]) successfully deleted
I20260811 18:57:17.991201 4169 raft_consensus.cc:2258] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.992760 4169 ts_tablet_manager.cc:1905] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.994378 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be30e8194fd44874a1fdd996128eb4d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.995195 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be30e8194fd44874a1fdd996128eb4d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.001415 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.004695 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:18.006580 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:17.950120 3676 leader_election.cc:341] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 151df87806694f46be5013592c68c01a
I20260811 18:57:18.012598 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.013012 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 (DEFAULT_TABLE table=default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.013206 3718 tablet_service.cc:1467] Processing CreateTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a (DEFAULT_TABLE table=default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.013746 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.014250 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe24bb6edb944008ff9cc73faa7374c (DEFAULT_TABLE table=default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b [id=c52d624d402741e8a390e01b9b5056b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.015468 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2bb9c598e2de422b86a8304bbece96db" } new_table_name: "default.Table_559eff9548514310840eee6c9a02b364" modify_external_catalogs: true
I20260811 18:57:18.019548 3706 tablet_service.cc:1467] Processing CreateTablet for tablet cd1fc828f73642c88171d6ac827de90e (DEFAULT_TABLE table=default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:18.020144 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 66869.4us
I20260811 18:57:18.020678 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4a64df7c295c4d9baded1b63a9980581 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.021322 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec4bba841dd446aa3db751358ce7f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.021428 3715 tablet_service.cc:1467] Processing CreateTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 (DEFAULT_TABLE table=default.Table_3cc27811ad29451daaebbf31fb3ad198 [id=17fdd095e10d41fe8033aad4d1034af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.021898 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfea0cd619024204b9bff1e2dd62b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.021369 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 330801031cc74eb2a5bc2caf53a18ead (DEFAULT_TABLE table=default.Table_e23d79bc476d496eaa713465bcfb1abf [id=54b88a6b23fe45e8b0f235baf8f7234f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.023916 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330801031cc74eb2a5bc2caf53a18ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.025117 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe24bb6edb944008ff9cc73faa7374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.026053 4169 ts_tablet_manager.cc:1918] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.026217 4169 log.cc:1196] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d3d6daff4b7540309ff58755758e9752
I20260811 18:57:18.026664 4169 ts_tablet_manager.cc:1939] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.026796 4729 raft_consensus.cc:482] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.026988 4729 raft_consensus.cc:504] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.027750 4729 leader_election.cc:288] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.027892 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1fc828f73642c88171d6ac827de90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.030558 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a64df7c295c4d9baded1b63a9980581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.031039 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf7720e31d84ae2ab592da1e7a3132a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:18.034984 3676 leader_election.cc:341] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: dde27d8cc3eb4d09879e717fc90305fd
I20260811 18:57:18.038069 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.039134 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 291) took 2244 ms. Trace:
I20260811 18:57:18.041402 4162 tablet_bootstrap.cc:654] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.041671 4169 rpcz_store.cc:276] 0811 18:57:15.794573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795803 (+ 1230us) service_pool.cc:227] Handling call
0811 18:57:17.987021 (+2191218us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:17.987033 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.999212 (+ 12179us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.025917 (+ 26705us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.039069 (+ 13152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2176147}
I20260811 18:57:18.043712 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d3d6daff4b7540309ff58755758e9752 (table default.Table_923c4369260d436fafb1aff17a2ec8d4 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]) successfully deleted
I20260811 18:57:18.044154 4169 tablet_replica.cc:330] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.044416 4169 raft_consensus.cc:2229] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.044665 4169 raft_consensus.cc:2258] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.045290 4169 ts_tablet_manager.cc:1905] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.068277 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 240) took 3827 ms. Trace:
I20260811 18:57:18.068542 4167 rpcz_store.cc:276] 0811 18:57:14.120397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.120500 (+ 103us) service_pool.cc:227] Handling call
0811 18:57:17.712502 (+3592002us) ts_tablet_manager.cc:1145] Deleting tablet 7f640c60e81b4a369458b1497a269c9c
0811 18:57:17.712511 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.836379 (+123868us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.881229 (+ 44850us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.947309 (+ 66080us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3591489}
I20260811 18:57:18.069460 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:18.069514 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 241) took 3875 ms. Trace:
I20260811 18:57:18.069650 4167 rpcz_store.cc:276] 0811 18:57:14.194541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.198569 (+ 4028us) service_pool.cc:227] Handling call
0811 18:57:18.069270 (+3870701us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:18.069288 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.069436 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3870186}
W20260811 18:57:18.070292 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8c07150e31994bdf9a00ac2e7a954691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c07150e31994bdf9a00ac2e7a954691 already in progress: creating tablet
I20260811 18:57:18.070740 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a0116997a1c64e658045c319f1c45861 (DEFAULT_TABLE table=default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.071374 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0116997a1c64e658045c319f1c45861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.075193 4167 tablet_replica.cc:330] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.075587 4167 raft_consensus.cc:2229] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.076617 4167 pending_rounds.cc:62] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:18.076879 4167 pending_rounds.cc:68] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400108963278848 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_44e9cbdec47545899467895fd869b389" dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" new_extra_config { } }
W20260811 18:57:18.078389 4167 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38286: Aborted: Op aborted
I20260811 18:57:18.079641 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38286 (request call id 281) took 2202 ms. Trace:
I20260811 18:57:18.080503 4163 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.081655 3676 consensus_peers.cc:487] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f8c0444ec2242299482736a73551f80. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.082139 4167 rpcz_store.cc:276] 0811 18:57:15.876834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.879172 (+ 2338us) service_pool.cc:227] Handling call
0811 18:57:18.079598 (+2200426us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.977183 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:15.977349 (+ 166us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:15.977379 (+ 30us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:15.977385 (+ 6us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:15.977433 (+ 48us) alter_schema_op.cc:126] START. Timestamp: 1786474635977363
0811 18:57:15.977437 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.977655 (+ 218us) log.cc:842] Serialized 173 byte log entry
0811 18:57:18.078255 (+2100600us) op_driver.cc:444] REPLICATION: finished
0811 18:57:18.078289 (+ 34us) op_driver.cc:372] HandleFailure(Aborted: Op aborted)
0811 18:57:18.078299 (+ 10us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":273,"prepare.run_cpu_time_us":6887,"prepare.run_wall_time_us":233422,"replication_time_us":2100769,"thread_start_us":219309,"threads_started":1,"wal-append.queue_time_us":190675,"wal-append.run_cpu_time_us":2476,"wal-append.run_wall_time_us":1215875}]]}
I20260811 18:57:18.082857 4163 ts_tablet_manager.cc:1397] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.158s user 0.007s sys 0.011s
I20260811 18:57:18.083889 4163 raft_consensus.cc:348] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.083961 4162 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.085772 4162 ts_tablet_manager.cc:1397] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.197s user 0.006s sys 0.000s
I20260811 18:57:18.086961 4162 raft_consensus.cc:348] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.087371 4162 raft_consensus.cc:374] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.087529 4162 raft_consensus.cc:729] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.088001 4162 consensus_queue.cc:260] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.090145 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.056s (SENT) [suppressed 13 similar messages]
I20260811 18:57:18.062252 4169 ts_tablet_manager.cc:1918] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.093575 4169 log.cc:1196] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cf285cdf5235451bb1531cb294017101
I20260811 18:57:18.093896 4163 raft_consensus.cc:374] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.095602 4162 ts_tablet_manager.cc:1428] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:18.094172 4169 ts_tablet_manager.cc:1939] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.096565 4163 raft_consensus.cc:729] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.097486 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--webserver_interface=127.3.72.126
--webserver_port=40953
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:57:18.097678 4163 consensus_queue.cc:260] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.098412 4163 ts_tablet_manager.cc:1428] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.001s
I20260811 18:57:18.098798 4163 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.114094 4167 raft_consensus.cc:2258] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.115231 4167 ts_tablet_manager.cc:1905] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.096984 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 292) took 2301 ms. Trace:
I20260811 18:57:18.115634 4169 rpcz_store.cc:276] 0811 18:57:15.795014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795475 (+ 461us) service_pool.cc:227] Handling call
0811 18:57:18.044047 (+2248572us) ts_tablet_manager.cc:1145] Deleting tablet cf285cdf5235451bb1531cb294017101
0811 18:57:18.044063 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.058852 (+ 14789us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.062113 (+ 3261us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.096942 (+ 34829us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2212969}
I20260811 18:57:18.116355 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cf285cdf5235451bb1531cb294017101 (table default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]) successfully deleted
I20260811 18:57:18.116849 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 295) took 2305 ms. Trace:
I20260811 18:57:18.117020 4169 rpcz_store.cc:276] 0811 18:57:15.811013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.811429 (+ 416us) service_pool.cc:227] Handling call
0811 18:57:18.116694 (+2305265us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:18.116708 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.116813 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2243388}
W20260811 18:57:18.117552 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ad424c8843ff434ba0449a86849bb5a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad424c8843ff434ba0449a86849bb5a0 already in progress: creating tablet
I20260811 18:57:18.118010 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 311) took 2043 ms. Trace:
I20260811 18:57:18.118175 4169 rpcz_store.cc:276] 0811 18:57:16.074900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.074986 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:18.117869 (+2042883us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:18.117881 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.117978 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2034890}
W20260811 18:57:18.118693 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b3ec38f9b55489ba00ed76f9e4456f5 already in progress: creating tablet
I20260811 18:57:18.119047 4169 tablet_replica.cc:330] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.119314 4169 raft_consensus.cc:2229] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.119557 4169 raft_consensus.cc:2258] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.120249 4169 ts_tablet_manager.cc:1905] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.123118 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5120347318f64721a3d6ef8cdc5564cd" } new_table_name: "default.Table_a8d2d56f43544d68adbbd2848fc00dde" modify_external_catalogs: true
I20260811 18:57:18.114424 4162 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.129352 4162 tablet_bootstrap.cc:654] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.136024 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 38e24f8d4522466d81605ea37fa7064a
I20260811 18:57:18.136510 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 38e24f8d4522466d81605ea37fa7064a deleted and replaced by tablet c17847a358e544b4b4e51f65463a9679
I20260811 18:57:18.139307 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.140015 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.152510 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 4687b95728fc485fa0e842bc447eb83f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.154834 4162 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.155055 4162 ts_tablet_manager.cc:1397] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.000s
I20260811 18:57:18.155886 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 8426ccbe530446fcbaa162a03cbb1892 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.156379 4162 raft_consensus.cc:348] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.156782 4162 raft_consensus.cc:374] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.156905 4162 raft_consensus.cc:729] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.157186 4162 consensus_queue.cc:260] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.159427 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01f64f05d1ee44029680c5423fd0d81b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.160223 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01f64f05d1ee44029680c5423fd0d81b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.164161 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.165304 4162 ts_tablet_manager.cc:1428] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:18.171603 4162 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.115921 4163 tablet_bootstrap.cc:654] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.175804 4167 ts_tablet_manager.cc:1918] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:18.177737 4167 log.cc:1196] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:18.181793 4162 tablet_bootstrap.cc:654] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.184104 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7bfc22ccd3e44c679f5e951318bb0af3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.184824 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bfc22ccd3e44c679f5e951318bb0af3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.185266 4167 ts_tablet_manager.cc:1939] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.186743 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.185990 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 242) took 3813 ms. Trace:
I20260811 18:57:18.187811 4167 rpcz_store.cc:276] 0811 18:57:14.372534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.372776 (+ 242us) service_pool.cc:227] Handling call
0811 18:57:18.075052 (+3702276us) ts_tablet_manager.cc:1145] Deleting tablet e7b3a14979e84b1e8ec6ed3f49b93b00
0811 18:57:18.075066 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.137895 (+ 62829us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.175624 (+ 37729us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.185925 (+ 10301us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3701810}
I20260811 18:57:18.188669 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]) successfully deleted
I20260811 18:57:18.189182 4167 tablet_replica.cc:330] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.189548 4167 raft_consensus.cc:2229] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.190058 4167 raft_consensus.cc:2258] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.191397 4167 ts_tablet_manager.cc:1905] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.200928 4162 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.201267 4162 ts_tablet_manager.cc:1397] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.001s
I20260811 18:57:18.202358 4162 raft_consensus.cc:348] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.202826 4162 raft_consensus.cc:374] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.202965 4162 raft_consensus.cc:729] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.203378 4162 consensus_queue.cc:260] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.205153 4163 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.205395 4163 ts_tablet_manager.cc:1397] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.003s
I20260811 18:57:18.206312 4163 raft_consensus.cc:348] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.206992 4163 raft_consensus.cc:374] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.207154 4163 raft_consensus.cc:729] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.207463 4163 consensus_queue.cc:260] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.209573 4163 ts_tablet_manager.cc:1428] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:57:18.209903 4163 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.213186 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:18.214327 4162 ts_tablet_manager.cc:1428] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260811 18:57:18.215492 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_914c8b90fadf48e88b75d13897462d84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.216181 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_914c8b90fadf48e88b75d13897462d84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.217484 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 4ec1fa93c2bf4008bdff3f8e86fec104
I20260811 18:57:18.217856 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c1631122dcd4c738fcb59926b7873fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.218482 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c1631122dcd4c738fcb59926b7873fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.225946 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.230751 4162 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.236996 4162 tablet_bootstrap.cc:654] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.244478 4167 ts_tablet_manager.cc:1918] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:18.244792 4167 log.cc:1196] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:18.245517 4167 ts_tablet_manager.cc:1939] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.246374 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 243) took 3803 ms. Trace:
I20260811 18:57:18.248509 4167 rpcz_store.cc:276] 0811 18:57:14.442751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.442818 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:18.188709 (+3745891us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:18.188723 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.231670 (+ 42947us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.244295 (+ 12625us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.246305 (+ 2010us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3745275}
I20260811 18:57:18.249394 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b1ce1042ae974f4dbac964bd7ea16a94 (table default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]) successfully deleted
I20260811 18:57:18.250067 4167 tablet_replica.cc:330] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.250497 4167 raft_consensus.cc:2229] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.251147 4167 raft_consensus.cc:2258] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.252122 4162 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.252434 4162 ts_tablet_manager.cc:1397] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260811 18:57:18.253576 4162 raft_consensus.cc:348] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.253968 4162 raft_consensus.cc:374] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.254149 4162 raft_consensus.cc:729] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.254599 4162 consensus_queue.cc:260] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.255684 4162 ts_tablet_manager.cc:1428] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:18.256786 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eb8f5b87062349c4b19ec99fa0175f24
I20260811 18:57:18.257472 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b071ee4d3e847099b98398e52574f8e" } new_table_name: "default.Table_3294569c49f84a2291010174baee10af" modify_external_catalogs: true
I20260811 18:57:18.258460 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cce97bfdb1d41e685e091c8c5cc6946 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.259224 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.259500 4162 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:18.259820 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.264590 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.264971 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.266274 4167 ts_tablet_manager.cc:1905] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.271451 4836 raft_consensus.cc:482] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.271720 4836 raft_consensus.cc:504] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.272640 4836 leader_election.cc:288] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:18.274482 3676 leader_election.cc:341] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:18.279433 4162 tablet_bootstrap.cc:654] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.224073 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.290581 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_655dd0bc17cc45d3a344bb5c02ff7f80" } new_table_name: "default.Table_d7a06761b8a145519b7bb645e7a73657" modify_external_catalogs: true
I20260811 18:57:18.293627 4163 tablet_bootstrap.cc:654] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.297685 4167 ts_tablet_manager.cc:1918] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.299079 4163 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.299118 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 866ed614511649459b22a9fce55220b2 consensus state has no leader
I20260811 18:57:18.299449 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 62c365f79d4a4201ac323f16bddf3e62 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.300606 4163 ts_tablet_manager.cc:1397] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.001s
I20260811 18:57:18.303570 4163 raft_consensus.cc:348] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.305039 4163 raft_consensus.cc:374] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.304862 4162 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.305424 4162 ts_tablet_manager.cc:1397] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.000s
I20260811 18:57:18.306609 4162 raft_consensus.cc:348] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.307067 4162 raft_consensus.cc:374] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.307226 4162 raft_consensus.cc:729] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.307597 4162 consensus_queue.cc:260] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.310904 4162 ts_tablet_manager.cc:1428] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260811 18:57:18.311223 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a994b3f4d7d4c92946ad8d595e6ad2b has no consensus state
I20260811 18:57:18.311421 4162 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.313988 4162 tablet_bootstrap.cc:654] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.317000 4167 log.cc:1196] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/151df87806694f46be5013592c68c01a
I20260811 18:57:18.263348 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.276921 4811 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.317469 4811 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.318225 4811 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.254122 4169 ts_tablet_manager.cc:1918] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.319798 4169 log.cc:1196] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a30bb384136f4a169a5751655437d5a3
I20260811 18:57:18.324113 4169 ts_tablet_manager.cc:1939] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.319074 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.325189 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1996443a5de84ded8919fb5c51dc6554" } new_table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" modify_external_catalogs: true
I20260811 18:57:18.325757 4167 ts_tablet_manager.cc:1939] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.326331 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 253) took 3555 ms. Trace:
I20260811 18:57:18.329249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 309) took 2262 ms. Trace:
W20260811 18:57:18.324985 3824 leader_election.cc:341] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.324286 4163 raft_consensus.cc:729] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.331158 4163 consensus_queue.cc:260] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.332197 4163 ts_tablet_manager.cc:1428] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20260811 18:57:18.332403 4169 rpcz_store.cc:276] 0811 18:57:16.066582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.066985 (+ 403us) service_pool.cc:227] Handling call
0811 18:57:18.118969 (+2051984us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:18.118980 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.196966 (+ 77986us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.253979 (+ 57013us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.329165 (+ 75186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1980251}
I20260811 18:57:18.337167 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_460a6647324c49b78cd79e59e974732b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.337883 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_460a6647324c49b78cd79e59e974732b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.326467 4167 rpcz_store.cc:276] 0811 18:57:14.770962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.773526 (+ 2564us) service_pool.cc:227] Handling call
0811 18:57:18.249976 (+3476450us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:18.249992 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.282999 (+ 33007us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.297530 (+ 14531us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.326275 (+ 28745us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3447994}
I20260811 18:57:18.339077 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 254) took 3492 ms. Trace:
I20260811 18:57:18.338392 4811 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.343849 4167 rpcz_store.cc:276] 0811 18:57:14.846824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.847111 (+ 287us) service_pool.cc:227] Handling call
0811 18:57:18.338807 (+3491696us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:18.338828 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.339002 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3471806}
I20260811 18:57:18.344732 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 255) took 3359 ms. Trace:
I20260811 18:57:18.344911 4167 rpcz_store.cc:276] 0811 18:57:14.985433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.985496 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:18.344523 (+3359027us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:18.344537 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.344669 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3358452}
I20260811 18:57:18.345329 4811 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.345588 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a30bb384136f4a169a5751655437d5a3 (table default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]) successfully deleted
I20260811 18:57:18.348589 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 151df87806694f46be5013592c68c01a (table default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]) successfully deleted
W20260811 18:57:18.348901 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4687b95728fc485fa0e842bc447eb83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4687b95728fc485fa0e842bc447eb83f already in progress: creating tablet
I20260811 18:57:18.352317 4163 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.346956 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.348222 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.355731 4811 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.355922 4811 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.356240 4811 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:18.357203 3824 leader_election.cc:341] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.358134 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:18.358937 3824 leader_election.cc:341] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.359243 4811 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.359439 4169 tablet_replica.cc:330] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.359834 4169 raft_consensus.cc:2229] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.360222 4169 raft_consensus.cc:2258] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.361132 4169 ts_tablet_manager.cc:1905] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.361928 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 256) took 3161 ms. Trace:
W20260811 18:57:18.361927 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ff1be026ca404eada53f60f257ea367c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff1be026ca404eada53f60f257ea367c already in progress: creating tablet
I20260811 18:57:18.364181 4167 rpcz_store.cc:276] 0811 18:57:15.200923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.200985 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:18.361687 (+3160702us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:18.361707 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.361855 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3153180}
I20260811 18:57:18.365846 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.370777 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9e896cd16f14d14800b4a3f697d2ad4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:18.388897 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fbba1923ce0049448fe4e28f2dd258dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbba1923ce0049448fe4e28f2dd258dc already in progress: creating tablet
I20260811 18:57:18.389110 4167 tablet_replica.cc:330] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.389436 4167 raft_consensus.cc:2229] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.389720 4167 raft_consensus.cc:2258] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.390517 4167 ts_tablet_manager.cc:1905] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.393463 4163 tablet_bootstrap.cc:654] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.402908 4162 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.414955 4811 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.415158 4811 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.415870 4811 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:18.415964 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 has no consensus state
I20260811 18:57:18.416006 4811 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.416160 4811 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.416656 4811 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.418043 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.418632 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.419354 4163 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.419464 3824 leader_election.cc:341] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.419670 4163 ts_tablet_manager.cc:1397] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.083s user 0.003s sys 0.003s
W20260811 18:57:18.419823 3824 leader_election.cc:341] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.420197 4162 ts_tablet_manager.cc:1397] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.109s user 0.006s sys 0.000s
I20260811 18:57:18.420768 4163 raft_consensus.cc:348] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.427528 4162 raft_consensus.cc:348] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.427956 4162 raft_consensus.cc:374] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.428081 4162 raft_consensus.cc:729] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.428134 4163 raft_consensus.cc:374] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.428406 4163 raft_consensus.cc:729] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.428787 4163 consensus_queue.cc:260] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.430799 4873 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.431241 4163 ts_tablet_manager.cc:1428] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260811 18:57:18.434026 4873 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.434849 4163 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.439610 4163 tablet_bootstrap.cc:654] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.446384 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.447527 4162 consensus_queue.cc:260] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.456595 3709 tablet_service.cc:1467] Processing CreateTablet for tablet ae7da0868b114052ba421801b76fb33c (DEFAULT_TABLE table=default.Table_e2b67d7e59584a2bb877f79bdafed3a1 [id=c417185e41c94dad8cc5e5f5937adb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.456804 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (DEFAULT_TABLE table=default.Table_a8d2d56f43544d68adbbd2848fc00dde [id=9c62165c055f4acea600e53873f88772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.457106 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7da0868b114052ba421801b76fb33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.457937 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
#2 0x7fc7bfa1f90d in std::unique_lock<kudu::rw_semaphore>::unlock() ../../../include/c++/8/bits/std_mutex.h:323:17
I20260811 18:57:18.462456 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.462994 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1e244addf5462499fcadaa3332cba1 (DEFAULT_TABLE table=default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.463568 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1e244addf5462499fcadaa3332cba1. 1 dirs total, 0 dirs full, 0 dirs failed
#3 0x7fc7bfa1f873 in std::unique_lock<kudu::rw_semaphore>::~unique_lock() ../../../include/c++/8/bits/std_mutex.h:232:4
#4 0x7fc7bfa14063 in kudu::tablet::AlterSchemaOpState::~AlterSchemaOpState() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:47:3
I20260811 18:57:18.472391 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d (DEFAULT_TABLE table=default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.473073 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0540e4e43ea34fbfbc66b6035c60684d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.474762 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 114397bea09444479df113552c22e154 (DEFAULT_TABLE table=default.Table_ba1e7c81d1bf4209b7089639928742f1 [id=cfba5a388f8f406d8b2a9ae1fc397585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.475261 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114397bea09444479df113552c22e154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.460816 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet d00d943b778741b4aeec076ba46a9c40 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.478448 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (DEFAULT_TABLE table=default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.479125 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b2ca385374abeb709fccbdf6fa0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.482776 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd (DEFAULT_TABLE table=default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.483397 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4059e5907b1e488ab9e1d04e563f96cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.491619 3723 tablet_service.cc:1467] Processing CreateTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 (DEFAULT_TABLE table=default.Table_24955fa6843f46dc883be60173049a16 [id=c5663aaa82454e73a920034172f2a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.492022 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c4738e833404ad7b1947bb598f05842" } modify_external_catalogs: true
I20260811 18:57:18.492380 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c4738e833404ad7b1947bb598f05842" } modify_external_catalogs: true
I20260811 18:57:18.492666 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.495688 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet c99ceca57cb245f4a3a6cb4b36f1306d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.495966 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b203df4124364737839bf4c1ae072988 (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.496462 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b203df4124364737839bf4c1ae072988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.497618 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" } modify_external_catalogs: true
I20260811 18:57:18.497871 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" } modify_external_catalogs: true
I20260811 18:57:18.507228 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_459873f5dbe2412c80b73ca35dce90ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.508004 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_459873f5dbe2412c80b73ca35dce90ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.455566 4162 ts_tablet_manager.cc:1428] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.029s user 0.003s sys 0.000s
I20260811 18:57:18.521431 4169 ts_tablet_manager.cc:1918] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:18.521569 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:18.525424 4811 raft_consensus.cc:482] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.525673 4811 raft_consensus.cc:504] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#5 0x7fc7bfab3cb2 in kudu::tablet::AlterSchemaOpState::~AlterSchemaOpState() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:46:25
I20260811 18:57:18.527040 4811 leader_election.cc:288] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.527339 4811 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:18.527613 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 10311ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
W20260811 18:57:18.528066 3824 leader_election.cc:341] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.527832 4811 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.530330 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 consensus state has no leader
I20260811 18:57:18.531165 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" } modify_external_catalogs: true
I20260811 18:57:18.531589 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" } modify_external_catalogs: true
W20260811 18:57:18.531718 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 81044.4us
I20260811 18:57:18.533528 4873 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.533833 4873 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.534233 4873 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.534961 4873 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.535987 4811 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.536703 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a" } modify_external_catalogs: true
I20260811 18:57:18.537536 3870 tablet_service.cc:1467] Processing CreateTablet for tablet ae7da0868b114052ba421801b76fb33c (DEFAULT_TABLE table=default.Table_e2b67d7e59584a2bb877f79bdafed3a1 [id=c417185e41c94dad8cc5e5f5937adb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.538113 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7da0868b114052ba421801b76fb33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.539124 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (DEFAULT_TABLE table=default.Table_a8d2d56f43544d68adbbd2848fc00dde [id=9c62165c055f4acea600e53873f88772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.539528 4163 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.539916 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a" } modify_external_catalogs: true
I20260811 18:57:18.542541 4169 log.cc:1196] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:18.542249 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.544478 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.544632 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.544768 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 114397bea09444479df113552c22e154 (DEFAULT_TABLE table=default.Table_ba1e7c81d1bf4209b7089639928742f1 [id=cfba5a388f8f406d8b2a9ae1fc397585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.552103 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.552510 4163 ts_tablet_manager.cc:1397] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.105s user 0.006s sys 0.021s
I20260811 18:57:18.554267 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.554440 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.554585 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1e244addf5462499fcadaa3332cba1 (DEFAULT_TABLE table=default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.559940 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114397bea09444479df113552c22e154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.567266 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d (DEFAULT_TABLE table=default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.569414 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f287040b879642448367abe2b1c73cd9 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.565476 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd (DEFAULT_TABLE table=default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.570201 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (DEFAULT_TABLE table=default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.571931 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1e244addf5462499fcadaa3332cba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.570428 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:18.576421 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48359.5us
W20260811 18:57:18.579263 3678 leader_election.cc:334] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.370s (SENT)
I20260811 18:57:18.579573 3678 leader_election.cc:302] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:18.580960 4169 ts_tablet_manager.cc:1939] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.581364 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f287040b879642448367abe2b1c73cd9 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.581665 4162 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.581679 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 312) took 2504 ms. Trace:
I20260811 18:57:18.583541 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0540e4e43ea34fbfbc66b6035c60684d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.584934 3863 tablet_service.cc:1467] Processing CreateTablet for tablet b203df4124364737839bf4c1ae072988 (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.585078 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4059e5907b1e488ab9e1d04e563f96cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.581923 4163 raft_consensus.cc:348] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.589954 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a0c48e6299546a2921a7a7db03f1032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.590917 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a0c48e6299546a2921a7a7db03f1032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.591106 3872 tablet_service.cc:1467] Processing CreateTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 (DEFAULT_TABLE table=default.Table_24955fa6843f46dc883be60173049a16 [id=c5663aaa82454e73a920034172f2a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.596182 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b203df4124364737839bf4c1ae072988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.596837 4163 raft_consensus.cc:374] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.597133 4169 rpcz_store.cc:276] 0811 18:57:16.077159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.077396 (+ 237us) service_pool.cc:227] Handling call
0811 18:57:18.343737 (+2266341us) ts_tablet_manager.cc:1145] Deleting tablet 0e1efffade0f486d8d627e56e0be6ff6
0811 18:57:18.343751 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.405050 (+ 61299us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.442153 (+ 37103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.581506 (+139353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2199216}
I20260811 18:57:18.597265 4836 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.598573 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.598869 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:18.599550 3553 catalog_manager.cc:6565] ReplaceTablet: tablet 62c365f79d4a4201ac323f16bddf3e62 deleted and replaced by tablet 4093564a88264ed59c945523c9ddc9b3
I20260811 18:57:18.599690 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f287040b879642448367abe2b1c73cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.600567 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c41131a66fdf4c178644890cff824752
I20260811 18:57:18.600899 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b4de110bd3348f0b0dae21c9f2b6a19 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.601001 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f287040b879642448367abe2b1c73cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.601388 4163 raft_consensus.cc:729] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.601469 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:18.601845 4169 tablet_replica.cc:330] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.601979 3547 catalog_manager.cc:6565] ReplaceTablet: tablet c99ceca57cb245f4a3a6cb4b36f1306d deleted and replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b
I20260811 18:57:18.604775 4836 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.605844 4836 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.616293 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b2ca385374abeb709fccbdf6fa0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.616948 4163 consensus_queue.cc:260] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.612833 3678 raft_consensus.cc:2670] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:18.618117 4163 ts_tablet_manager.cc:1428] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.065s user 0.004s sys 0.000s
I20260811 18:57:18.620610 4163 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:18.622628 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 45784.2us
I20260811 18:57:18.625051 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8eb041f3d704762868a048faec0e0fc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.626040 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.634853 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.638154 4162 tablet_bootstrap.cc:654] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.638326 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0e1efffade0f486d8d627e56e0be6ff6 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
I20260811 18:57:18.640241 4169 raft_consensus.cc:2229] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.645839 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.650442 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6bf41429faeb4785ad747d207e4641f8" } new_table_name: "default.Table_eff26ab40dd947859e4f83ff55b793d4" modify_external_catalogs: true
I20260811 18:57:18.653733 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a93ee06ed76342e082697ded810c4c99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.654800 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:18.654924 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a93ee06ed76342e082697ded810c4c99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.657825 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b" } modify_external_catalogs: true
I20260811 18:57:18.658298 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b" } modify_external_catalogs: true
I20260811 18:57:18.659364 4167 ts_tablet_manager.cc:1918] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.660079 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.660952 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c1631122dcd4c738fcb59926b7873fc" } new_table_name: "default.Table_e6a28da9c6fa458d9daa95fdbd30920c" modify_external_catalogs: true
I20260811 18:57:18.661505 4167 log.cc:1196] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/80cf2375b4c64dbfae4e20003545da60
I20260811 18:57:18.662837 4167 ts_tablet_manager.cc:1939] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.664314 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 268) took 2912 ms. Trace:
I20260811 18:57:18.667500 4167 rpcz_store.cc:276] 0811 18:57:15.751514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.751728 (+ 214us) service_pool.cc:227] Handling call
0811 18:57:18.388974 (+2637246us) ts_tablet_manager.cc:1145] Deleting tablet 80cf2375b4c64dbfae4e20003545da60
0811 18:57:18.388995 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.574334 (+185339us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.658873 (+ 84539us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.664251 (+ 5378us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2609983}
I20260811 18:57:18.668161 4167 tablet_replica.cc:330] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.668439 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 80cf2375b4c64dbfae4e20003545da60 (table default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]) successfully deleted
I20260811 18:57:18.673262 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.674924 4169 raft_consensus.cc:2258] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.679265 4167 raft_consensus.cc:2229] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.679437 4162 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:18.682511 3824 leader_election.cc:341] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:18.683153 3824 leader_election.cc:341] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.684267 4162 ts_tablet_manager.cc:1397] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.108s user 0.017s sys 0.000s
W20260811 18:57:18.685386 3824 leader_election.cc:341] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.686444 4162 raft_consensus.cc:348] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.687206 4162 raft_consensus.cc:374] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.687420 4162 raft_consensus.cc:729] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.688145 4162 consensus_queue.cc:260] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.690297 4169 ts_tablet_manager.cc:1905] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.696566 4162 ts_tablet_manager.cc:1428] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.012s user 0.000s sys 0.004s
I20260811 18:57:18.698390 4162 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.700106 4163 tablet_bootstrap.cc:654] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.700568 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:18.704296 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.706924 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:18.707239 3888 raft_consensus.cc:2454] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:18.707970 4162 tablet_bootstrap.cc:654] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.715167 4167 pending_rounds.cc:62] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:18.715397 4167 pending_rounds.cc:68] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400105548996608 op_type: NO_OP noop_request { }
I20260811 18:57:18.715610 4167 raft_consensus.cc:2866] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:18.715799 4167 raft_consensus.cc:2258] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.716709 4167 ts_tablet_manager.cc:1905] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.733115 4162 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.733368 4162 ts_tablet_manager.cc:1397] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260811 18:57:18.734704 4162 raft_consensus.cc:348] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.735117 4162 raft_consensus.cc:374] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.735266 4162 raft_consensus.cc:729] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.735638 4162 consensus_queue.cc:260] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.738811 4836 consensus_queue.cc:1035] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:18.740077 3676 consensus_peers.cc:487] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d6daff4b7540309ff58755758e9752. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.740566 3676 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:18.740900 4836 raft_consensus.cc:2781] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:18.741057 4836 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:18.741159 4836 raft_consensus.cc:3037] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.743978 4162 ts_tablet_manager.cc:1428] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260811 18:57:18.744951 4836 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.746186 4836 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:18.748098 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_d7a06761b8a145519b7bb645e7a73657 [id=f51f38f3582f44db99007a265b665fd4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:18.751127 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.754065 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.753607 4162 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.758466 4729 raft_consensus.cc:482] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.758708 4729 raft_consensus.cc:504] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.759444 4729 leader_election.cc:288] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.761664 4162 tablet_bootstrap.cc:654] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.768472 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:18.769016 3549 catalog_manager.cc:6565] ReplaceTablet: tablet d00d943b778741b4aeec076ba46a9c40 deleted and replaced by tablet ccc63b46a4e346f79d1e03a82490794c
I20260811 18:57:18.770617 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.775532 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.775663 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdae8856d30a46259e91345b60181c2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.776585 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdae8856d30a46259e91345b60181c2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.780112 4167 ts_tablet_manager.cc:1918] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:18.780325 4167 log.cc:1196] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/508f0b39cd654aa991a2928a4b067762
I20260811 18:57:18.784185 4167 ts_tablet_manager.cc:1939] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.784718 4836 raft_consensus.cc:482] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.784929 4836 raft_consensus.cc:504] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.786584 4836 leader_election.cc:288] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.787133 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a443fe209556489abb47f4faa5d696e5
I20260811 18:57:18.788350 4162 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.788548 4162 ts_tablet_manager.cc:1397] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.004s
I20260811 18:57:18.788784 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.789446 4162 raft_consensus.cc:348] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.789729 4162 raft_consensus.cc:374] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.789806 4162 raft_consensus.cc:729] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.790048 4162 consensus_queue.cc:260] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.790736 4162 ts_tablet_manager.cc:1428] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:57:18.791313 4162 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.793320 4162 tablet_bootstrap.cc:654] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.793757 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea0cc7a576d64d1c934364316cae3c70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.794492 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea0cc7a576d64d1c934364316cae3c70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.796490 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cd4eb198addc42f2b6d860ef805c589f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.797047 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd4eb198addc42f2b6d860ef805c589f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.798214 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.802116 4162 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.802330 4162 ts_tablet_manager.cc:1397] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260811 18:57:18.788267 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 269) took 3026 ms. Trace:
I20260811 18:57:18.803227 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:18.803522 3891 raft_consensus.cc:3037] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.803390 4162 raft_consensus.cc:348] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.803674 4162 raft_consensus.cc:374] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.803812 4162 raft_consensus.cc:729] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.804034 4162 consensus_queue.cc:260] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.807471 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.807706 4162 ts_tablet_manager.cc:1428] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:18.809289 4162 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.811378 4162 tablet_bootstrap.cc:654] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.815207 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c17847a358e544b4b4e51f65463a9679
I20260811 18:57:18.815478 4836 raft_consensus.cc:482] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.815723 4836 raft_consensus.cc:504] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.816637 4836 leader_election.cc:288] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.817271 4162 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.817459 4162 ts_tablet_manager.cc:1397] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:57:18.818387 4162 raft_consensus.cc:348] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.818691 4162 raft_consensus.cc:374] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.818778 4162 raft_consensus.cc:729] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.821239 4162 consensus_queue.cc:260] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.822000 4162 ts_tablet_manager.cc:1428] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260811 18:57:18.822206 3676 consensus_peers.cc:487] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.822427 4162 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:18.822635 3676 leader_election.cc:341] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 0e1efffade0f486d8d627e56e0be6ff6
W20260811 18:57:18.823030 3676 leader_election.cc:341] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5ae200f3814e431a8288c30fe94bffcf
W20260811 18:57:18.823429 3676 consensus_peers.cc:487] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8426ccbe530446fcbaa162a03cbb1892. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:18.824383 3676 leader_election.cc:341] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a30bb384136f4a169a5751655437d5a3
W20260811 18:57:18.827483 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bfb54175c834517b7af291c942333cd consensus state has no leader
I20260811 18:57:18.828184 4162 tablet_bootstrap.cc:654] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:18.828405 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e0747ef88574c68ae074732cb7095b9 has no consensus state
I20260811 18:57:18.830037 3891 raft_consensus.cc:2454] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:18.833000 4169 ts_tablet_manager.cc:1918] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:18.833196 4169 log.cc:1196] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a81334e7da3741a9b24f8978312181b9
I20260811 18:57:18.803264 4167 rpcz_store.cc:276] 0811 18:57:15.761398 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.761966 (+ 568us) service_pool.cc:227] Handling call
0811 18:57:18.668051 (+2906085us) ts_tablet_manager.cc:1145] Deleting tablet 508f0b39cd654aa991a2928a4b067762
0811 18:57:18.668065 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.731296 (+ 63231us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.779910 (+ 48614us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.788214 (+ 8304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2860772}
I20260811 18:57:18.833299 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cafb4240a8584ec184a99a633b46eeb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.833686 4169 ts_tablet_manager.cc:1939] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:18.833997 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cafb4240a8584ec184a99a633b46eeb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.834059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 315) took 2407 ms. Trace:
I20260811 18:57:18.835347 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 508f0b39cd654aa991a2928a4b067762 (table default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]) successfully deleted
I20260811 18:57:18.835791 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51c66177bc494cd0a293e2959ff854e5" } new_table_name: "default.Table_b3ee7e916a0c4d2180027e84bc055f54" modify_external_catalogs: true
I20260811 18:57:18.837062 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e746326e8bff404aa5b3bc1fac099489" } new_table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" modify_external_catalogs: true
I20260811 18:57:18.838871 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e12ffa777d684216bfc19d52f2dad4be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.839435 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e12ffa777d684216bfc19d52f2dad4be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.839572 4163 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.839742 4163 ts_tablet_manager.cc:1397] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.221s user 0.003s sys 0.007s
I20260811 18:57:18.840553 4163 raft_consensus.cc:348] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.840816 4163 raft_consensus.cc:374] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.840935 4163 raft_consensus.cc:729] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.841137 4167 tablet_replica.cc:330] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.841274 4163 consensus_queue.cc:260] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.841562 4167 raft_consensus.cc:2229] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.841861 4167 raft_consensus.cc:2258] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.842094 4163 ts_tablet_manager.cc:1428] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 18:57:18.842585 4163 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.842834 4167 ts_tablet_manager.cc:1905] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.841189 4162 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.844800 4162 ts_tablet_manager.cc:1397] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.001s
I20260811 18:57:18.846065 4162 raft_consensus.cc:348] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.846590 4162 raft_consensus.cc:374] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.846721 4162 raft_consensus.cc:729] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.847083 4162 consensus_queue.cc:260] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.850492 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_786c886ed1164646855f804b8397bed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.851224 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_786c886ed1164646855f804b8397bed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.854278 4836 raft_consensus.cc:482] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.854518 4836 raft_consensus.cc:504] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.855386 4836 leader_election.cc:288] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.858248 4162 ts_tablet_manager.cc:1428] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:18.860041 4162 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.865285 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9c6377231db8495796c02e71f7f71991" } new_table_name: "default.Table_4d1fcb681d504d57b9d288e08be657c2" modify_external_catalogs: true
I20260811 18:57:18.871320 3676 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:18.871716 4836 raft_consensus.cc:2781] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:18.871963 4836 raft_consensus.cc:686] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:18.872354 4836 consensus_queue.cc:237] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.874595 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.875335 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.876334 4162 tablet_bootstrap.cc:654] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.885305 4167 ts_tablet_manager.cc:1918] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:18.888446 3676 leader_election.cc:341] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: cf285cdf5235451bb1531cb294017101
I20260811 18:57:18.901913 4162 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.902988 4162 ts_tablet_manager.cc:1397] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.004s
I20260811 18:57:18.904223 4162 raft_consensus.cc:348] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.904804 4162 raft_consensus.cc:374] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.904947 4162 raft_consensus.cc:729] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.905308 4162 consensus_queue.cc:260] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.906167 4162 ts_tablet_manager.cc:1428] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:18.906738 4162 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.909170 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" } modify_external_catalogs: true
I20260811 18:57:18.909438 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" } modify_external_catalogs: true
I20260811 18:57:18.910676 4162 tablet_bootstrap.cc:654] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.911484 4878 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.911692 4878 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.911966 4167 log.cc:1196] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:18.912518 4167 ts_tablet_manager.cc:1939] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.912704 4878 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.912765 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:18.913239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 271) took 3115 ms. Trace:
I20260811 18:57:18.913631 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 3744b3d608754de68c4c57b072d1dc15 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.913944 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.913378 4167 rpcz_store.cc:276] 0811 18:57:15.797696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.799952 (+ 2256us) service_pool.cc:227] Handling call
0811 18:57:18.833809 (+3033857us) ts_tablet_manager.cc:1145] Deleting tablet 2f9fa42654cf4ec18220e5fb173ddef7
0811 18:57:18.833832 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.880755 (+ 46923us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.885174 (+ 4419us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.913161 (+ 27987us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2996941}
W20260811 18:57:18.915128 3824 leader_election.cc:341] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.915853 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 276) took 3040 ms. Trace:
I20260811 18:57:18.916008 4167 rpcz_store.cc:276] 0811 18:57:15.875490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.877707 (+ 2217us) service_pool.cc:227] Handling call
0811 18:57:18.915544 (+3037837us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:18.915562 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.915764 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3037337}
I20260811 18:57:18.917372 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2f9fa42654cf4ec18220e5fb173ddef7 (table default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]) successfully deleted
I20260811 18:57:18.919209 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:18.925678 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 135a8c34b93544d98383fc3103966a54 (DEFAULT_TABLE table=default.Table_7bfc22ccd3e44c679f5e951318bb0af3 [id=e07603a929ca4721b5b978a215d52fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.926368 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135a8c34b93544d98383fc3103966a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.930511 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:18.930963 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:18.931169 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9722572533de436cb9149e6230570382 (DEFAULT_TABLE table=default.Table_3e695533c42647128860db601dcc0b5f [id=d1ff7e5df21346dc9c4f609331322f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931172 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 930b6e8839e24ec9a69854349bc3ecf8 (DEFAULT_TABLE table=default.Table_3294569c49f84a2291010174baee10af [id=d7d11af2eee54ab880a15c525b7ad0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931352 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 (DEFAULT_TABLE table=default.Table_be30e8194fd44874a1fdd996128eb4d4 [id=bd75749ddd8d4224a75817f8d95f59d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931749 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b6e8839e24ec9a69854349bc3ecf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.931816 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 (DEFAULT_TABLE table=default.Table_be30e8194fd44874a1fdd996128eb4d4 [id=bd75749ddd8d4224a75817f8d95f59d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931907 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d81e93739f44d9e99b6ebc647e1d0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.932164 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d81e93739f44d9e99b6ebc647e1d0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.933574 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_704e71bf00da49609103fb3fde6afb2c" } modify_external_catalogs: true
I20260811 18:57:18.933818 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_704e71bf00da49609103fb3fde6afb2c" } modify_external_catalogs: true
I20260811 18:57:18.934617 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9722572533de436cb9149e6230570382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.934707 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 930b6e8839e24ec9a69854349bc3ecf8 (DEFAULT_TABLE table=default.Table_3294569c49f84a2291010174baee10af [id=d7d11af2eee54ab880a15c525b7ad0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.935177 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b6e8839e24ec9a69854349bc3ecf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.935537 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.935812 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:18.938383 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:18.941054 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 9722572533de436cb9149e6230570382 (DEFAULT_TABLE table=default.Table_3e695533c42647128860db601dcc0b5f [id=d1ff7e5df21346dc9c4f609331322f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.941684 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9722572533de436cb9149e6230570382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.946482 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 135a8c34b93544d98383fc3103966a54 (DEFAULT_TABLE table=default.Table_7bfc22ccd3e44c679f5e951318bb0af3 [id=e07603a929ca4721b5b978a215d52fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.947064 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135a8c34b93544d98383fc3103966a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.947840 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 (DEFAULT_TABLE table=default.Table_0b53eaa03dd34a22a14fa65da3866fe4 [id=45f3f13a059e4e2e8624e5530abca5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.948282 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc6cce2c19746219b8b0fe54b13dcd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.951049 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 (DEFAULT_TABLE table=default.Table_0b53eaa03dd34a22a14fa65da3866fe4 [id=45f3f13a059e4e2e8624e5530abca5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.951529 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc6cce2c19746219b8b0fe54b13dcd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.952986 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 (DEFAULT_TABLE table=default.Table_914c8b90fadf48e88b75d13897462d84 [id=cf8fdaf8a86f4648880083c2ccd856d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.953912 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0e0d139c924f1dafe42da0bedada72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.955392 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a (DEFAULT_TABLE table=default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 [id=dc139df6f9814c7891ddbd2d85d16e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.955894 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54fa32f8fa1144c4bd9dcd00a550001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.959128 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a (DEFAULT_TABLE table=default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 [id=dc139df6f9814c7891ddbd2d85d16e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.959580 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54fa32f8fa1144c4bd9dcd00a550001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.960742 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0747ef88574c68ae074732cb7095b9 (DEFAULT_TABLE table=default.Table_e6a28da9c6fa458d9daa95fdbd30920c [id=a2e4dc5b6af440d7bdf3d7b9dfffc9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.961301 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0747ef88574c68ae074732cb7095b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:18.965018 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b404e99e9414c67b3cd5aff12203a53 consensus state has no leader
I20260811 18:57:18.967267 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 (DEFAULT_TABLE table=default.Table_914c8b90fadf48e88b75d13897462d84 [id=cf8fdaf8a86f4648880083c2ccd856d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.967857 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0e0d139c924f1dafe42da0bedada72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.976145 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2be6762e326448d7b677a5641e2b3103 (DEFAULT_TABLE table=default.Table_01f64f05d1ee44029680c5423fd0d81b [id=9fd91de4ece84d0d8ecf793ca643843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.976787 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be6762e326448d7b677a5641e2b3103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.977731 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.979178 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 2be6762e326448d7b677a5641e2b3103 (DEFAULT_TABLE table=default.Table_01f64f05d1ee44029680c5423fd0d81b [id=9fd91de4ece84d0d8ecf793ca643843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.979772 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be6762e326448d7b677a5641e2b3103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.981065 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e (DEFAULT_TABLE table=default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f [id=c4fcc501bfd9482da8e0e24cae8380ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.981557 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.982908 4729 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.983134 4729 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.991577 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_655b56a4862144f1afd52a620c3715d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.992264 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e (DEFAULT_TABLE table=default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f [id=c4fcc501bfd9482da8e0e24cae8380ee]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:18.992352 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_655b56a4862144f1afd52a620c3715d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.992892 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.994258 4729 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.969805 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0747ef88574c68ae074732cb7095b9 (DEFAULT_TABLE table=default.Table_e6a28da9c6fa458d9daa95fdbd30920c [id=a2e4dc5b6af440d7bdf3d7b9dfffc9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.995002 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd" } modify_external_catalogs: true
I20260811 18:57:18.995268 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd" } modify_external_catalogs: true
I20260811 18:57:18.995821 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0747ef88574c68ae074732cb7095b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.996307 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.009454 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.010228 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:19.014598 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c29af3615e74722a809d3e5ede9b9a1 consensus state has no leader
I20260811 18:57:19.016919 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_278e0a9db8364b7cb00a0920de4b0561"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.017637 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_278e0a9db8364b7cb00a0920de4b0561 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.021121 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4" } modify_external_catalogs: true
I20260811 18:57:19.021400 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4" } modify_external_catalogs: true
I20260811 18:57:19.028844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 277) took 3153 ms. Trace:
I20260811 18:57:19.029131 4167 rpcz_store.cc:276] 0811 18:57:15.875756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878248 (+ 2492us) service_pool.cc:227] Handling call
0811 18:57:18.990795 (+3112547us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:18.990814 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.028760 (+ 37946us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112287}
W20260811 18:57:19.029794 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 already in progress: creating tablet
I20260811 18:57:19.030038 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 278) took 3154 ms. Trace:
I20260811 18:57:19.030170 4167 rpcz_store.cc:276] 0811 18:57:15.875988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878531 (+ 2543us) service_pool.cc:227] Handling call
0811 18:57:19.029797 (+3151266us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:19.029813 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.029963 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3150981}
I20260811 18:57:19.030828 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 279) took 3154 ms. Trace:
I20260811 18:57:19.030942 4878 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.030948 4167 rpcz_store.cc:276] 0811 18:57:15.876245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878871 (+ 2626us) service_pool.cc:227] Handling call
0811 18:57:19.030652 (+3151781us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:19.030664 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.030760 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3151592}
W20260811 18:57:19.031206 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45aff3e8cb0c4f7191e4522c6ef7d156 already in progress: creating tablet
I20260811 18:57:19.031195 4878 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.031467 4167 tablet_replica.cc:330] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:19.031527 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 36724a83bf9844b9a27a88ca36a9bf50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36724a83bf9844b9a27a88ca36a9bf50 already in progress: creating tablet
I20260811 18:57:19.031858 4167 raft_consensus.cc:2229] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.031980 4878 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.032291 4167 raft_consensus.cc:2258] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.032586 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4fbcb57a32f54fc3a493bc5412e94051" } new_table_name: "default.Table_69803efc857941408d32f7da625acfb6" modify_external_catalogs: true
I20260811 18:57:19.038678 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.039171 3891 raft_consensus.cc:2454] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.043840 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0540e4e43ea34fbfbc66b6035c60684d
I20260811 18:57:19.046330 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:19.049227 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_460a6647324c49b78cd79e59e974732b" } new_table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" modify_external_catalogs: true
I20260811 18:57:18.834169 4169 rpcz_store.cc:276] 0811 18:57:16.426244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.426326 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:18.597574 (+2171248us) ts_tablet_manager.cc:1145] Deleting tablet a81334e7da3741a9b24f8978312181b9
0811 18:57:18.597588 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.755402 (+157814us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.832866 (+ 77464us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.834002 (+ 1136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2168629}
I20260811 18:57:19.068478 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 316) took 2639 ms. Trace:
I20260811 18:57:19.068742 4169 rpcz_store.cc:276] 0811 18:57:16.429380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.431284 (+ 1904us) service_pool.cc:227] Handling call
0811 18:57:19.068250 (+2636966us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:19.068263 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.068403 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633325}
I20260811 18:57:19.069381 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a81334e7da3741a9b24f8978312181b9 (table default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd]) successfully deleted
I20260811 18:57:19.069636 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 317) took 2634 ms. Trace:
I20260811 18:57:19.033030 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:19.070091 4162 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.070254 4162 ts_tablet_manager.cc:1397] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.164s user 0.002s sys 0.003s
I20260811 18:57:19.070353 4167 ts_tablet_manager.cc:1905] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.071162 4162 raft_consensus.cc:348] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.071426 4162 raft_consensus.cc:374] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.050936 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.071463 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:19.071522 4162 raft_consensus.cc:729] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.071785 4162 consensus_queue.cc:260] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.072019 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 3744b3d608754de68c4c57b072d1dc15 deleted and replaced by tablet 91818d0cbe144a67a458a6648968bf5b
I20260811 18:57:19.072393 4162 ts_tablet_manager.cc:1428] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:19.072361 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.073033 4162 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.075124 4162 tablet_bootstrap.cc:654] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.075623 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3744b3d608754de68c4c57b072d1dc15 with delete_type TABLET_DATA_DELETED (replaced by tablet 91818d0cbe144a67a458a6648968bf5b) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:19.076364 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2c0f6df7a291456d80538e2b76417873 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c0f6df7a291456d80538e2b76417873 already in progress: creating tablet
I20260811 18:57:19.081122 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:19.083930 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab9634fb958f4d44840fa6f1709ce326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.084728 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9634fb958f4d44840fa6f1709ce326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.084753 4162 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.084937 4162 ts_tablet_manager.cc:1397] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20260811 18:57:19.085656 4162 raft_consensus.cc:348] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.085914 4162 raft_consensus.cc:374] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.085983 4162 raft_consensus.cc:729] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.086222 4162 consensus_queue.cc:260] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.089118 4162 ts_tablet_manager.cc:1428] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:19.089505 4162 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.091794 4162 tablet_bootstrap.cc:654] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.092005 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.092320 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.097836 3676 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.098197 4836 raft_consensus.cc:2781] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.098309 4836 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.098464 4836 raft_consensus.cc:3037] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.101418 4836 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.104949 4836 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.107244 4167 ts_tablet_manager.cc:1918] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:19.070809 3824 leader_election.cc:341] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.109892 4167 log.cc:1196] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:19.110749 4167 ts_tablet_manager.cc:1939] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.084514 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.111498 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 3744b3d608754de68c4c57b072d1dc15 with delete_type TABLET_DATA_DELETED (replaced by tablet 91818d0cbe144a67a458a6648968bf5b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.112164 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.112286 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a0116997a1c64e658045c319f1c45861
I20260811 18:57:19.112459 3891 raft_consensus.cc:3037] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.114286 4169 rpcz_store.cc:276] 0811 18:57:16.435177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.435234 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:19.069457 (+2634223us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:19.069469 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.069575 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2631868}
I20260811 18:57:19.114861 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 318) took 2676 ms. Trace:
I20260811 18:57:19.114941 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.114974 4169 rpcz_store.cc:276] 0811 18:57:16.437945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.438000 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:19.114627 (+2676627us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:19.114641 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.114799 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2676286}
I20260811 18:57:19.115288 4169 tablet_replica.cc:330] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:19.115299 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c1ceb39258a049f08a1f48c40df65d8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.115530 4169 raft_consensus.cc:2229] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.115623 3891 raft_consensus.cc:2454] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.115654 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 274) took 3278 ms. Trace:
I20260811 18:57:19.115728 4169 raft_consensus.cc:2258] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.115767 4167 rpcz_store.cc:276] 0811 18:57:15.837619 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.837757 (+ 138us) service_pool.cc:227] Handling call
0811 18:57:19.031364 (+3193607us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:19.031378 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.102634 (+ 71256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.107092 (+ 4458us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.115614 (+ 8522us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2983666}
W20260811 18:57:19.115974 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1ceb39258a049f08a1f48c40df65d8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.116441 4169 ts_tablet_manager.cc:1905] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.116447 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 280) took 3239 ms. Trace:
I20260811 18:57:19.116576 3676 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.116885 4836 raft_consensus.cc:2781] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.117024 4836 raft_consensus.cc:686] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.117277 4836 consensus_queue.cc:237] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.117453 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8426ccbe530446fcbaa162a03cbb1892 (table default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]) successfully deleted
I20260811 18:57:19.118429 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 460e03cc14444c83b6c08ca194ca5a8d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.119197 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c8efe53501a143ed90881e375a449e81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.119807 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8efe53501a143ed90881e375a449e81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.119771 4167 rpcz_store.cc:276] 0811 18:57:15.876676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.879125 (+ 2449us) service_pool.cc:227] Handling call
0811 18:57:19.116269 (+3237144us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:19.116281 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.116402 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3068006}
W20260811 18:57:19.120275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ad424c8843ff434ba0449a86849bb5a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad424c8843ff434ba0449a86849bb5a0 already in progress: creating tablet
I20260811 18:57:19.120322 4167 tablet_replica.cc:330] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.120831 4167 raft_consensus.cc:2229] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.121033 4167 raft_consensus.cc:2258] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:19.121089 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 799ba9a044e145ec89d0025c03f391ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 799ba9a044e145ec89d0025c03f391ea already in progress: creating tablet
I20260811 18:57:19.121732 4167 ts_tablet_manager.cc:1905] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.122932 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.124433 4900 tablet.cc:1722] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:19.085913 4163 tablet_bootstrap.cc:654] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.140364 4898 tablet.cc:1722] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:19.140949 4836 raft_consensus.cc:482] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.141144 4836 raft_consensus.cc:504] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.143636 4836 leader_election.cc:288] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:19.144014 3676 leader_election.cc:341] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:19.146449 4162 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.146608 4162 ts_tablet_manager.cc:1397] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.057s user 0.001s sys 0.004s
I20260811 18:57:19.147132 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:19.147970 4162 raft_consensus.cc:348] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.148475 4836 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.148631 4836 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.148896 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:19.149348 4836 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.150272 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.150527 3888 raft_consensus.cc:2454] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.150800 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 27cda53b7ca54346a7a22bf26f5db990 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.150936 3676 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.151088 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.151324 4729 raft_consensus.cc:2781] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.151444 4729 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.151541 4729 raft_consensus.cc:3037] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.152042 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.152452 4162 raft_consensus.cc:374] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.152575 4162 raft_consensus.cc:729] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.152810 4162 consensus_queue.cc:260] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.153520 4162 ts_tablet_manager.cc:1428] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260811 18:57:19.155093 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.156153 4729 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.158159 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.158478 3888 raft_consensus.cc:3037] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.159441 4162 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.161271 4162 tablet_bootstrap.cc:654] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.163533 4878 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.163769 4878 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.164691 4878 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.166679 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:19.167877 3824 leader_election.cc:341] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.169660 3888 raft_consensus.cc:2454] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.169759 4167 ts_tablet_manager.cc:1918] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:19.171077 3676 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.171479 4836 raft_consensus.cc:2781] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.171635 4836 raft_consensus.cc:686] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.171737 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f6a3985ef18b44138437cd5cd54f34ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.171927 4836 consensus_queue.cc:237] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.172427 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6a3985ef18b44138437cd5cd54f34ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.172859 4167 log.cc:1196] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:19.173329 4167 ts_tablet_manager.cc:1939] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.174026 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 297) took 2874 ms. Trace:
I20260811 18:57:19.174152 4167 rpcz_store.cc:276] 0811 18:57:16.299180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.443931 (+144751us) service_pool.cc:227] Handling call
0811 18:57:19.120245 (+2676314us) ts_tablet_manager.cc:1145] Deleting tablet 0e1efffade0f486d8d627e56e0be6ff6
0811 18:57:19.120261 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.143647 (+ 23386us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.169617 (+ 25970us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.173950 (+ 4333us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659976}
I20260811 18:57:19.175276 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c25e17a6776443429e46c4aab57537c7
I20260811 18:57:19.175654 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_27d36ce508354815add475acc3f872eb" } new_table_name: "default.Table_821a790ed161401bb9758d71a0760916" modify_external_catalogs: true
I20260811 18:57:19.177147 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 296) took 2883 ms. Trace:
I20260811 18:57:19.177295 4167 rpcz_store.cc:276] 0811 18:57:16.293383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.442227 (+148844us) service_pool.cc:227] Handling call
0811 18:57:19.176897 (+2734670us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:19.176915 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.177066 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2606719}
W20260811 18:57:19.177825 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b3ec38f9b55489ba00ed76f9e4456f5 already in progress: creating tablet
I20260811 18:57:19.178033 4167 tablet_replica.cc:330] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.178313 4167 raft_consensus.cc:2229] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.178766 4167 raft_consensus.cc:2258] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.183473 4163 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.183697 4163 ts_tablet_manager.cc:1397] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.341s user 0.004s sys 0.001s
I20260811 18:57:19.184639 4163 raft_consensus.cc:348] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.184945 4163 raft_consensus.cc:374] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.185039 4163 raft_consensus.cc:729] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.185431 4163 consensus_queue.cc:260] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.186961 4167 ts_tablet_manager.cc:1905] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.191627 4162 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.194418 4162 ts_tablet_manager.cc:1397] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.005s
I20260811 18:57:19.195443 4729 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.196556 4729 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.196767 4729 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.198889 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.199303 3888 raft_consensus.cc:2454] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.199752 3676 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.200045 4903 raft_consensus.cc:2781] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.200160 4903 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.200281 4903 raft_consensus.cc:3037] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.202863 4162 raft_consensus.cc:348] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.203164 4162 raft_consensus.cc:374] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.203270 4162 raft_consensus.cc:729] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.203316 4903 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.204298 4729 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.204766 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.205242 3888 raft_consensus.cc:3037] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.204557 4903 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.211236 4162 consensus_queue.cc:260] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.212651 4162 ts_tablet_manager.cc:1428] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260811 18:57:19.214303 3888 raft_consensus.cc:2454] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.214978 3676 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.215359 4729 raft_consensus.cc:2781] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.215627 4729 raft_consensus.cc:686] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.219717 4729 consensus_queue.cc:237] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:19.222463 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6432abd4fc5c4f9ca95971ca4795e1cf consensus state has no leader
I20260811 18:57:19.222905 4167 ts_tablet_manager.cc:1918] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:19.223053 4167 log.cc:1196] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a81334e7da3741a9b24f8978312181b9
W20260811 18:57:19.223558 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf5749d4eebd42319df1b52eeba4c062 has no consensus state
I20260811 18:57:19.223562 4167 ts_tablet_manager.cc:1939] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.224859 4162 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.226773 4162 tablet_bootstrap.cc:654] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.231180 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 300) took 2850 ms. Trace:
I20260811 18:57:19.231369 4167 rpcz_store.cc:276] 0811 18:57:16.380305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.471548 (+ 91243us) service_pool.cc:227] Handling call
0811 18:57:19.177915 (+2706367us) ts_tablet_manager.cc:1145] Deleting tablet a81334e7da3741a9b24f8978312181b9
0811 18:57:19.177931 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.197101 (+ 19170us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.222759 (+ 25658us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.231133 (+ 8374us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2607207}
I20260811 18:57:19.231935 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a81334e7da3741a9b24f8978312181b9 (table default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd]) successfully deleted
I20260811 18:57:19.232138 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 301) took 2848 ms. Trace:
I20260811 18:57:19.232302 4167 rpcz_store.cc:276] 0811 18:57:16.383646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.495343 (+111697us) service_pool.cc:227] Handling call
0811 18:57:19.231945 (+2736602us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:19.231962 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.232102 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660828}
W20260811 18:57:19.232827 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2c0f6df7a291456d80538e2b76417873 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c0f6df7a291456d80538e2b76417873 already in progress: creating tablet
I20260811 18:57:19.232933 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 302) took 2839 ms. Trace:
I20260811 18:57:19.233071 4167 rpcz_store.cc:276] 0811 18:57:16.399319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.525560 (+126241us) service_pool.cc:227] Handling call
0811 18:57:19.232785 (+2707225us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:19.232799 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.232899 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660567}
I20260811 18:57:19.233578 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 303) took 2828 ms. Trace:
I20260811 18:57:19.233732 4167 rpcz_store.cc:276] 0811 18:57:16.404801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.538119 (+133318us) service_pool.cc:227] Handling call
0811 18:57:19.233438 (+2695319us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:19.233451 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.233544 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660722}
W20260811 18:57:19.234123 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 799ba9a044e145ec89d0025c03f391ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 799ba9a044e145ec89d0025c03f391ea already in progress: creating tablet
I20260811 18:57:19.234297 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 304) took 2553 ms. Trace:
I20260811 18:57:19.234452 4167 rpcz_store.cc:276] 0811 18:57:16.681162 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.681233 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:19.234113 (+2552880us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:19.234125 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.234248 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2525839}
W20260811 18:57:19.234910 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f8eb041f3d704762868a048faec0e0fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f8eb041f3d704762868a048faec0e0fc already in progress: creating tablet
I20260811 18:57:19.235005 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 314) took 2389 ms. Trace:
I20260811 18:57:19.235133 4167 rpcz_store.cc:276] 0811 18:57:16.845875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.845952 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:19.234858 (+2388906us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:19.234872 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.234965 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2382677}
W20260811 18:57:19.235572 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 19243ab3d6744aaa87a133f1462cd2fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19243ab3d6744aaa87a133f1462cd2fa already in progress: creating tablet
I20260811 18:57:19.235721 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 321) took 2182 ms. Trace:
I20260811 18:57:19.235867 4167 rpcz_store.cc:276] 0811 18:57:17.053413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.053478 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:19.235561 (+2182083us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:19.235574 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.235680 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2170290}
I20260811 18:57:19.236425 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 323) took 2161 ms. Trace:
I20260811 18:57:19.236583 4167 rpcz_store.cc:276] 0811 18:57:17.074603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.075715 (+ 1112us) service_pool.cc:227] Handling call
0811 18:57:19.236271 (+2160556us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:19.236284 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.236392 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2143879}
I20260811 18:57:19.237933 4163 ts_tablet_manager.cc:1428] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.054s user 0.001s sys 0.002s
I20260811 18:57:19.240495 4163 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.242591 4167 tablet_replica.cc:330] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.242951 4167 raft_consensus.cc:2229] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.243124 4163 tablet_bootstrap.cc:654] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.243399 4167 raft_consensus.cc:2258] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.244514 4167 ts_tablet_manager.cc:1905] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.248392 4169 ts_tablet_manager.cc:1918] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:19.250890 4162 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.251078 4162 ts_tablet_manager.cc:1397] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.001s
I20260811 18:57:19.252050 4162 raft_consensus.cc:348] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.252368 4162 raft_consensus.cc:374] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.252483 4162 raft_consensus.cc:729] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.252758 4162 consensus_queue.cc:260] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.263523 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.264228 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74385eb07bdb4cb39d7783a47ae5781c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.265708 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_42b47e0e7e404c8c976e7472833fd243" } new_table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" modify_external_catalogs: true
I20260811 18:57:19.267483 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a8d2d56f43544d68adbbd2848fc00dde" } new_table_name: "default.Table_079d99e744054f2a8a2d6ac002aca4e3" modify_external_catalogs: true
I20260811 18:57:19.268796 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea042ebbfb444a2086b87857b6775140"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.269384 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea042ebbfb444a2086b87857b6775140 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.270411 4162 ts_tablet_manager.cc:1428] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:19.270893 4162 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.271041 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_771bec99eb584420a45d22e9b038229a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.271579 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_771bec99eb584420a45d22e9b038229a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.272959 4162 tablet_bootstrap.cc:654] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.276786 4167 ts_tablet_manager.cc:1918] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260811 18:57:19.277145 4167 log.cc:1196] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:19.277866 4167 ts_tablet_manager.cc:1939] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.278824 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 324) took 2203 ms. Trace:
I20260811 18:57:19.278995 4167 rpcz_store.cc:276] 0811 18:57:17.074831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.088514 (+ 13683us) service_pool.cc:227] Handling call
0811 18:57:19.242496 (+2153982us) ts_tablet_manager.cc:1145] Deleting tablet 7b227bc7305f4b2dae28f04e7001553b
0811 18:57:19.242515 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.248911 (+ 6396us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.276602 (+ 27691us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.278741 (+ 2139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2145625}
I20260811 18:57:19.281126 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]) successfully deleted
I20260811 18:57:19.281164 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 322) took 2217 ms. Trace:
I20260811 18:57:19.281330 4167 rpcz_store.cc:276] 0811 18:57:17.063501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.064471 (+ 970us) service_pool.cc:227] Handling call
0811 18:57:19.280925 (+2216454us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:19.280939 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.281086 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2183572}
I20260811 18:57:19.281786 4167 tablet_replica.cc:330] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:19.281875 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cbf6ce167fb348be8d61372791f0cbca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbf6ce167fb348be8d61372791f0cbca already in progress: creating tablet
I20260811 18:57:19.282146 4167 raft_consensus.cc:2229] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.290845 4167 pending_rounds.cc:62] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.290934 4167 pending_rounds.cc:68] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106479046656 op_type: NO_OP noop_request { }
I20260811 18:57:19.291134 4167 raft_consensus.cc:2866] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.291268 4167 raft_consensus.cc:2258] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.291990 4167 ts_tablet_manager.cc:1905] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.289397 4162 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.298739 4162 ts_tablet_manager.cc:1397] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.004s
W20260811 18:57:19.298847 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68a7051bc539468aafba30fbaf0669d4 consensus state has no leader
I20260811 18:57:19.300948 4162 raft_consensus.cc:348] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.301273 4162 raft_consensus.cc:374] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.301364 4162 raft_consensus.cc:729] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.302608 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_786c886ed1164646855f804b8397bed1" } modify_external_catalogs: true
I20260811 18:57:19.302575 4162 consensus_queue.cc:260] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.302836 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_786c886ed1164646855f804b8397bed1" } modify_external_catalogs: true
I20260811 18:57:19.309249 4162 ts_tablet_manager.cc:1428] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:19.309275 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet c2247194db704f9abd54a8a0df1ca566 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.312314 3553 catalog_manager.cc:5495] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:19.312834 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:19.265894 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.315479 4169 log.cc:1196] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f8eb041f3d704762868a048faec0e0fc
I20260811 18:57:19.316437 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 91c8101778b54239b834a4f11ef4a08a (DEFAULT_TABLE table=default.Table_cafb4240a8584ec184a99a633b46eeb1 [id=471bee2cce33409c8c4b890f9ee1297c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.316704 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 91c8101778b54239b834a4f11ef4a08a (DEFAULT_TABLE table=default.Table_cafb4240a8584ec184a99a633b46eeb1 [id=471bee2cce33409c8c4b890f9ee1297c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.316902 4162 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.316968 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c8101778b54239b834a4f11ef4a08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.317215 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c8101778b54239b834a4f11ef4a08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.318661 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 (DEFAULT_TABLE table=default.Table_cd4eb198addc42f2b6d860ef805c589f [id=f2e5c5f123ca451f8f217e96692247ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.319044 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7991dabe263b44f6b7b7d1d43b96ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.320595 4162 tablet_bootstrap.cc:654] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.322036 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7f48893a8614cacb6393af837f050f3" } modify_external_catalogs: true
I20260811 18:57:19.322312 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7f48893a8614cacb6393af837f050f3" } modify_external_catalogs: true
I20260811 18:57:19.323366 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 92e49846ab714dd294acdd09d49a83bd (DEFAULT_TABLE table=default.Table_ea0cc7a576d64d1c934364316cae3c70 [id=fa447a325fe54e78a18b949dd3706dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.323865 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e49846ab714dd294acdd09d49a83bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.326689 3861 tablet_service.cc:1467] Processing CreateTablet for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (DEFAULT_TABLE table=default.Table_fdae8856d30a46259e91345b60181c2b [id=de9b6daef0ed416aa79b0dc0f4b02c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.327085 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.328863 3715 tablet_service.cc:1467] Processing CreateTablet for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (DEFAULT_TABLE table=default.Table_fdae8856d30a46259e91345b60181c2b [id=de9b6daef0ed416aa79b0dc0f4b02c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.331100 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 (DEFAULT_TABLE table=default.Table_cd4eb198addc42f2b6d860ef805c589f [id=f2e5c5f123ca451f8f217e96692247ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.331606 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7991dabe263b44f6b7b7d1d43b96ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.336303 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c0886d985c1343cab4196151b02eff8d (DEFAULT_TABLE table=default.Table_a93ee06ed76342e082697ded810c4c99 [id=a50bb8f0718e4073bfe4254492df7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.336808 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0886d985c1343cab4196151b02eff8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.339145 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 380c1f53b3ac4509a09b25a2800028af (DEFAULT_TABLE table=default.Table_6a0c48e6299546a2921a7a7db03f1032 [id=f7bad72dc09b4b97abea39686ec4ae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.339612 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 380c1f53b3ac4509a09b25a2800028af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.342659 3860 tablet_service.cc:1467] Processing CreateTablet for tablet bf5749d4eebd42319df1b52eeba4c062 (DEFAULT_TABLE table=default.Table_3428bf646eff41ebb8394b062d83cbe0 [id=14e90fa7300e4a5ea10d2da1e28e24a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.342679 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 92e49846ab714dd294acdd09d49a83bd (DEFAULT_TABLE table=default.Table_ea0cc7a576d64d1c934364316cae3c70 [id=fa447a325fe54e78a18b949dd3706dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.343199 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5749d4eebd42319df1b52eeba4c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.344399 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.346283 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e49846ab714dd294acdd09d49a83bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.349143 3862 tablet_service.cc:1467] Processing CreateTablet for tablet d8aea8dc31914052832bdc83d638461f (DEFAULT_TABLE table=default.Table_459873f5dbe2412c80b73ca35dce90ed [id=a2bf91886ddb4b9bb10366b77185f531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.349561 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8aea8dc31914052832bdc83d638461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.351764 4167 ts_tablet_manager.cc:1918] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:19.353344 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4093564a88264ed59c945523c9ddc9b3 (DEFAULT_TABLE table=default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.353960 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4093564a88264ed59c945523c9ddc9b3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:19.355510 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 122b7898fea44ef78f269b1505d8ddc4 consensus state has no leader
I20260811 18:57:19.356000 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef613c8dc0524f318790523ba7652317" } modify_external_catalogs: true
I20260811 18:57:19.356283 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef613c8dc0524f318790523ba7652317" } modify_external_catalogs: true
I20260811 18:57:19.356526 4167 log.cc:1196] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ed3346bd12d04c45a511308c427856e5
I20260811 18:57:19.357426 4167 ts_tablet_manager.cc:1939] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.359372 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 325) took 2265 ms. Trace:
I20260811 18:57:19.359563 4167 rpcz_store.cc:276] 0811 18:57:17.092618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.092733 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:19.281687 (+2188954us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:19.281699 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.295821 (+ 14122us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.345724 (+ 49903us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.358178 (+ 12454us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2173744}
I20260811 18:57:19.360860 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 320) took 2310 ms. Trace:
I20260811 18:57:19.361064 4167 rpcz_store.cc:276] 0811 18:57:17.050242 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.088459 (+ 38217us) service_pool.cc:227] Handling call
0811 18:57:19.359964 (+2271505us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:19.359981 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.360122 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2250579}
W20260811 18:57:19.361696 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a994b3f4d7d4c92946ad8d595e6ad2b consensus state has no leader
I20260811 18:57:19.362388 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.363080 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.361711 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 380c1f53b3ac4509a09b25a2800028af (DEFAULT_TABLE table=default.Table_6a0c48e6299546a2921a7a7db03f1032 [id=f7bad72dc09b4b97abea39686ec4ae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.368300 3708 tablet_service.cc:1467] Processing CreateTablet for tablet c0886d985c1343cab4196151b02eff8d (DEFAULT_TABLE table=default.Table_a93ee06ed76342e082697ded810c4c99 [id=a50bb8f0718e4073bfe4254492df7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.369599 3721 tablet_service.cc:1467] Processing CreateTablet for tablet bf5749d4eebd42319df1b52eeba4c062 (DEFAULT_TABLE table=default.Table_3428bf646eff41ebb8394b062d83cbe0 [id=14e90fa7300e4a5ea10d2da1e28e24a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.375262 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ed3346bd12d04c45a511308c427856e5 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:19.375808 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 48d32bb4eca047e4ba786c04b8bebf8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48d32bb4eca047e4ba786c04b8bebf8b already in progress: creating tablet
I20260811 18:57:19.375993 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ccc63b46a4e346f79d1e03a82490794c (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.386550 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 380c1f53b3ac4509a09b25a2800028af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.394599 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 4093564a88264ed59c945523c9ddc9b3 (DEFAULT_TABLE table=default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.395566 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d8aea8dc31914052832bdc83d638461f (DEFAULT_TABLE table=default.Table_459873f5dbe2412c80b73ca35dce90ed [id=a2bf91886ddb4b9bb10366b77185f531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.396596 4729 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.396944 4729 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.398375 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.398540 3851 tablet_service.cc:1467] Processing CreateTablet for tablet ccc63b46a4e346f79d1e03a82490794c (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.398665 3888 raft_consensus.cc:2454] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.398991 4729 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.399276 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5749d4eebd42319df1b52eeba4c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.399340 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc63b46a4e346f79d1e03a82490794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.400497 3676 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.401006 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0886d985c1343cab4196151b02eff8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.403713 4167 tablet_replica.cc:330] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.405182 4729 raft_consensus.cc:2781] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.410454 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8aea8dc31914052832bdc83d638461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.411262 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b (DEFAULT_TABLE table=default.Table_d9b7a2cfa85347bf88241286c4bf0083 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.411466 4729 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.411672 4729 raft_consensus.cc:3037] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.412068 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1800e9e2b064a6fb546b9d33cefa68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.414810 4167 raft_consensus.cc:2229] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.415454 3717 tablet_service.cc:1467] Processing CreateTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b (DEFAULT_TABLE table=default.Table_d9b7a2cfa85347bf88241286c4bf0083 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.418665 4836 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.419216 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4093564a88264ed59c945523c9ddc9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.423550 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1800e9e2b064a6fb546b9d33cefa68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.429425 4836 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.431373 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.431747 3888 raft_consensus.cc:2454] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.430301 4836 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.432411 4903 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.432546 3676 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.433583 4903 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.434616 4903 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.435858 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.436257 3888 raft_consensus.cc:2454] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.437341 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc63b46a4e346f79d1e03a82490794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.441505 3676 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.443131 4836 raft_consensus.cc:2781] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.443397 4836 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.445219 4167 pending_rounds.cc:62] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.454763 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be1e5fa6b9304fd6bcb4304038be24f3" } modify_external_catalogs: true
I20260811 18:57:19.455173 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be1e5fa6b9304fd6bcb4304038be24f3" } modify_external_catalogs: true
I20260811 18:57:19.457351 4163 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.457697 4163 ts_tablet_manager.cc:1397] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.217s user 0.005s sys 0.000s
I20260811 18:57:19.445478 4836 raft_consensus.cc:3037] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.446506 4903 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.462863 4167 pending_rounds.cc:68] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400113725657088 op_type: NO_OP noop_request { }
I20260811 18:57:19.463454 4167 raft_consensus.cc:2866] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.463775 4167 raft_consensus.cc:2258] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.464704 4167 ts_tablet_manager.cc:1905] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.462394 4903 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.469857 4903 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.470263 4903 raft_consensus.cc:2781] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.471297 4903 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.471442 4903 raft_consensus.cc:3037] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.471662 4169 ts_tablet_manager.cc:1939] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:19.472187 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 319) took 2813 ms. Trace:
I20260811 18:57:19.472368 4169 rpcz_store.cc:276] 0811 18:57:16.659030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.659153 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:19.115205 (+2456052us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:19.115214 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.243539 (+128325us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.248217 (+ 4678us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.472138 (+223921us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2455645}
I20260811 18:57:19.473304 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 329) took 2633 ms. Trace:
I20260811 18:57:19.473697 4169 rpcz_store.cc:276] 0811 18:57:16.839609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.839680 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:19.472982 (+2633302us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:19.472997 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.473204 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2632860}
I20260811 18:57:19.474754 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 334) took 2539 ms. Trace:
I20260811 18:57:19.475224 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c] alter complete (version 2)
I20260811 18:57:19.476452 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 470efca550444d9cb55bb2fe2d951b1c
I20260811 18:57:19.477401 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:19.478202 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 deleted and replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7
I20260811 18:57:19.479521 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f8eb041f3d704762868a048faec0e0fc (table default.Table_45c439603b1140e998b5d9199026da49 [id=32914b6ff485486f843da54878e2d49b]) successfully deleted
W20260811 18:57:19.480036 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 19243ab3d6744aaa87a133f1462cd2fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19243ab3d6744aaa87a133f1462cd2fa already in progress: creating tablet
I20260811 18:57:19.493937 4162 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.494413 4162 ts_tablet_manager.cc:1397] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.178s user 0.020s sys 0.015s
I20260811 18:57:19.484264 4729 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.508244 4169 rpcz_store.cc:276] 0811 18:57:16.935221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.935298 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:19.474468 (+2539170us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:19.474482 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.474653 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2531308}
I20260811 18:57:19.509425 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 336) took 2567 ms. Trace:
I20260811 18:57:19.509788 4169 rpcz_store.cc:276] 0811 18:57:16.941869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.941952 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:19.509068 (+2567116us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:19.509086 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.509312 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2565489}
I20260811 18:57:19.511121 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 335) took 2574 ms. Trace:
W20260811 18:57:19.511044 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 48d32bb4eca047e4ba786c04b8bebf8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48d32bb4eca047e4ba786c04b8bebf8b already in progress: creating tablet
I20260811 18:57:19.511550 4169 rpcz_store.cc:276] 0811 18:57:16.936480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.936548 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:19.510767 (+2574219us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:19.510780 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.511012 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2566972}
I20260811 18:57:19.485929 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.459578 4163 raft_consensus.cc:348] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.512303 4163 raft_consensus.cc:374] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.512589 4163 raft_consensus.cc:729] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.513159 4163 consensus_queue.cc:260] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.515690 4162 raft_consensus.cc:348] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.521513 4729 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.525517 4903 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.527405 4163 ts_tablet_manager.cc:1428] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.069s user 0.001s sys 0.002s
I20260811 18:57:19.526052 4162 raft_consensus.cc:374] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.528223 4163 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.528825 4162 raft_consensus.cc:729] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.529230 4162 consensus_queue.cc:260] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.532315 4163 tablet_bootstrap.cc:654] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.533702 4903 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.533957 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:19.534739 3555 catalog_manager.cc:6565] ReplaceTablet: tablet c2247194db704f9abd54a8a0df1ca566 deleted and replaced by tablet 2dce6143309948d49e229dc9890d904f
I20260811 18:57:19.537528 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:19.540190 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8d1e244addf5462499fcadaa3332cba1
I20260811 18:57:19.542543 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a122fe2820f4d23b12edb970a94e958" } new_table_name: "default.Table_e496287ddd104a6db81655bea6a19802" modify_external_catalogs: true
I20260811 18:57:19.544608 4836 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.546686 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7455a95b7b2a4c5db623ad62d8332150"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.547569 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7455a95b7b2a4c5db623ad62d8332150 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.548547 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4981573f7a434b5da77a06d251ea684c" } modify_external_catalogs: true
I20260811 18:57:19.548900 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4981573f7a434b5da77a06d251ea684c" } modify_external_catalogs: true
I20260811 18:57:19.550877 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_914d7e775ebf4b26beed57cb21140166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.551563 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_914d7e775ebf4b26beed57cb21140166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.558517 4836 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.559983 4162 ts_tablet_manager.cc:1428] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.052s user 0.000s sys 0.004s
I20260811 18:57:19.560369 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.562067 4162 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.566423 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cdd3a653fc1b405cbfd89c5d98728e45" } new_table_name: "default.Table_2103ce2744174eec9f603b49604123e4" modify_external_catalogs: true
I20260811 18:57:19.558851 4914 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.566197 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a795d5395cc64155b84057a53c701064"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.571964 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:19.572444 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 91719.2us
W20260811 18:57:19.581918 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a795d5395cc64155b84057a53c701064 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.583820 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.584988 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01f622bd12ce4b2bb39355d87ed79fb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.586423 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d1eb56bed2494b7f85229d2c92bcb60c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.587146 3678 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.305s (SENT)
W20260811 18:57:19.587424 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1eb56bed2494b7f85229d2c92bcb60c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.568089 4914 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.594321 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.596972 4162 tablet_bootstrap.cc:654] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:19.538646 3824 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.175s (SENT)
I20260811 18:57:19.598173 3824 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:19.598628 3824 raft_consensus.cc:2670] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:19.599534 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 60931.1us
W20260811 18:57:19.600010 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cbf6ce167fb348be8d61372791f0cbca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbf6ce167fb348be8d61372791f0cbca already in progress: creating tablet
I20260811 18:57:19.540310 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.541530 4878 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.600710 4878 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.601632 4878 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.602834 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.607769 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.562498 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 337) took 2593 ms. Trace:
I20260811 18:57:19.575578 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.610737 3888 raft_consensus.cc:2454] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.576767 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.611231 3891 raft_consensus.cc:3037] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.597434 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.587903 4915 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 57e8f62fa63a4fee97f268f5b7d3ae24)
I20260811 18:57:19.577649 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.578526 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.614718 3890 raft_consensus.cc:3037] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.614740 3892 raft_consensus.cc:3037] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.607285 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:19.609313 3678 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.942s (SENT)
I20260811 18:57:19.619962 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.620122 3892 raft_consensus.cc:2454] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.620273 3888 raft_consensus.cc:2454] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.620916 4169 rpcz_store.cc:276] 0811 18:57:16.968908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.968996 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:19.562080 (+2593084us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:19.562106 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.562299 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2592128}
I20260811 18:57:19.621662 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 338) took 2647 ms. Trace:
I20260811 18:57:19.621949 4915 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.621760 3824 leader_election.cc:341] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.623168 4169 rpcz_store.cc:276] 0811 18:57:16.974651 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.974729 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:19.621374 (+2646645us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:19.621388 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.621606 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2646309}
I20260811 18:57:19.624775 4169 tablet_replica.cc:330] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:19.624688 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 151df87806694f46be5013592c68c01a because the tablet was not found. No further retry: Not found: Tablet not found: 151df87806694f46be5013592c68c01a
I20260811 18:57:19.622741 3891 raft_consensus.cc:2454] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.624050 3890 raft_consensus.cc:2454] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.625967 4914 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:19.626497 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 57937.7us
I20260811 18:57:19.626718 4915 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.629411 4169 raft_consensus.cc:2229] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.633222 3676 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.634042 4915 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.634831 4915 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.636205 4915 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.636822 3676 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.643863 3676 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.637261 4915 raft_consensus.cc:2781] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.645310 4915 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.645843 4915 raft_consensus.cc:3037] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.644312 4914 raft_consensus.cc:2781] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.650761 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:19.651719 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a33a6c24205421f9fc488087672c6e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.652693 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a33a6c24205421f9fc488087672c6e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.653954 4915 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.655613 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.658840 4903 raft_consensus.cc:2781] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.659192 4903 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.659565 4903 raft_consensus.cc:3037] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.664837 4903 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.669786 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:19.672410 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a98bd35bd444195fd32fe6c67a04f consensus state has no leader
I20260811 18:57:19.672995 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_86432f719d4b48c7ae9f1e0fd8095fcc" } new_table_name: "default.Table_4da51fec1a044237ab13da18366dde64" modify_external_catalogs: true
I20260811 18:57:19.666443 4914 raft_consensus.cc:686] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.674610 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e18242f762d8407db0e2f8716890feb4
I20260811 18:57:19.675117 4914 consensus_queue.cc:237] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.675882 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47718da43cfe46efab776245606d964f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.667537 3676 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:19.676620 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47718da43cfe46efab776245606d964f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.676700 4836 raft_consensus.cc:2781] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.677631 4836 raft_consensus.cc:686] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.678031 4836 consensus_queue.cc:237] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.680449 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
I20260811 18:57:19.680474 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.680702 4169 raft_consensus.cc:2258] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.681066 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cb88cf12cee8407389570138b87a10d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.682034 4169 ts_tablet_manager.cc:1905] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:19.682930 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb88cf12cee8407389570138b87a10d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.684223 3676 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.684388 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet f75dc515d7f3473b8e43f20255a70e3e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.684861 4729 raft_consensus.cc:2781] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.684942 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5409ed17441242119ff946163da07a21" } new_table_name: "default.Table_70493900fc9346389c1d4b71c611b962" modify_external_catalogs: true
I20260811 18:57:19.686474 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef1abae384a9404089b0f6600e935a5f" } new_table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" modify_external_catalogs: true
I20260811 18:57:19.669961 4167 ts_tablet_manager.cc:1918] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:19.696687 4167 log.cc:1196] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9f8c0444ec2242299482736a73551f80
I20260811 18:57:19.697539 4167 ts_tablet_manager.cc:1939] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:19.702546 3676 leader_election.cc:341] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 33979de5b48c4674a46c8da3b7151753
I20260811 18:57:19.703871 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.673521 4162 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.704820 4162 ts_tablet_manager.cc:1397] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.143s user 0.007s sys 0.000s
I20260811 18:57:19.705180 3889 raft_consensus.cc:2454] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.708039 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6446af9e35994d3c856c29a9c3ff32d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.708844 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6446af9e35994d3c856c29a9c3ff32d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.711239 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4" } modify_external_catalogs: true
I20260811 18:57:19.711526 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4" } modify_external_catalogs: true
I20260811 18:57:19.714838 4162 raft_consensus.cc:348] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.715349 4162 raft_consensus.cc:374] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.715541 4162 raft_consensus.cc:729] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.716110 4162 consensus_queue.cc:260] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.717103 4162 ts_tablet_manager.cc:1428] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:19.717713 4162 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.705598 4914 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.706714 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.718998 4914 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.712808 4729 raft_consensus.cc:686] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.721273 4729 consensus_queue.cc:237] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.724287 4903 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.744290 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.744899 3890 raft_consensus.cc:3037] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.749243 4163 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.749655 4163 ts_tablet_manager.cc:1397] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.222s user 0.004s sys 0.007s
I20260811 18:57:19.751076 3890 raft_consensus.cc:2454] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:19.752616 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_d7a06761b8a145519b7bb645e7a73657 [id=f51f38f3582f44db99007a265b665fd4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:19.752539 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 326) took 2654 ms. Trace:
I20260811 18:57:19.753230 4167 rpcz_store.cc:276] 0811 18:57:17.098121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.104245 (+ 6124us) service_pool.cc:227] Handling call
0811 18:57:19.361397 (+2257152us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:19.361412 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.521070 (+159658us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.669677 (+148607us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.752446 (+ 82769us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2245336}
I20260811 18:57:19.755801 4167 tablet_replica.cc:330] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.756244 4167 raft_consensus.cc:2229] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.756672 4167 pending_rounds.cc:62] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.756855 4167 pending_rounds.cc:68] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106077646848 op_type: NO_OP noop_request { }
I20260811 18:57:19.757200 4167 raft_consensus.cc:2866] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.757402 4167 raft_consensus.cc:2258] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.759245 4915 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.759898 3676 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.760980 3676 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.761829 4836 raft_consensus.cc:2781] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.762027 4836 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.762405 4836 raft_consensus.cc:3037] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:19.764357 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cf6c8d542ae2489ba466ffadd518db12 consensus state has no leader
I20260811 18:57:19.767334 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9f8c0444ec2242299482736a73551f80 (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:19.770557 4903 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.771020 4924 raft_consensus.cc:2781] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.773448 4914 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.751400 4163 raft_consensus.cc:348] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.775101 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.777592 4903 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.779230 4903 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.779779 4915 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.781411 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406" } modify_external_catalogs: true
I20260811 18:57:19.781888 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406" } modify_external_catalogs: true
I20260811 18:57:19.782049 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.781975 4915 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.782879 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914c8b90fadf48e88b75d13897462d84" } modify_external_catalogs: true
I20260811 18:57:19.783274 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914c8b90fadf48e88b75d13897462d84" } modify_external_catalogs: true
I20260811 18:57:19.784250 4169 ts_tablet_manager.cc:1918] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260811 18:57:19.784475 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.784873 3889 raft_consensus.cc:2454] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:19.789198 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4dc2df4c6464d71bb13fe25e0046758 consensus state has no leader
I20260811 18:57:19.790629 3891 raft_consensus.cc:2454] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.790940 3890 raft_consensus.cc:3037] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.789768 4167 ts_tablet_manager.cc:1905] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.793809 4915 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.796249 4836 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.797149 3676 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.798014 3890 raft_consensus.cc:2454] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.798804 4163 raft_consensus.cc:374] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.798949 4163 raft_consensus.cc:729] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.798956 4915 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.799216 4915 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.799333 4163 consensus_queue.cc:260] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.801625 4927 raft_consensus.cc:2781] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.801893 4927 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.803372 4927 raft_consensus.cc:3037] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.804996 4915 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.807595 4163 ts_tablet_manager.cc:1428] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.058s user 0.004s sys 0.000s
I20260811 18:57:19.811080 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.811575 3889 raft_consensus.cc:2454] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:19.811970 3824 leader_election.cc:334] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:19.812294 3824 leader_election.cc:302] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:19.812563 3676 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.812870 4878 raft_consensus.cc:2726] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:19.816298 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 515d5b5afbd74ec5ab90a503aef2f4fb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.820183 3676 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.823151 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.823531 3890 raft_consensus.cc:3037] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.800580 4924 raft_consensus.cc:686] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.827090 4903 raft_consensus.cc:2781] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.827379 4903 raft_consensus.cc:686] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.828814 4915 raft_consensus.cc:2781] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.829780 3551 catalog_manager.cc:5495] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:19.830900 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
W20260811 18:57:19.834566 3824 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.081s (SENT)
I20260811 18:57:19.834789 3676 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.835711 4924 consensus_queue.cc:237] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.837337 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a (DEFAULT_TABLE table=default.Table_f6a3985ef18b44138437cd5cd54f34ed [id=2bc826b3f7464185a308efa3382cd966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.838500 4915 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.839640 4915 raft_consensus.cc:3037] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.839963 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f3060822021431584a494e6941be756 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.840438 4836 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.844079 3860 tablet_service.cc:1467] Processing CreateTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 (DEFAULT_TABLE table=default.Table_74385eb07bdb4cb39d7783a47ae5781c [id=def664eaf5774792acf9f58193e6d6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.844758 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ebc32eb5d5443eaaeba245b566f98a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.847132 4162 tablet_bootstrap.cc:654] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.851536 3721 tablet_service.cc:1467] Processing CreateTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 (DEFAULT_TABLE table=default.Table_74385eb07bdb4cb39d7783a47ae5781c [id=def664eaf5774792acf9f58193e6d6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.852342 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2816cc2dfe45f0815ce50a5a929324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.852946 4914 raft_consensus.cc:2781] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.853303 4169 log.cc:1196] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:19.854267 4169 ts_tablet_manager.cc:1939] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:19.840687 4903 consensus_queue.cc:237] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.860870 4914 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.861123 4914 raft_consensus.cc:3037] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:19.863966 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 11137ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:19.869401 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a (DEFAULT_TABLE table=default.Table_f6a3985ef18b44138437cd5cd54f34ed [id=2bc826b3f7464185a308efa3382cd966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.870455 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ebc32eb5d5443eaaeba245b566f98a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.859836 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.875159 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 85140a3800f34a23b716703fd4e73d17 (DEFAULT_TABLE table=default.Table_c1ceb39258a049f08a1f48c40df65d8b [id=ba51142694a0491f857bcb698e29d93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.875866 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85140a3800f34a23b716703fd4e73d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.878183 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 14e34732bda44670ae9da8facc405221 (DEFAULT_TABLE table=default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 [id=c2c29eaa8e054b14a1d3ae3d75be99d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.878902 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e34732bda44670ae9da8facc405221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.887791 4729 consensus_queue.cc:1035] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260811 18:57:19.895393 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bc6cce2c19746219b8b0fe54b13dcd6
I20260811 18:57:19.896327 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7a0e0d139c924f1dafe42da0bedada72
I20260811 18:57:19.897238 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8efe53501a143ed90881e375a449e81" } modify_external_catalogs: true
I20260811 18:57:19.897526 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8efe53501a143ed90881e375a449e81" } modify_external_catalogs: true
I20260811 18:57:19.899271 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 (DEFAULT_TABLE table=default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 [id=da25cefa0734439090e25c2e1f491657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.899456 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
I20260811 18:57:19.901289 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 14e34732bda44670ae9da8facc405221 (DEFAULT_TABLE table=default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 [id=c2c29eaa8e054b14a1d3ae3d75be99d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.894255 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_90566ec48eaf42c8be4c06681c3c0268"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.902714 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90566ec48eaf42c8be4c06681c3c0268 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.903568 3889 raft_consensus.cc:2454] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.905611 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.906468 3891 raft_consensus.cc:1261] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:19.834100 4927 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.906087 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 (DEFAULT_TABLE table=default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 [id=da25cefa0734439090e25c2e1f491657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.916281 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f3670f8908d54ebb82f698cdd207e48f (DEFAULT_TABLE table=default.Table_e12ffa777d684216bfc19d52f2dad4be [id=7071461f50fc49958d87c5575b94335e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.920039 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" } modify_external_catalogs: true
I20260811 18:57:19.920234 3870 tablet_service.cc:1467] Processing CreateTablet for tablet f3670f8908d54ebb82f698cdd207e48f (DEFAULT_TABLE table=default.Table_e12ffa777d684216bfc19d52f2dad4be [id=7071461f50fc49958d87c5575b94335e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.920383 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" } modify_external_catalogs: true
I20260811 18:57:19.920871 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc (DEFAULT_TABLE table=default.Table_278e0a9db8364b7cb00a0920de4b0561 [id=ff9e8d54b42c44d294858300dd87d043]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.921518 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 91818d0cbe144a67a458a6648968bf5b (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.919355 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.923074 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.926767 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.926700 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 85140a3800f34a23b716703fd4e73d17 (DEFAULT_TABLE table=default.Table_c1ceb39258a049f08a1f48c40df65d8b [id=ba51142694a0491f857bcb698e29d93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.927189 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.927326 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3670f8908d54ebb82f698cdd207e48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.927697 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.927934 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.928037 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 (DEFAULT_TABLE table=default.Table_c8efe53501a143ed90881e375a449e81 [id=978fc92b4923484d9a4fdc96f4d4dba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.928385 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.928676 3889 raft_consensus.cc:3037] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.934561 3889 raft_consensus.cc:2454] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.935155 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc (DEFAULT_TABLE table=default.Table_278e0a9db8364b7cb00a0920de4b0561 [id=ff9e8d54b42c44d294858300dd87d043]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.935708 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289e607f10e14a2983bc6ecc93ce9bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.936116 4163 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.937564 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 91818d0cbe144a67a458a6648968bf5b (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.938200 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91818d0cbe144a67a458a6648968bf5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.944509 3890 raft_consensus.cc:2454] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.949488 4927 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.952984 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 (DEFAULT_TABLE table=default.Table_c8efe53501a143ed90881e375a449e81 [id=978fc92b4923484d9a4fdc96f4d4dba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.953276 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 (DEFAULT_TABLE table=default.Table_ab9634fb958f4d44840fa6f1709ce326 [id=5350668f89b84b7899da4209b6bc2822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.953999 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7645a242bc4e4845ad49bfbe20b252f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.956238 4163 tablet_bootstrap.cc:654] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.956189 3676 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.957029 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 1811e5f62a714f8388460dfcb1717e81 (DEFAULT_TABLE table=default.Table_655b56a4862144f1afd52a620c3715d3 [id=a5bce8098d7f4150944fc21295d519c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.957329 3676 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.958240 4729 raft_consensus.cc:2781] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.958457 4729 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.959983 4729 raft_consensus.cc:3037] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.960614 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cee8eb42df9b40cd85eb35169133787a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.961551 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cee8eb42df9b40cd85eb35169133787a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.962126 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f20eabb4925482bbf71fe270dde24af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.962807 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f20eabb4925482bbf71fe270dde24af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.963413 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a85e1485efcd445a972fe25c37f2bb9e" } new_table_name: "default.Table_e5be012fe5c34e9bb842907f4b154b8e" modify_external_catalogs: true
I20260811 18:57:19.964161 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a594cb48b586458783a36cc76df1332c" } new_table_name: "default.Table_1b2efb13eaf1468c92c8030667191b84" modify_external_catalogs: true
I20260811 18:57:19.964587 4836 consensus_queue.cc:1035] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260811 18:57:19.964828 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b78f30561e284a9596437b5fc75df24f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.965619 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b78f30561e284a9596437b5fc75df24f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.968729 4924 raft_consensus.cc:2781] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.969450 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 1811e5f62a714f8388460dfcb1717e81 (DEFAULT_TABLE table=default.Table_655b56a4862144f1afd52a620c3715d3 [id=a5bce8098d7f4150944fc21295d519c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.976454 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1811e5f62a714f8388460dfcb1717e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.972774 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 (DEFAULT_TABLE table=default.Table_ab9634fb958f4d44840fa6f1709ce326 [id=5350668f89b84b7899da4209b6bc2822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.981921 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3468b39bd73c4272a84dad37fcfd6f27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:19.986560 3678 leader_election.cc:334] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.151s (SENT)
I20260811 18:57:19.986936 3678 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:19.988406 3678 raft_consensus.cc:2670] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:19.991863 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2816cc2dfe45f0815ce50a5a929324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.997192 3676 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.998935 4927 raft_consensus.cc:2781] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.999486 4927 raft_consensus.cc:686] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.000350 4927 consensus_queue.cc:237] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.012413 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.015030 4903 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.015476 4903 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.024799 4924 raft_consensus.cc:686] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.025878 4914 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.026407 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e34732bda44670ae9da8facc405221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.028587 4924 consensus_queue.cc:237] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.030692 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289e607f10e14a2983bc6ecc93ce9bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.034591 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7645a242bc4e4845ad49bfbe20b252f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.038743 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3670f8908d54ebb82f698cdd207e48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.042791 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91818d0cbe144a67a458a6648968bf5b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.042865 3678 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.444s (SENT)
I20260811 18:57:20.057799 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1811e5f62a714f8388460dfcb1717e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.060886 4162 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:20.061311 4162 ts_tablet_manager.cc:1397] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.344s user 0.004s sys 0.004s
I20260811 18:57:20.062606 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85140a3800f34a23b716703fd4e73d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.065333 4915 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.066781 4915 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.070956 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3468b39bd73c4272a84dad37fcfd6f27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.074944 3824 leader_election.cc:334] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.576s (SENT)
I20260811 18:57:20.075242 3824 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:20.075600 3824 raft_consensus.cc:2670] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.075521 4162 raft_consensus.cc:348] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.076241 4162 raft_consensus.cc:374] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.076468 4162 raft_consensus.cc:729] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.076781 4162 consensus_queue.cc:260] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.076949 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48782.1us
I20260811 18:57:20.078960 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.079298 3888 raft_consensus.cc:3037] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.083863 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 339) took 3005 ms. Trace:
I20260811 18:57:20.084180 4169 rpcz_store.cc:276] 0811 18:57:17.017120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.018895 (+ 1775us) service_pool.cc:227] Handling call
0811 18:57:19.624548 (+2605653us) ts_tablet_manager.cc:1145] Deleting tablet 7b227bc7305f4b2dae28f04e7001553b
0811 18:57:19.624576 (+ 28us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.731026 (+106450us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.784094 (+ 53068us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.022319 (+238225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2605271}
I20260811 18:57:20.085243 4914 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.085769 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]) successfully deleted
I20260811 18:57:20.086565 4729 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.086958 4169 tablet_replica.cc:330] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:20.089527 4903 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.091303 4729 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.099225 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.099597 3891 raft_consensus.cc:3037] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.108527 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:20.108886 4878 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.113304 3893 raft_consensus.cc:2454] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:20.113191 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.114501 3888 raft_consensus.cc:2454] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.136670 3892 raft_consensus.cc:3037] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.142077 4878 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.144562 4167 ts_tablet_manager.cc:1918] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.148981 4167 log.cc:1196] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/33979de5b48c4674a46c8da3b7151753
I20260811 18:57:20.148182 3676 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.151269 3676 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.151980 4924 raft_consensus.cc:2781] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.153724 4878 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:20.154420 4924 raft_consensus.cc:686] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.154886 4924 consensus_queue.cc:237] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.151418 4927 raft_consensus.cc:2781] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:20.155859 4927 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:20.156173 4927 raft_consensus.cc:3037] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.157547 3892 raft_consensus.cc:2454] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.146071 4162 ts_tablet_manager.cc:1428] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.084s user 0.005s sys 0.001s
I20260811 18:57:20.160383 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.159560 4162 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.160624 3676 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.164703 4729 raft_consensus.cc:2781] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.165271 4729 raft_consensus.cc:686] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.165951 4729 consensus_queue.cc:237] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.168418 3824 leader_election.cc:341] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.174144 4167 ts_tablet_manager.cc:1939] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.175421 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 327) took 3070 ms. Trace:
W20260811 18:57:20.181833 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9166ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:20.192169 4167 rpcz_store.cc:276] 0811 18:57:17.104642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.105872 (+ 1230us) service_pool.cc:227] Handling call
0811 18:57:19.755633 (+2649761us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:19.755647 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.890165 (+134518us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.144381 (+254216us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.175318 (+ 30937us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2639274}
I20260811 18:57:20.201788 4167 tablet_replica.cc:330] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.202432 4167 raft_consensus.cc:2229] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.202723 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 33979de5b48c4674a46c8da3b7151753 (table default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]) successfully deleted
I20260811 18:57:20.203116 4167 raft_consensus.cc:2258] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.204119 4167 ts_tablet_manager.cc:1905] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.205477 4162 tablet_bootstrap.cc:654] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:20.209110 4163 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:20.218243 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 85609.2us
I20260811 18:57:20.220569 4163 ts_tablet_manager.cc:1397] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.350s user 0.001s sys 0.007s
I20260811 18:57:20.226723 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6f3060822021431584a494e6941be756
I20260811 18:57:20.227550 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 6f3060822021431584a494e6941be756 deleted and replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76
I20260811 18:57:20.229676 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7645a242bc4e4845ad49bfbe20b252f2
I20260811 18:57:20.235864 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.237460 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_680160aaac0a4c8da66634dc41d6abb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.239955 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.240965 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.243378 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_680160aaac0a4c8da66634dc41d6abb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.245456 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.246836 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:20.248018 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4d5db1f0452e40eea01177ba93223341"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.248889 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d5db1f0452e40eea01177ba93223341 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.256352 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.257973 4878 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.260167 4878 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.261051 3891 raft_consensus.cc:2454] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.261754 3676 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.262279 4941 raft_consensus.cc:2781] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.263180 4941 raft_consensus.cc:686] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.263630 4941 consensus_queue.cc:237] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.265170 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.266479 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.271255 4162 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:20.271845 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.272560 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:20.273818 3824 leader_election.cc:341] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.191524 4927 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.282073 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 62440.2us
W20260811 18:57:20.282650 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:20.285823 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.286440 4162 ts_tablet_manager.cc:1397] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.006s
I20260811 18:57:20.286728 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.300436 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.309434 4878 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:20.310525 4729 consensus_queue.cc:1035] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:20.311959 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.312477 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.313081 4167 ts_tablet_manager.cc:1918] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.313244 4167 log.cc:1196] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:20.317286 4927 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.322067 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.322417 3892 raft_consensus.cc:3037] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.329556 4169 raft_consensus.cc:2229] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.332145 4169 raft_consensus.cc:2258] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.335960 4162 raft_consensus.cc:348] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.337700 4169 ts_tablet_manager.cc:1905] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.338121 4927 consensus_queue.cc:1035] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:20.342468 4162 raft_consensus.cc:374] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.342700 4162 raft_consensus.cc:729] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.343151 4162 consensus_queue.cc:260] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.344092 4941 raft_consensus.cc:482] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.344455 4941 raft_consensus.cc:504] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.344703 4162 ts_tablet_manager.cc:1428] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.058s user 0.000s sys 0.004s
I20260811 18:57:20.345255 4162 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.345513 4941 leader_election.cc:288] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:20.351154 3676 leader_election.cc:341] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: f8eb041f3d704762868a048faec0e0fc
W20260811 18:57:20.357061 3678 leader_election.cc:334] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.269s (SENT)
I20260811 18:57:20.357326 3678 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:20.357651 3678 raft_consensus.cc:2670] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.367069 4163 raft_consensus.cc:348] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.368963 4163 raft_consensus.cc:374] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.369997 4163 raft_consensus.cc:729] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.371222 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ebb36a4f14a0491692aec7b72295b80c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.371953 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebb36a4f14a0491692aec7b72295b80c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.373869 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7abca85e5a6d4b55a38c0f9e50f02f76 has no consensus state
W20260811 18:57:20.373986 3678 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.075s (SENT)
I20260811 18:57:20.374564 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b2c3c3b4a2948bbba5b4b556faf1015" } new_table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" modify_external_catalogs: true
W20260811 18:57:20.376236 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4c7f500a5444bae9a015a5e903c0622 consensus state has no leader
I20260811 18:57:20.373376 3892 raft_consensus.cc:2454] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.376936 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d00075c1e094c1cb6a03b6760496fb5" } new_table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" modify_external_catalogs: true
I20260811 18:57:20.379182 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47718da43cfe46efab776245606d964f" } new_table_name: "default.Table_709cd55123504b35877d469f96354d28" modify_external_catalogs: true
I20260811 18:57:20.379660 3676 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.381254 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2e58ff1b0c44ee8b71812ae6399b4da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.381336 4927 raft_consensus.cc:2781] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:20.381929 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2e58ff1b0c44ee8b71812ae6399b4da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.381976 4927 raft_consensus.cc:686] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.382639 4927 consensus_queue.cc:237] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.383795 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4da51fec1a044237ab13da18366dde64" } modify_external_catalogs: true
I20260811 18:57:20.384102 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4da51fec1a044237ab13da18366dde64" } modify_external_catalogs: true
I20260811 18:57:20.386700 4169 ts_tablet_manager.cc:1918] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.386868 4169 log.cc:1196] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:20.387367 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d" } modify_external_catalogs: true
I20260811 18:57:20.387491 4169 ts_tablet_manager.cc:1939] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:20.387660 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d" } modify_external_catalogs: true
I20260811 18:57:20.387903 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 340) took 3149 ms. Trace:
I20260811 18:57:20.388062 4169 rpcz_store.cc:276] 0811 18:57:17.238141 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.238401 (+ 260us) service_pool.cc:227] Handling call
0811 18:57:20.084840 (+2846439us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:20.084860 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.329978 (+245118us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x615000128f24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:20.382559 (+ 52581us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.386533 (+ 3974us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.387850 (+ 1317us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":556880128,"tablet-delete.queue_time_us":2841653}
I20260811 18:57:20.391129 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" } modify_external_catalogs: true
I20260811 18:57:20.391381 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" } modify_external_catalogs: true
I20260811 18:57:20.425534 4167 ts_tablet_manager.cc:1939] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.397949 4163 consensus_queue.cc:260] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.427018 4163 ts_tablet_manager.cc:1428] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.206s user 0.004s sys 0.000s
I20260811 18:57:20.427052 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 328) took 3320 ms. Trace:
I20260811 18:57:20.428323 4163 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:20.430466 4162 tablet_bootstrap.cc:654] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:20.376696 4950 log.cc:924] Time spent T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260811 18:57:20.423003 3824 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.427s (SENT)
I20260811 18:57:20.434881 4163 tablet_bootstrap.cc:654] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:20.437631 4169 tablet_replica.cc:330] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:20.437652 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f1a55a42d1e64f04ad7f433614a3a1aa (table default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]) successfully deleted
I20260811 18:57:20.443317 4958 tablet.cc:1722] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:20.448776 3824 leader_election.cc:334] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.702s (SENT)
W20260811 18:57:20.475102 3678 leader_election.cc:334] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.313s (SENT)
I20260811 18:57:20.475492 3678 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:20.475955 3678 raft_consensus.cc:2670] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:20.477116 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 54115.2us
W20260811 18:57:20.477942 3678 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.620s (SENT)
I20260811 18:57:20.478176 3678 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:20.478909 4957 tablet.cc:1722] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:20.485512 3824 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:20.485941 3824 raft_consensus.cc:2670] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.492033 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet d2d7d986900244ddbc05dbe73a708dcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.492684 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet a732c700ff79432aaf1f0d1c1e90aabe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.492704 4167 rpcz_store.cc:276] 0811 18:57:17.106095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.107484 (+ 1389us) service_pool.cc:227] Handling call
0811 18:57:20.198843 (+3091359us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:20.198863 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.239321 (+ 40458us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.307547 (+ 68226us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.426933 (+119386us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3072199}
W20260811 18:57:20.493273 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45fde25bcb40493aafb083e161aa6932 consensus state has no leader
I20260811 18:57:20.493646 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 331396c9962f48ed9ea6dc66f1660902 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.494376 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 3744b3d608754de68c4c57b072d1dc15 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.495173 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3652d29236a4b12b7abd44659c70311"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.496178 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3652d29236a4b12b7abd44659c70311 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.497915 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28af2d2c61254b0ca399dddb05e9ee70 has no consensus state
I20260811 18:57:20.498584 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_97097723c5264ab781135d328c9be026" } modify_external_catalogs: true
I20260811 18:57:20.499006 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_97097723c5264ab781135d328c9be026" } modify_external_catalogs: true
I20260811 18:57:20.500366 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de899ce498b24cd5b270adf6fbb81b92" } modify_external_catalogs: true
I20260811 18:57:20.500758 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de899ce498b24cd5b270adf6fbb81b92" } modify_external_catalogs: true
W20260811 18:57:20.501367 3678 raft_consensus.cc:2670] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.501888 4169 raft_consensus.cc:2229] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.502213 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6ce8db230c7e41b6a4792273a0f07e6e (table default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]) successfully deleted
W20260811 18:57:20.502213 3676 consensus_peers.cc:487] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:20.502462 4169 pending_rounds.cc:62] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:20.502694 4169 pending_rounds.cc:68] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400122051297280 op_type: NO_OP noop_request { }
I20260811 18:57:20.502913 4169 raft_consensus.cc:2866] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:20.503270 4169 raft_consensus.cc:2258] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.504714 4167 tablet_replica.cc:330] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.505710 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6381a951a966473c80670b0dd1aff062
I20260811 18:57:20.505987 4962 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:20.507336 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 6381a951a966473c80670b0dd1aff062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.513226 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6381a951a966473c80670b0dd1aff062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.513777 4167 raft_consensus.cc:2229] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.515974 4167 raft_consensus.cc:2258] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:20.523599 3678 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.417s (SENT)
I20260811 18:57:20.524971 4169 ts_tablet_manager.cc:1905] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.525054 4167 ts_tablet_manager.cc:1905] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.528277 4927 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.536227 4163 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:20.536747 4163 ts_tablet_manager.cc:1397] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.009s
I20260811 18:57:20.529476 4927 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.544386 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:20.545347 3888 raft_consensus.cc:2454] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:20.547474 4163 raft_consensus.cc:348] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.548135 3676 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.548710 4163 raft_consensus.cc:374] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.548606 4957 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:20.548834 4914 raft_consensus.cc:2781] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:20.549064 4914 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:20.549139 4163 raft_consensus.cc:729] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.549336 4914 raft_consensus.cc:3037] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.552515 4163 consensus_queue.cc:260] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.554239 4914 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.556377 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.563063 3888 raft_consensus.cc:3037] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.571105 3888 raft_consensus.cc:2454] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.578853 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c7fb552f75e54888873b50a45cb0a475 (DEFAULT_TABLE table=default.Table_6446af9e35994d3c856c29a9c3ff32d2 [id=b2dab2b70454407ab25aefe2843ada47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.579417 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7fb552f75e54888873b50a45cb0a475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.584072 3861 tablet_service.cc:1467] Processing CreateTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 (DEFAULT_TABLE table=default.Table_cb88cf12cee8407389570138b87a10d2 [id=fe56a8d6cdca49f1ba5a900cf37399b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.584092 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0b59cafb154416952aecaa418e861f (DEFAULT_TABLE table=default.Table_5a33a6c24205421f9fc488087672c6e0 [id=9f664fe34e8348f0989ba7b3583e0e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.585026 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0b59cafb154416952aecaa418e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.585214 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 (DEFAULT_TABLE table=default.Table_709cd55123504b35877d469f96354d28 [id=26e34b672a354ee0a67f8eae9b099af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.585263 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]). Expected version 5 got 4
I20260811 18:57:20.585834 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28af2d2c61254b0ca399dddb05e9ee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.588680 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeec0c5672b4b3190d7d7cf341cb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.589061 4914 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.590418 4927 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.593840 3676 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.594205 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.597070 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 (DEFAULT_TABLE table=default.Table_709cd55123504b35877d469f96354d28 [id=26e34b672a354ee0a67f8eae9b099af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.597698 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28af2d2c61254b0ca399dddb05e9ee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.598659 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2345d5c2154329b601762c8085bc07 (DEFAULT_TABLE table=default.Table_d1eb56bed2494b7f85229d2c92bcb60c [id=5ebb2f681b264b2caa3fb73f68af73bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.599192 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2345d5c2154329b601762c8085bc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.602411 3718 tablet_service.cc:1467] Processing CreateTablet for tablet c7fb552f75e54888873b50a45cb0a475 (DEFAULT_TABLE table=default.Table_6446af9e35994d3c856c29a9c3ff32d2 [id=b2dab2b70454407ab25aefe2843ada47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.602833 4952 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.603168 4952 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.606024 4952 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:20.603014 4914 raft_consensus.cc:2781] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.607966 4914 raft_consensus.cc:686] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.608403 4914 consensus_queue.cc:237] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.603319 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0b59cafb154416952aecaa418e861f (DEFAULT_TABLE table=default.Table_5a33a6c24205421f9fc488087672c6e0 [id=9f664fe34e8348f0989ba7b3583e0e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.605131 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7fb552f75e54888873b50a45cb0a475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.619659 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7bc942d59fd34beda3b8fb9e03e84cb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.620662 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.605871 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2345d5c2154329b601762c8085bc07 (DEFAULT_TABLE table=default.Table_d1eb56bed2494b7f85229d2c92bcb60c [id=5ebb2f681b264b2caa3fb73f68af73bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.623093 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2345d5c2154329b601762c8085bc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.624413 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_886df95cce4342f998d27c8661df890e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.625250 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_886df95cce4342f998d27c8661df890e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.606575 3710 tablet_service.cc:1467] Processing CreateTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 (DEFAULT_TABLE table=default.Table_cb88cf12cee8407389570138b87a10d2 [id=fe56a8d6cdca49f1ba5a900cf37399b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.607510 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.627035 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeec0c5672b4b3190d7d7cf341cb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.627049 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_459873f5dbe2412c80b73ca35dce90ed" } new_table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" modify_external_catalogs: true
I20260811 18:57:20.628959 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea042ebbfb444a2086b87857b6775140" } new_table_name: "default.Table_315c3cbf08354dae86745d91bbc2cd11" modify_external_catalogs: true
I20260811 18:57:20.630039 4836 consensus_queue.cc:1035] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:20.630811 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0b59cafb154416952aecaa418e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.641204 4162 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:20.641506 3824 leader_election.cc:341] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.642688 3871 tablet_service.cc:1467] Processing CreateTablet for tablet a5bbd3274980470b9fe71457b49bdd62 (DEFAULT_TABLE table=default.Table_ea042ebbfb444a2086b87857b6775140 [id=bfc7c5f6b2f041a1a215254c038a1fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.643297 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bbd3274980470b9fe71457b49bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.645392 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e407d34d3c984ecb8c8ffde304f23ce6 (DEFAULT_TABLE table=default.Table_771bec99eb584420a45d22e9b038229a [id=a7de43e4f94740fbb4fd1ba5aa5009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.645509 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe733ace0064e2d886cecbdd6decc76 (DEFAULT_TABLE table=default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 [id=ca9587388aae4f4e9eedc7ff9a137476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.646109 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e407d34d3c984ecb8c8ffde304f23ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.646517 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe733ace0064e2d886cecbdd6decc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.646999 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.647696 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d24aff9fd34f3eb05fdf942cc7efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.680415 4957 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260811 18:57:20.682152 4162 ts_tablet_manager.cc:1397] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.300s user 0.008s sys 0.005s
I20260811 18:57:20.683203 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 22e1a33348e34e35bab786f8966c408b (DEFAULT_TABLE table=default.Table_01f622bd12ce4b2bb39355d87ed79fb2 [id=bf02bf181b36444382ab04e4bc399e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.688876 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce6143309948d49e229dc9890d904f (DEFAULT_TABLE table=default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.691831 4163 ts_tablet_manager.cc:1428] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.131s user 0.002s sys 0.002s
I20260811 18:57:20.694051 4958 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260811 18:57:20.690210 3856 tablet_service.cc:1467] Processing CreateTablet for tablet aeb83eaf12444d6fbd52af568389cb3c (DEFAULT_TABLE table=default.Table_7455a95b7b2a4c5db623ad62d8332150 [id=93a003db2a41483b8675373b7ab6772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.696552 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:20.697743 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fc78208e540445719638da3757163df5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.698740 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 91818d0cbe144a67a458a6648968bf5b
I20260811 18:57:20.699883 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_30ac5178633e44aeafd707e48d7c4cba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.700769 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30ac5178633e44aeafd707e48d7c4cba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.703029 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:20.704568 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab9ea53bcc98448384f4fd3d30d2528f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.705556 3824 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.942s (SENT)
W20260811 18:57:20.705729 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9ea53bcc98448384f4fd3d30d2528f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.715516 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc78208e540445719638da3757163df5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.718088 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:20.718953 3547 catalog_manager.cc:6565] ReplaceTablet: tablet a732c700ff79432aaf1f0d1c1e90aabe deleted and replaced by tablet fb878de2c0d94408aa6e98447bd7e958
I20260811 18:57:20.719734 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f713f72dfeb48658564074b098a97d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.720551 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f713f72dfeb48658564074b098a97d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.723012 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:20.723583 3554 catalog_manager.cc:5495] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.724140 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:20.724911 3554 catalog_manager.cc:5495] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.725718 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:20.726914 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 059b325df1db4fde9d4b26b559121316
I20260811 18:57:20.727562 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e1a33348e34e35bab786f8966c408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.728018 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9dd303f12b584056a04e385cc80faffe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.728938 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9dd303f12b584056a04e385cc80faffe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.730360 4163 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:20.731374 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 91818d0cbe144a67a458a6648968bf5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.733875 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce6143309948d49e229dc9890d904f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.734876 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.735668 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.736668 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe with delete_type TABLET_DATA_DELETED (replaced by tablet fb878de2c0d94408aa6e98447bd7e958) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.737164 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.740718 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb83eaf12444d6fbd52af568389cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.749873 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d8aea8dc31914052832bdc83d638461f consensus state has no leader
I20260811 18:57:20.750846 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_badc11f1b0554e629ddb3f8a20e151e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.751703 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_badc11f1b0554e629ddb3f8a20e151e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.753412 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5bbd3274980470b9fe71457b49bdd62 consensus state has no leader
I20260811 18:57:20.753978 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_89b7704fd6264ef8bd2289a13f9a488d" } new_table_name: "default.Table_8e4cf372a70649839b2083d6740f8c99" modify_external_catalogs: true
I20260811 18:57:20.757264 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafb4240a8584ec184a99a633b46eeb1" } new_table_name: "default.Table_235289a804494101acba3a4f14192af3" modify_external_catalogs: true
W20260811 18:57:20.759374 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d73a98bd35bd444195fd32fe6c67a04f consensus state has no leader
I20260811 18:57:20.783845 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe733ace0064e2d886cecbdd6decc76 (DEFAULT_TABLE table=default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 [id=ca9587388aae4f4e9eedc7ff9a137476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.784653 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe733ace0064e2d886cecbdd6decc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.782013 3718 tablet_service.cc:1467] Processing CreateTablet for tablet a5bbd3274980470b9fe71457b49bdd62 (DEFAULT_TABLE table=default.Table_ea042ebbfb444a2086b87857b6775140 [id=bfc7c5f6b2f041a1a215254c038a1fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.790803 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e407d34d3c984ecb8c8ffde304f23ce6 (DEFAULT_TABLE table=default.Table_771bec99eb584420a45d22e9b038229a [id=a7de43e4f94740fbb4fd1ba5aa5009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.794720 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.794898 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 5a75a480f38047d880b0c1fa51403786 (DEFAULT_TABLE table=default.Table_914d7e775ebf4b26beed57cb21140166 [id=b39c0641dd714d8a89839cadc2c567eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.795272 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d24aff9fd34f3eb05fdf942cc7efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.796753 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a75a480f38047d880b0c1fa51403786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.799855 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bbd3274980470b9fe71457b49bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.802028 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5fef93c0a9b546199a3217a8b8bd67dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.803077 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5fef93c0a9b546199a3217a8b8bd67dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.804709 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 22e1a33348e34e35bab786f8966c408b (DEFAULT_TABLE table=default.Table_01f622bd12ce4b2bb39355d87ed79fb2 [id=bf02bf181b36444382ab04e4bc399e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.812987 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e407d34d3c984ecb8c8ffde304f23ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.819764 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce6143309948d49e229dc9890d904f (DEFAULT_TABLE table=default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.820708 3712 tablet_service.cc:1467] Processing CreateTablet for tablet aeb83eaf12444d6fbd52af568389cb3c (DEFAULT_TABLE table=default.Table_7455a95b7b2a4c5db623ad62d8332150 [id=93a003db2a41483b8675373b7ab6772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.820489 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 (DEFAULT_TABLE table=default.Table_a795d5395cc64155b84057a53c701064 [id=0e2d7e83bb53429db83d32be7ea5b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.822530 4162 raft_consensus.cc:348] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.827093 4162 raft_consensus.cc:374] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.824658 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.826514 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e1a33348e34e35bab786f8966c408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.829411 4162 raft_consensus.cc:729] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.829849 4162 consensus_queue.cc:260] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.832702 4162 ts_tablet_manager.cc:1428] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.148s user 0.002s sys 0.002s
I20260811 18:57:20.835361 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 91818d0cbe144a67a458a6648968bf5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.836452 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d988ef7e9dd43158d107998ea4d82b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.838672 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb83eaf12444d6fbd52af568389cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.851121 4162 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.852525 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce6143309948d49e229dc9890d904f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.854143 3824 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.087s (SENT)
I20260811 18:57:20.854298 4167 ts_tablet_manager.cc:1918] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:20.854399 3824 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:20.854665 4167 log.cc:1196] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/dde27d8cc3eb4d09879e717fc90305fd
W20260811 18:57:20.854770 3824 raft_consensus.cc:2670] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.855448 4167 ts_tablet_manager.cc:1939] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.856211 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 345) took 3336 ms. Trace:
I20260811 18:57:20.856333 4167 rpcz_store.cc:276] 0811 18:57:17.519879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.519940 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:20.493260 (+2973320us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:20.493274 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.642394 (+149120us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.682506 (+ 40112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.856144 (+173638us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2957939}
I20260811 18:57:20.857016 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 344) took 3337 ms. Trace:
I20260811 18:57:20.857203 4167 rpcz_store.cc:276] 0811 18:57:17.519741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.520199 (+ 458us) service_pool.cc:227] Handling call
0811 18:57:20.856690 (+3336491us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:20.856703 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.856886 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3300834}
I20260811 18:57:20.857820 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 346) took 3321 ms. Trace:
I20260811 18:57:20.858012 4167 rpcz_store.cc:276] 0811 18:57:17.535946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.536125 (+ 179us) service_pool.cc:227] Handling call
0811 18:57:20.857546 (+3321421us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:20.857557 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.857686 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3300614}
I20260811 18:57:20.858496 4167 tablet_replica.cc:330] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.858829 4167 raft_consensus.cc:2229] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:20.859315 4167 raft_consensus.cc:2258] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.860267 4167 ts_tablet_manager.cc:1905] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:20.868438 3678 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.126s (SENT)
I20260811 18:57:20.869624 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.869881 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.873030 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet dde27d8cc3eb4d09879e717fc90305fd (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
W20260811 18:57:20.873442 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c already in progress: creating tablet
W20260811 18:57:20.875202 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b4de110bd3348f0b0dae21c9f2b6a19 already in progress: creating tablet
I20260811 18:57:20.877668 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 (DEFAULT_TABLE table=default.Table_a795d5395cc64155b84057a53c701064 [id=0e2d7e83bb53429db83d32be7ea5b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.896981 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_278e0a9db8364b7cb00a0920de4b0561" } modify_external_catalogs: true
I20260811 18:57:20.898115 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d988ef7e9dd43158d107998ea4d82b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.900918 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 5a75a480f38047d880b0c1fa51403786 (DEFAULT_TABLE table=default.Table_914d7e775ebf4b26beed57cb21140166 [id=b39c0641dd714d8a89839cadc2c567eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.902355 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a75a480f38047d880b0c1fa51403786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.909883 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_278e0a9db8364b7cb00a0920de4b0561" } modify_external_catalogs: true
I20260811 18:57:20.911743 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe with delete_type TABLET_DATA_DELETED (replaced by tablet fb878de2c0d94408aa6e98447bd7e958) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.898779 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9fe9862d3266414295d433fd5b136638" } new_table_name: "default.Table_7f7c3f328d7e49db85f19e13aa462cd6" modify_external_catalogs: true
I20260811 18:57:20.918699 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" } modify_external_catalogs: true
I20260811 18:57:20.919121 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" } modify_external_catalogs: true
I20260811 18:57:20.928136 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" } modify_external_catalogs: true
I20260811 18:57:20.928696 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" } modify_external_catalogs: true
I20260811 18:57:20.933800 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9634fb958f4d44840fa6f1709ce326" } modify_external_catalogs: true
I20260811 18:57:20.934235 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9634fb958f4d44840fa6f1709ce326" } modify_external_catalogs: true
I20260811 18:57:20.900236 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_99de6d48691743d798f56442d85d5be8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.937593 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:20.938279 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99de6d48691743d798f56442d85d5be8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.905267 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 62c365f79d4a4201ac323f16bddf3e62 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:20.906064 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb878de2c0d94408aa6e98447bd7e958 has no consensus state
W20260811 18:57:20.906491 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91c8101778b54239b834a4f11ef4a08a consensus state has no leader
I20260811 18:57:20.949719 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb88cf12cee8407389570138b87a10d2" } modify_external_catalogs: true
I20260811 18:57:20.950377 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb88cf12cee8407389570138b87a10d2" } modify_external_catalogs: true
W20260811 18:57:20.960992 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb51e419dd24459caf3ba3adfbbd4517 consensus state has no leader
I20260811 18:57:20.964299 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3468b39bd73c4272a84dad37fcfd6f27
I20260811 18:57:20.968024 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf5749d4eebd42319df1b52eeba4c062
I20260811 18:57:20.968257 4914 consensus_queue.cc:1035] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260811 18:57:20.969586 4924 consensus_queue.cc:1035] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260811 18:57:20.970279 3678 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.036s (SENT)
I20260811 18:57:20.975189 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8214722ffbdc49bf98d5aa539aebed52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.976153 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8214722ffbdc49bf98d5aa539aebed52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.982679 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6d87a15f366a4e118c84e294c3b9926e" } new_table_name: "default.Table_10f91c342797480e9545a00a8d6be4a9" modify_external_catalogs: true
I20260811 18:57:20.988174 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:20.988646 3546 catalog_manager.cc:5495] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.989506 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:20.991899 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be30e8194fd44874a1fdd996128eb4d4" } new_table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" modify_external_catalogs: true
I20260811 18:57:20.994082 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65d5ebf213ce401984be01bac1813640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.995407 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65d5ebf213ce401984be01bac1813640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.001557 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.002928 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5749d4eebd42319df1b52eeba4c062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.004294 4914 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.006844 4914 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.008036 4914 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:21.026654 3678 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.026s (SENT)
I20260811 18:57:21.026961 3678 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:21.030231 3678 raft_consensus.cc:2670] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:21.038472 4162 tablet_bootstrap.cc:654] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.034170 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5749d4eebd42319df1b52eeba4c062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
#6 0x7fc7c23bfbb1 in std::default_delete<kudu::tablet::AlterSchemaOpState>::operator()(kudu::tablet::AlterSchemaOpState*) const ../../../include/c++/8/bits/unique_ptr.h:81:2
I20260811 18:57:21.044723 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.062283 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.066112 3890 raft_consensus.cc:2454] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
#7 0x7fc7c237cf7b in std::unique_ptr<kudu::tablet::AlterSchemaOpState, std::default_delete<kudu::tablet::AlterSchemaOpState> >::~unique_ptr() ../../../include/c++/8/bits/unique_ptr.h:274:4
I20260811 18:57:21.073127 4169 ts_tablet_manager.cc:1918] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
#8 0x7fc7bfab3d04 in kudu::tablet::AlterSchemaOp::~AlterSchemaOp() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:126:7
I20260811 18:57:21.078022 4162 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.078629 4162 ts_tablet_manager.cc:1397] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.245s user 0.006s sys 0.004s
#9 0x7fc7bfab3d42 in kudu::tablet::AlterSchemaOp::~AlterSchemaOp() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:126:7
I20260811 18:57:21.079121 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 42405aec21e3405486b6c658438b5fb2
I20260811 18:57:21.080523 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fdeec0c5672b4b3190d7d7cf341cb118
I20260811 18:57:21.080631 4162 raft_consensus.cc:348] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.081511 4162 raft_consensus.cc:374] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.081707 4162 raft_consensus.cc:729] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.082525 4162 consensus_queue.cc:260] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.084630 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.085191 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.094691 4162 ts_tablet_manager.cc:1428] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.016s user 0.001s sys 0.003s
I20260811 18:57:21.097700 4162 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.100497 4169 log.cc:1196] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d9421bc220774f66b90bf537a47f1e13
I20260811 18:57:21.101045 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d8f42fcd065448e5a6332ff5512645ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.101917 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8f42fcd065448e5a6332ff5512645ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.103792 4163 tablet_bootstrap.cc:654] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.105407 3676 consensus_peers.cc:487] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1a55a42d1e64f04ad7f433614a3a1aa. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:21.106443 4914 consensus_queue.cc:1035] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.107803 3676 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.108420 4914 raft_consensus.cc:2781] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.108758 4914 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.109074 4924 consensus_queue.cc:1035] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.109762 4914 raft_consensus.cc:3037] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.115005 4914 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.117379 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b9115c298dfa4ecb970e76c7de2cd8e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.117656 4914 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:21.120034 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.121932 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c94c9b20d784df0811302a8381a7301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.123606 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.124043 4162 tablet_bootstrap.cc:654] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.125058 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c94c9b20d784df0811302a8381a7301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:21.126237 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 11432ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:21.127442 4167 ts_tablet_manager.cc:1918] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:21.130661 4167 log.cc:1196] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9c18120da2d245f495c52a670047760d
I20260811 18:57:21.132068 4167 ts_tablet_manager.cc:1939] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.133118 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.135195 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 351) took 3391 ms. Trace:
I20260811 18:57:21.138564 4167 rpcz_store.cc:276] 0811 18:57:17.743813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.743880 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:20.858247 (+3114367us) ts_tablet_manager.cc:1145] Deleting tablet 9c18120da2d245f495c52a670047760d
0811 18:57:20.858257 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.100126 (+241869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.127197 (+ 27071us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.135134 (+ 7937us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112188}
I20260811 18:57:21.141067 4167 tablet_replica.cc:330] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.141036 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9c18120da2d245f495c52a670047760d (table default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]) successfully deleted
I20260811 18:57:21.142673 4167 raft_consensus.cc:2229] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:21.142171 4162 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.143168 4162 ts_tablet_manager.cc:1397] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.001s
I20260811 18:57:21.143003 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.144711 4914 consensus_queue.cc:1035] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.145417 4836 consensus_queue.cc:1035] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.145921 4162 raft_consensus.cc:348] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.146248 4162 raft_consensus.cc:374] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.146497 4162 raft_consensus.cc:729] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.146880 4162 consensus_queue.cc:260] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.147359 4167 pending_rounds.cc:62] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:21.148974 4167 pending_rounds.cc:68] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400125563219968 op_type: NO_OP noop_request { }
I20260811 18:57:21.150151 3890 raft_consensus.cc:3037] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.151580 4167 raft_consensus.cc:2866] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.151965 4167 raft_consensus.cc:2258] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.153084 4167 ts_tablet_manager.cc:1905] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:21.158435 3824 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.056s (SENT)
I20260811 18:57:21.151400 4162 ts_tablet_manager.cc:1428] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260811 18:57:21.161175 4162 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.162621 4169 ts_tablet_manager.cc:1939] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.163321 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 341) took 3924 ms. Trace:
I20260811 18:57:21.164518 4162 tablet_bootstrap.cc:654] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.173243 4169 rpcz_store.cc:276] 0811 18:57:17.238328 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.238639 (+ 311us) service_pool.cc:227] Handling call
0811 18:57:20.437490 (+3198851us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:20.437502 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.502193 (+ 64691us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x6150001b3b24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:20.551452 (+ 49259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.875463 (+324011us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.163269 (+287806us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":146215168,"tablet-delete.queue_time_us":3194249}
I20260811 18:57:21.175544 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d9421bc220774f66b90bf537a47f1e13 (table default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]) successfully deleted
I20260811 18:57:21.183238 4836 consensus_queue.cc:1035] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.190992 4162 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.191476 4162 ts_tablet_manager.cc:1397] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.002s
I20260811 18:57:21.193382 4162 raft_consensus.cc:348] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.198676 4162 raft_consensus.cc:374] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.199018 4162 raft_consensus.cc:729] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.200117 4162 consensus_queue.cc:260] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.206398 4162 ts_tablet_manager.cc:1428] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.002s
I20260811 18:57:21.208562 4162 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.202615 4167 ts_tablet_manager.cc:1918] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.209911 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 357) took 3692 ms. Trace:
I20260811 18:57:21.210239 4167 log.cc:1196] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:21.211216 4167 ts_tablet_manager.cc:1939] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.218712 4169 rpcz_store.cc:276] 0811 18:57:17.511075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.511155 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:21.176005 (+3664850us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:21.202360 (+ 26355us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.203298 (+ 938us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3664560}
I20260811 18:57:21.219303 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 358) took 3685 ms. Trace:
I20260811 18:57:21.218097 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 348) took 3478 ms. Trace:
I20260811 18:57:21.220002 4167 rpcz_store.cc:276] 0811 18:57:17.739260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.739354 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:21.140950 (+3401596us) ts_tablet_manager.cc:1145] Deleting tablet 987b9ac6859444e3a86c56af786cc84e
0811 18:57:21.140968 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.176327 (+ 35359us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.202312 (+ 25985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.218016 (+ 15704us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3394614}
W20260811 18:57:21.220434 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c already in progress: creating tablet
I20260811 18:57:21.223701 4162 tablet_bootstrap.cc:654] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.228552 4924 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.224589 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 349) took 3484 ms. Trace:
I20260811 18:57:21.229238 4924 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.230417 4924 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.231448 4167 rpcz_store.cc:276] 0811 18:57:17.739664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.739746 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:21.224206 (+3484460us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:21.224223 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.224472 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3477669}
W20260811 18:57:21.232378 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 58c515c153df4317b3fb427983ee15e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58c515c153df4317b3fb427983ee15e0 already in progress: creating tablet
I20260811 18:57:21.232741 4924 consensus_queue.cc:1035] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.233652 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 347) took 3526 ms. Trace:
I20260811 18:57:21.230563 4169 rpcz_store.cc:276] 0811 18:57:17.534042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.534263 (+ 221us) service_pool.cc:227] Handling call
0811 18:57:21.219090 (+3684827us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:21.219114 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.219260 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3684499}
I20260811 18:57:21.234014 4167 rpcz_store.cc:276] 0811 18:57:17.707470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.707630 (+ 160us) service_pool.cc:227] Handling call
0811 18:57:21.232174 (+3524544us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:21.232194 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.233496 (+ 1302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3484594}
W20260811 18:57:21.234905 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b4de110bd3348f0b0dae21c9f2b6a19 already in progress: creating tablet
W20260811 18:57:21.235533 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 968fa8b8912b40a8bdba77db1d39c5c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 968fa8b8912b40a8bdba77db1d39c5c0 already in progress: creating tablet
I20260811 18:57:21.236797 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 350) took 3496 ms. Trace:
I20260811 18:57:21.238404 4167 rpcz_store.cc:276] 0811 18:57:17.740268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.740336 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:21.236416 (+3496080us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:21.236437 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.236656 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3487878}
I20260811 18:57:21.239181 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 359) took 3598 ms. Trace:
W20260811 18:57:21.239333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d2d7d986900244ddbc05dbe73a708dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2d7d986900244ddbc05dbe73a708dcf already in progress: creating tablet
I20260811 18:57:21.239516 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 363) took 3230 ms. Trace:
I20260811 18:57:21.239831 4167 rpcz_store.cc:276] 0811 18:57:18.009233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.009331 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:21.239301 (+3229970us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:21.239328 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.239476 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3226459}
W20260811 18:57:21.240599 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f75dc515d7f3473b8e43f20255a70e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f75dc515d7f3473b8e43f20255a70e3e already in progress: creating tablet
I20260811 18:57:21.245160 4169 rpcz_store.cc:276] 0811 18:57:17.640524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.640595 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:21.236765 (+3596170us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:21.236785 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.239087 (+ 2302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3595910}
I20260811 18:57:21.245920 4169 tablet_replica.cc:330] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:21.246169 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 968fa8b8912b40a8bdba77db1d39c5c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 968fa8b8912b40a8bdba77db1d39c5c0 already in progress: creating tablet
I20260811 18:57:21.246858 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 362) took 3237 ms. Trace:
I20260811 18:57:21.247264 4167 rpcz_store.cc:276] 0811 18:57:18.009104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.010031 (+ 927us) service_pool.cc:227] Handling call
0811 18:57:21.246531 (+3236500us) ts_tablet_manager.cc:1145] Deleting tablet ebf7720e31d84ae2ab592da1e7a3132a
0811 18:57:21.246551 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.246758 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3232618}
I20260811 18:57:21.248127 4167 tablet_replica.cc:330] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:21.248278 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ebf7720e31d84ae2ab592da1e7a3132a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf7720e31d84ae2ab592da1e7a3132a already in progress: creating tablet
I20260811 18:57:21.248641 4167 raft_consensus.cc:2229] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:21.249984 4167 pending_rounds.cc:62] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:21.250317 4167 pending_rounds.cc:68] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400116300894208 op_type: NO_OP noop_request { }
I20260811 18:57:21.251466 4167 raft_consensus.cc:2866] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.253616 4167 raft_consensus.cc:2258] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.254719 4167 ts_tablet_manager.cc:1905] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.253147 4162 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.255230 4162 ts_tablet_manager.cc:1397] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.006s
I20260811 18:57:21.257027 4162 raft_consensus.cc:348] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.257484 4162 raft_consensus.cc:374] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.257814 4162 raft_consensus.cc:729] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.258296 4162 consensus_queue.cc:260] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.261337 3890 raft_consensus.cc:2454] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.262867 4162 ts_tablet_manager.cc:1428] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:21.263285 4169 raft_consensus.cc:2229] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.261812 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.265333 3676 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.269132 4169 pending_rounds.cc:62] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:21.269433 4924 raft_consensus.cc:2781] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.269946 4924 raft_consensus.cc:686] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.270449 4924 consensus_queue.cc:237] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.274907 4169 pending_rounds.cc:68] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400125563219968 op_type: NO_OP noop_request { }
I20260811 18:57:21.275220 4169 raft_consensus.cc:2866] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.276104 4169 raft_consensus.cc:2258] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.277518 4169 ts_tablet_manager.cc:1905] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.282588 4162 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.282750 3892 raft_consensus.cc:2454] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.298874 4163 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.306216 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 289e607f10e14a2983bc6ecc93ce9bbc
I20260811 18:57:21.307497 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7ddb21035302480fbfaf4738e6229f2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.307749 3676 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:21.308351 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ddb21035302480fbfaf4738e6229f2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.309092 4729 raft_consensus.cc:2781] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.309240 4729 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.309284 4163 ts_tablet_manager.cc:1397] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.616s user 0.001s sys 0.015s
I20260811 18:57:21.309536 4729 raft_consensus.cc:3037] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.310883 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_632b7ba5285744ee8f4bdc969820dae6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.311884 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_632b7ba5285744ee8f4bdc969820dae6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.320417 4729 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.320773 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d81e93739f44d9e99b6ebc647e1d0e4 consensus state has no leader
I20260811 18:57:21.321645 4729 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.321684 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_22bccdb99d2b4a87be82af7e1e97bebd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.322654 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22bccdb99d2b4a87be82af7e1e97bebd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.324000 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3e16ca699564a61a49c4cb989ad865f" } new_table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" modify_external_catalogs: true
I20260811 18:57:21.326915 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:21.337139 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:21.338034 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fce143fe6d0849a0a3af8939df804986" } new_table_name: "default.Table_ddcc89fb831040ed9752150dc2ad4b51" modify_external_catalogs: true
I20260811 18:57:21.341702 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.349556 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c37e3fb34b3f446d854006b05dbe5a94" } new_table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" modify_external_catalogs: true
I20260811 18:57:21.352083 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" } modify_external_catalogs: true
I20260811 18:57:21.352550 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" } modify_external_catalogs: true
I20260811 18:57:21.354149 4169 ts_tablet_manager.cc:1918] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.354713 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c" } modify_external_catalogs: true
I20260811 18:57:21.355193 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c" } modify_external_catalogs: true
W20260811 18:57:21.359277 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 50939.5us
W20260811 18:57:21.368247 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 76308.3us
I20260811 18:57:21.374262 4836 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.374755 4836 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.382902 4836 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.387166 4162 tablet_bootstrap.cc:654] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.387681 4169 log.cc:1196] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:21.390566 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.396344 3888 raft_consensus.cc:3037] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.393919 4163 raft_consensus.cc:348] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.404048 3888 raft_consensus.cc:2454] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.404874 4163 raft_consensus.cc:374] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.408882 4167 ts_tablet_manager.cc:1918] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.410498 4163 raft_consensus.cc:729] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.411132 4163 consensus_queue.cc:260] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.412856 4163 ts_tablet_manager.cc:1428] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.103s user 0.003s sys 0.001s
I20260811 18:57:21.413475 4163 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.414551 4167 log.cc:1196] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d3d6daff4b7540309ff58755758e9752
I20260811 18:57:21.416272 4167 ts_tablet_manager.cc:1939] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.418629 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 364) took 3380 ms. Trace:
I20260811 18:57:21.419832 4162 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.420281 4162 ts_tablet_manager.cc:1397] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.138s user 0.005s sys 0.005s
I20260811 18:57:21.420799 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.421283 3888 raft_consensus.cc:2454] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.421864 4162 raft_consensus.cc:348] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.423517 4914 consensus_queue.cc:1035] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.425132 3676 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.426760 4836 raft_consensus.cc:2781] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.426986 4836 raft_consensus.cc:686] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.427445 4836 consensus_queue.cc:237] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.428535 4169 ts_tablet_manager.cc:1939] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.429355 4986 consensus_queue.cc:1035] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.459882 4162 raft_consensus.cc:374] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.460147 4162 raft_consensus.cc:729] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.460774 4162 consensus_queue.cc:260] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.462133 4162 ts_tablet_manager.cc:1428] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.042s user 0.002s sys 0.001s
I20260811 18:57:21.463073 4162 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.469717 4167 rpcz_store.cc:276] 0811 18:57:18.037862 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.037931 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:21.247966 (+3210035us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:21.247985 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.395870 (+147885us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.408602 (+ 12732us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.418570 (+ 9968us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3209700}
I20260811 18:57:21.470894 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d3d6daff4b7540309ff58755758e9752 (table default.Table_923c4369260d436fafb1aff17a2ec8d4 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]) successfully deleted
I20260811 18:57:21.471084 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 360) took 3829 ms. Trace:
I20260811 18:57:21.471465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 365) took 3331 ms. Trace:
I20260811 18:57:21.471378 4169 rpcz_store.cc:276] 0811 18:57:17.641643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.642049 (+ 406us) service_pool.cc:227] Handling call
0811 18:57:21.245775 (+3603726us) ts_tablet_manager.cc:1145] Deleting tablet 987b9ac6859444e3a86c56af786cc84e
0811 18:57:21.245793 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.290934 (+ 45141us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.353894 (+ 62960us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.470993 (+117099us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3603176}
I20260811 18:57:21.471747 4167 rpcz_store.cc:276] 0811 18:57:18.139835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.139907 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:21.471039 (+3331132us) ts_tablet_manager.cc:1145] Deleting tablet 38e24f8d4522466d81605ea37fa7064a
0811 18:57:21.471061 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.471312 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3329440}
I20260811 18:57:21.472288 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 361) took 3830 ms. Trace:
I20260811 18:57:21.472556 4167 tablet_replica.cc:330] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.472515 4169 rpcz_store.cc:276] 0811 18:57:17.641984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.642158 (+ 174us) service_pool.cc:227] Handling call
0811 18:57:21.471998 (+3829840us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:21.472015 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.472180 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3829076}
I20260811 18:57:21.473026 4167 raft_consensus.cc:2229] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:21.473079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 58c515c153df4317b3fb427983ee15e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58c515c153df4317b3fb427983ee15e0 already in progress: creating tablet
I20260811 18:57:21.473331 4167 raft_consensus.cc:2258] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.474546 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 362) took 3814 ms. Trace:
I20260811 18:57:21.474809 4169 rpcz_store.cc:276] 0811 18:57:17.659953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.660016 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:21.474291 (+3814275us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:21.474315 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.474456 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3814006}
I20260811 18:57:21.479290 4163 tablet_bootstrap.cc:654] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.479775 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.480997 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f" } modify_external_catalogs: true
I20260811 18:57:21.481349 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f" } modify_external_catalogs: true
I20260811 18:57:21.482801 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_25065dcc52b34b50986b88e1376ee9bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.483944 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25065dcc52b34b50986b88e1376ee9bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.485503 3552 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27] alter complete (version 5)
I20260811 18:57:21.486356 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.487323 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.490762 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cd4eb198addc42f2b6d860ef805c589f" } modify_external_catalogs: true
I20260811 18:57:21.491025 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cd4eb198addc42f2b6d860ef805c589f" } modify_external_catalogs: true
I20260811 18:57:21.491554 4952 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.491768 4952 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.492532 4952 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:21.492805 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet ffa8a4a1c0cb4834be501bdd7d468472 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.494071 4162 tablet_bootstrap.cc:654] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.496507 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d2d7d986900244ddbc05dbe73a708dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2d7d986900244ddbc05dbe73a708dcf already in progress: creating tablet
W20260811 18:57:21.500661 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 452ee450778e4d33adfa7e2745b5d970 consensus state has no leader
I20260811 18:57:21.515954 4169 tablet_replica.cc:330] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:21.516520 4169 raft_consensus.cc:2229] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.517153 4169 raft_consensus.cc:2258] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.518244 4169 ts_tablet_manager.cc:1905] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.525032 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:21.525576 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.525928 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:21.526620 3824 leader_election.cc:341] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:21.527143 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 99553.8us
I20260811 18:57:21.527835 3676 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.528352 4986 raft_consensus.cc:2781] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.528503 4986 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.528614 4986 raft_consensus.cc:3037] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:21.534219 3678 leader_election.cc:334] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.623s (SENT)
I20260811 18:57:21.534487 3678 leader_election.cc:302] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:21.534775 3678 raft_consensus.cc:2670] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:21.536235 4986 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.537825 4986 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.540216 3891 raft_consensus.cc:1261] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:57:21.544750 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:21.548055 4167 ts_tablet_manager.cc:1905] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:21.548481 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0cccebfc54484a9b9abffcce4e459fb5 consensus state has no leader
I20260811 18:57:21.549079 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" } modify_external_catalogs: true
I20260811 18:57:21.549412 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" } modify_external_catalogs: true
I20260811 18:57:21.550055 4163 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.550724 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d8aea8dc31914052832bdc83d638461f
I20260811 18:57:21.552030 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_66121f6795f240ea88c01d7c0ae02e00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.552932 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66121f6795f240ea88c01d7c0ae02e00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.554037 4163 ts_tablet_manager.cc:1397] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.137s user 0.001s sys 0.005s
I20260811 18:57:21.554855 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf2816cc2dfe45f0815ce50a5a929324
I20260811 18:57:21.555996 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b78f30561e284a9596437b5fc75df24f" } new_table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" modify_external_catalogs: true
W20260811 18:57:21.557346 3824 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.877s (SENT)
I20260811 18:57:21.559432 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.561827 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.562063 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet d8aea8dc31914052832bdc83d638461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.562307 3891 raft_consensus.cc:3037] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.565174 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d8aea8dc31914052832bdc83d638461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.565688 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.569653 3891 raft_consensus.cc:2454] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.572431 4995 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.572849 4995 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.575425 4995 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.581430 4924 consensus_queue.cc:1035] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.582284 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a01b26055fd433693bcb44191617f54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.583411 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a01b26055fd433693bcb44191617f54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.585955 4162 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.585915 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ffa8a4a1c0cb4834be501bdd7d468472
I20260811 18:57:21.586314 4162 ts_tablet_manager.cc:1397] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.124s user 0.005s sys 0.003s
I20260811 18:57:21.587654 4952 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.587936 4952 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.588091 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd120e0ab3b34418960cefc5eb459cdb" } new_table_name: "default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28" modify_external_catalogs: true
I20260811 18:57:21.589052 4952 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:21.587663 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.590479 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:21.591570 3824 leader_election.cc:341] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:21.594041 3676 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.595405 4994 raft_consensus.cc:2781] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.595728 4994 raft_consensus.cc:686] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.597535 4994 consensus_queue.cc:237] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.605067 3678 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.751s (SENT)
I20260811 18:57:21.611268 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:21.620218 3824 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.023s (SENT)
I20260811 18:57:21.620633 3824 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:21.621223 4952 raft_consensus.cc:2726] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:21.621631 3676 leader_election.cc:341] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:21.598373 4162 raft_consensus.cc:348] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.631472 4162 raft_consensus.cc:374] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.633016 4163 raft_consensus.cc:348] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.633811 4163 raft_consensus.cc:374] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.634128 4163 raft_consensus.cc:729] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.636683 4162 raft_consensus.cc:729] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:21.647855 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2605007cc2454599868f44c717deb2d2 has no consensus state
I20260811 18:57:21.648764 3548 catalog_manager.cc:6565] ReplaceTablet: tablet ffa8a4a1c0cb4834be501bdd7d468472 deleted and replaced by tablet 07a78361bbc44fcf86985f9f20c5665b
I20260811 18:57:21.648178 4162 consensus_queue.cc:260] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.637319 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.655323 4163 consensus_queue.cc:260] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.657229 4162 ts_tablet_manager.cc:1428] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.062s user 0.004s sys 0.000s
I20260811 18:57:21.657680 4162 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.661321 4914 raft_consensus.cc:482] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.661675 4914 raft_consensus.cc:504] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.661732 5008 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:21.665838 4914 leader_election.cc:288] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.666980 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2a926bfdbff42feaa7cfed27e0c04a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.667271 4163 ts_tablet_manager.cc:1428] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.113s user 0.000s sys 0.004s
W20260811 18:57:21.667783 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.670488 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_50769590dc5544e0a551369ba2c9768c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.671187 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50769590dc5544e0a551369ba2c9768c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.671324 4163 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.672287 5010 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:21.677374 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_533afa7d5e834d5cb281ddc9619575f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.678085 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_533afa7d5e834d5cb281ddc9619575f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:21.681936 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb82243777a9494b96959ced0c24de5a consensus state has no leader
I20260811 18:57:21.683516 4167 ts_tablet_manager.cc:1918] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:21.684011 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "125757b8c1a6448b841c63339f392339" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.684531 3890 raft_consensus.cc:2454] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.687963 4167 log.cc:1196] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a30bb384136f4a169a5751655437d5a3
I20260811 18:57:21.688827 4163 tablet_bootstrap.cc:654] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.688891 4167 ts_tablet_manager.cc:1939] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.689889 3676 leader_election.cc:302] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.690459 4914 raft_consensus.cc:2781] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.690783 4914 raft_consensus.cc:482] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.691061 4914 raft_consensus.cc:3037] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.695801 4914 raft_consensus.cc:504] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.697700 4914 leader_election.cc:288] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.703321 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd82f86a4579497fb4e50639ea1bf581"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.704066 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 366) took 3529 ms. Trace:
I20260811 18:57:21.704330 4167 rpcz_store.cc:276] 0811 18:57:18.160437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.163999 (+ 3562us) service_pool.cc:227] Handling call
0811 18:57:21.472407 (+3308408us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:21.472424 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.601304 (+128880us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.648085 (+ 46781us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.689791 (+ 41706us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307633}
W20260811 18:57:21.704670 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd82f86a4579497fb4e50639ea1bf581 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.705143 4167 tablet_replica.cc:330] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.705592 4167 raft_consensus.cc:2229] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.706034 4167 raft_consensus.cc:2258] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.706816 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a30bb384136f4a169a5751655437d5a3 (table default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]) successfully deleted
I20260811 18:57:21.707576 4167 ts_tablet_manager.cc:1905] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.713145 4162 tablet_bootstrap.cc:654] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.719604 3824 leader_election.cc:334] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.111s (SENT)
W20260811 18:57:21.724287 3824 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.119s (SENT)
I20260811 18:57:21.734745 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "125757b8c1a6448b841c63339f392339" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.735396 3890 raft_consensus.cc:3037] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.741257 3890 raft_consensus.cc:2454] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.742286 3676 leader_election.cc:302] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.743057 4914 raft_consensus.cc:2781] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.743369 4914 raft_consensus.cc:686] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.744843 4914 consensus_queue.cc:237] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.752562 4163 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.754065 4163 ts_tablet_manager.cc:1397] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.021s
I20260811 18:57:21.755488 4163 raft_consensus.cc:348] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.755862 4163 raft_consensus.cc:374] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.756105 4163 raft_consensus.cc:729] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.756491 4163 consensus_queue.cc:260] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.759487 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_ddcc89fb831040ed9752150dc2ad4b51 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:21.760928 4163 ts_tablet_manager.cc:1428] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:21.761386 4163 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.766891 4163 tablet_bootstrap.cc:654] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.773136 4162 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.773507 4162 ts_tablet_manager.cc:1397] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.116s user 0.006s sys 0.003s
I20260811 18:57:21.774955 4162 raft_consensus.cc:348] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.775048 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.775404 4162 raft_consensus.cc:374] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.775550 4162 raft_consensus.cc:729] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.776014 4162 consensus_queue.cc:260] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.776844 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.784148 3872 tablet_service.cc:1467] Processing CreateTablet for tablet cdb92902c2584751af77d3117f52cb4e (DEFAULT_TABLE table=default.Table_d3652d29236a4b12b7abd44659c70311 [id=868b4bfe43bb47e3ae5a5b5308ff85bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.772903 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_853572d26c9642b9b8ad616f574301dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.789793 3713 tablet_service.cc:1467] Processing CreateTablet for tablet cdb92902c2584751af77d3117f52cb4e (DEFAULT_TABLE table=default.Table_d3652d29236a4b12b7abd44659c70311 [id=868b4bfe43bb47e3ae5a5b5308ff85bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.790591 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb92902c2584751af77d3117f52cb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.790820 3861 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6378d1219433cb724148f1ae74b25 (DEFAULT_TABLE table=default.Table_cee8eb42df9b40cd85eb35169133787a [id=b8fedf26521040b1bb58aa7edb6373c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.790279 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 (DEFAULT_TABLE table=default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.794507 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 (DEFAULT_TABLE table=default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.792443 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:21.795451 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302cf69ea7c14fdaac72f6a9a9aff113. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.797793 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_853572d26c9642b9b8ad616f574301dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.798573 3706 tablet_service.cc:1467] Processing CreateTablet for tablet f5910ff87f9e4a7e8fbf877adf8c7d56 (DEFAULT_TABLE table=default.Table_4f20eabb4925482bbf71fe270dde24af [id=f7310226124846a29f01f43aff0bafcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.799196 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5910ff87f9e4a7e8fbf877adf8c7d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.800349 3856 tablet_service.cc:1467] Processing CreateTablet for tablet f5910ff87f9e4a7e8fbf877adf8c7d56 (DEFAULT_TABLE table=default.Table_4f20eabb4925482bbf71fe270dde24af [id=f7310226124846a29f01f43aff0bafcb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:21.800846 3678 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.165s (SENT)
I20260811 18:57:21.801281 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2605007cc2454599868f44c717deb2d2 (DEFAULT_TABLE table=default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4 [id=1323eccbf4b04ee8b1a819101b8ad9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.801888 3865 tablet_service.cc:1467] Processing CreateTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 (DEFAULT_TABLE table=default.Table_886df95cce4342f998d27c8661df890e [id=abbf777ec3574c3e9fe6c01c4073fd57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.803830 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_25065dcc52b34b50986b88e1376ee9bc" } new_table_name: "default.Table_ed61380b12e548ce843c5056342085b9" modify_external_catalogs: true
I20260811 18:57:21.804957 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 2605007cc2454599868f44c717deb2d2 (DEFAULT_TABLE table=default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4 [id=1323eccbf4b04ee8b1a819101b8ad9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.805523 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2605007cc2454599868f44c717deb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.806639 3715 tablet_service.cc:1467] Processing CreateTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 (DEFAULT_TABLE table=default.Table_886df95cce4342f998d27c8661df890e [id=abbf777ec3574c3e9fe6c01c4073fd57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.806691 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 90576c068ce4468a9c572307dcb79b10 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.807765 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa95131ce50943e3b0c6b5bdd5a30939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.812186 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 90576c068ce4468a9c572307dcb79b10 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.812858 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90576c068ce4468a9c572307dcb79b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.815486 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 (DEFAULT_TABLE table=default.Table_680160aaac0a4c8da66634dc41d6abb9 [id=8e026fcb9e9d456f8b9326a3b4712e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.816439 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb30e64e86040098dcee5fc2e9b7266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.816972 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 (DEFAULT_TABLE table=default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 [id=f9a016e11f694e16b5b75c9a742f6f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.817546 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f389d2b7ff354ac4bab5d68b9ff201c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.819988 3862 tablet_service.cc:1467] Processing CreateTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 (DEFAULT_TABLE table=default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 [id=f9a016e11f694e16b5b75c9a742f6f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.815585 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6378d1219433cb724148f1ae74b25 (DEFAULT_TABLE table=default.Table_cee8eb42df9b40cd85eb35169133787a [id=b8fedf26521040b1bb58aa7edb6373c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.819381 3863 tablet_service.cc:1467] Processing CreateTablet for tablet fb878de2c0d94408aa6e98447bd7e958 (DEFAULT_TABLE table=default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.821161 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa95131ce50943e3b0c6b5bdd5a30939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.826776 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6378d1219433cb724148f1ae74b25. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.816556 3678 leader_election.cc:334] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.648s (SENT)
I20260811 18:57:21.831161 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fb878de2c0d94408aa6e98447bd7e958 (DEFAULT_TABLE table=default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.832880 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb878de2c0d94408aa6e98447bd7e958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.833227 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb878de2c0d94408aa6e98447bd7e958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.833791 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6378d1219433cb724148f1ae74b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.834745 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 (DEFAULT_TABLE table=default.Table_680160aaac0a4c8da66634dc41d6abb9 [id=8e026fcb9e9d456f8b9326a3b4712e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.836369 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 7216c095cc03424bbbe97a52a9600ebb (DEFAULT_TABLE table=default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 [id=cf2ed3c747254c8981e9e04fa0a40d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.837041 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7216c095cc03424bbbe97a52a9600ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.838125 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 6432abd4fc5c4f9ca95971ca4795e1cf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.838498 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 7216c095cc03424bbbe97a52a9600ebb (DEFAULT_TABLE table=default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 [id=cf2ed3c747254c8981e9e04fa0a40d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.839071 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb30e64e86040098dcee5fc2e9b7266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.840094 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5910ff87f9e4a7e8fbf877adf8c7d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.840483 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f389d2b7ff354ac4bab5d68b9ff201c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.841400 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:21.842692 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:21.843377 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:21.847028 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 (DEFAULT_TABLE table=default.Table_4d5db1f0452e40eea01177ba93223341 [id=82ab42a0b1e448a39ec7d6a27771daa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.848591 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc1822d4b94081ad86a6177acb3afe (DEFAULT_TABLE table=default.Table_90566ec48eaf42c8be4c06681c3c0268 [id=fc2d04fe3bad4dbd9dccd38355861a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.849061 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2605007cc2454599868f44c717deb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.849776 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302cf69ea7c14fdaac72f6a9a9aff113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.841178 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7216c095cc03424bbbe97a52a9600ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.851284 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cc1822d4b94081ad86a6177acb3afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.843858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 (DEFAULT_TABLE table=default.Table_4d5db1f0452e40eea01177ba93223341 [id=82ab42a0b1e448a39ec7d6a27771daa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.857719 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb92902c2584751af77d3117f52cb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.862573 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90576c068ce4468a9c572307dcb79b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.862941 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b922f2a71544e8a18533aa0a1ccb52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.868781 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b922f2a71544e8a18533aa0a1ccb52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.871519 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc1822d4b94081ad86a6177acb3afe (DEFAULT_TABLE table=default.Table_90566ec48eaf42c8be4c06681c3c0268 [id=fc2d04fe3bad4dbd9dccd38355861a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.872157 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cc1822d4b94081ad86a6177acb3afe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.874583 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.115s (SENT) [suppressed 27 similar messages]
I20260811 18:57:21.879936 4169 ts_tablet_manager.cc:1918] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:21.880400 4169 log.cc:1196] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9c18120da2d245f495c52a670047760d
I20260811 18:57:21.882962 4169 ts_tablet_manager.cc:1939] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.883328 3857 tablet_service.cc:1467] Processing CreateTablet for tablet f06e5b6a30764caa978ba0022ca1d6bc (DEFAULT_TABLE table=default.Table_c2e58ff1b0c44ee8b71812ae6399b4da [id=d81e384f7a3e426ca75fe55e69ef3051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.885023 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e5b6a30764caa978ba0022ca1d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.904174 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 363) took 4227 ms. Trace:
I20260811 18:57:21.904479 4169 rpcz_store.cc:276] 0811 18:57:17.664828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.664905 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:21.515767 (+3850862us) ts_tablet_manager.cc:1145] Deleting tablet 9c18120da2d245f495c52a670047760d
0811 18:57:21.515780 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.636601 (+120821us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.815003 (+178402us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.892221 (+ 77218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3850539}
I20260811 18:57:21.906541 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 366) took 4018 ms. Trace:
I20260811 18:57:21.906708 4169 rpcz_store.cc:276] 0811 18:57:17.887279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.888560 (+ 1281us) service_pool.cc:227] Handling call
0811 18:57:21.905259 (+4016699us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:21.905282 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.905405 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4016367}
I20260811 18:57:21.907960 4169 tablet_replica.cc:330] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:21.908445 4169 raft_consensus.cc:2229] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.933262 3706 tablet_service.cc:1467] Processing CreateTablet for tablet f06e5b6a30764caa978ba0022ca1d6bc (DEFAULT_TABLE table=default.Table_c2e58ff1b0c44ee8b71812ae6399b4da [id=d81e384f7a3e426ca75fe55e69ef3051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.947364 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e5b6a30764caa978ba0022ca1d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.944550 4169 raft_consensus.cc:2258] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.948333 4990 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.954382 4914 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.958628 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7abca85e5a6d4b55a38c0f9e50f02f76 (DEFAULT_TABLE table=default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.961395 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abca85e5a6d4b55a38c0f9e50f02f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.961757 4169 ts_tablet_manager.cc:1905] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.953092 4990 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.844755 3547 catalog_manager.cc:5495] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:21.969667 4162 ts_tablet_manager.cc:1428] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.186s user 0.005s sys 0.000s
I20260811 18:57:21.966984 4914 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.971735 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 57896.7us
I20260811 18:57:21.974625 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9c18120da2d245f495c52a670047760d (table default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]) successfully deleted
W20260811 18:57:21.978080 3539 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 69655.4us
W20260811 18:57:21.978951 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b1ce1042ae974f4dbac964bd7ea16a94 because the tablet was not found. No further retry: Not found: Tablet not found: b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:21.996753 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_22bccdb99d2b4a87be82af7e1e97bebd" } new_table_name: "default.Table_c05597a50f6e4aea9c286eae0cf9a54d" modify_external_catalogs: true
I20260811 18:57:21.997507 4163 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.000730 4914 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.002606 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:22.018141 3678 leader_election.cc:334] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.684s (SENT)
I20260811 18:57:22.019021 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7abca85e5a6d4b55a38c0f9e50f02f76 (DEFAULT_TABLE table=default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.019659 3890 raft_consensus.cc:2454] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.031337 5010 tablet.cc:1722] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:22.033046 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.033448 3893 raft_consensus.cc:2454] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.033749 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abca85e5a6d4b55a38c0f9e50f02f76. 1 dirs total, 0 dirs full, 0 dirs failed
#10 0x7fc7bfa84211 in std::default_delete<kudu::tablet::Op>::operator()(kudu::tablet::Op*) const ../../../include/c++/8/bits/unique_ptr.h:81:2
W20260811 18:57:22.041298 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b40da8cb7324da9804e6fdafd8e8266 has no consensus state
I20260811 18:57:22.042116 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
I20260811 18:57:22.043120 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:22.045764 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e23d79bc476d496eaa713465bcfb1abf" } new_table_name: "default.Table_04ac8209b46549e3bb3f12971ab01b73" modify_external_catalogs: true
I20260811 18:57:22.047528 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" } modify_external_catalogs: true
#11 0x7fc7bfa76c5b in std::unique_ptr<kudu::tablet::Op, std::default_delete<kudu::tablet::Op> >::~unique_ptr() ../../../include/c++/8/bits/unique_ptr.h:274:4
I20260811 18:57:22.047878 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" } modify_external_catalogs: true
#12 0x7fc7bfa93862 in kudu::tablet::OpDriver::~OpDriver() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/op_driver.h:304:16
I20260811 18:57:22.048859 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4927a23edf074354bdd4cf94e4b74eff" } modify_external_catalogs: true
I20260811 18:57:22.049135 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4927a23edf074354bdd4cf94e4b74eff" } modify_external_catalogs: true
I20260811 18:57:22.051409 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.052181 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dbfb0ccfdef4c018c7eca1d0452375a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#13 0x7fc7bfa9382a in kudu::RefCountedThreadSafe<kudu::tablet::OpDriver, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver> >::DeleteInternal(kudu::tablet::OpDriver const*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:153:44
#14 0x7fc7bfa93800 in kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver>::Destruct(kudu::tablet::OpDriver const*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:116:5
I20260811 18:57:22.062770 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.062865 4990 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.063443 3676 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.064208 3676 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.065017 4994 raft_consensus.cc:2781] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
#15 0x7fc7bfa937ce in kudu::RefCountedThreadSafe<kudu::tablet::OpDriver, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver> >::Release() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:144:7
I20260811 18:57:22.066746 5008 tablet.cc:1722] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
#16 0x7fc7bfa76d37 in scoped_refptr<kudu::tablet::OpDriver>::~scoped_refptr() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:266:13
#17 0x7fc7bfa60add in kudu::tablet::TabletReplica::SubmitAlterSchema(std::unique_ptr<kudu::tablet::AlterSchemaOpState, std::default_delete<kudu::tablet::AlterSchemaOpState> >, kudu::MonoTime) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_replica.cc:627:1
W20260811 18:57:22.074508 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9342ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
#18 0x7fc7c232926d in kudu::tserver::TabletServiceAdminImpl::AlterSchema(kudu::tserver::AlterSchemaRequestPB const*, kudu::tserver::AlterSchemaResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1213:23
I20260811 18:57:22.077044 4163 ts_tablet_manager.cc:1397] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.237s user 0.088s sys 0.064s
I20260811 18:57:22.087790 4994 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.088097 4994 raft_consensus.cc:3037] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.088878 4162 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:22.091390 4990 raft_consensus.cc:2781] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.091754 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.093466 4990 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.093729 4990 raft_consensus.cc:3037] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.097509 4169 ts_tablet_manager.cc:1918] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.097738 4169 log.cc:1196] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:22.098037 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.098577 4169 ts_tablet_manager.cc:1939] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.098996 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 367) took 4211 ms. Trace:
I20260811 18:57:22.099159 4169 rpcz_store.cc:276] 0811 18:57:17.887441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.888919 (+ 1478us) service_pool.cc:227] Handling call
0811 18:57:21.907462 (+4018543us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:21.907476 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.024012 (+116536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.097322 (+ 73310us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.098957 (+ 1635us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4018272}
I20260811 18:57:22.079913 4163 raft_consensus.cc:348] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:22.102495 4163 raft_consensus.cc:374] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.102626 4163 raft_consensus.cc:729] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.102988 4163 consensus_queue.cc:260] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:22.104043 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.104880 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_971d1dda6c9c4e65a732e83f6528e2de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.106405 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7991dabe263b44f6b7b7d1d43b96ee44
I20260811 18:57:22.107429 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a01b26055fd433693bcb44191617f54" } new_table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" modify_external_catalogs: true
W20260811 18:57:22.108373 3824 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.845s (SENT)
I20260811 18:57:22.108681 3824 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:22.106206 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.109345 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f25ff8a9e3a944269e9799995c73ebe0 (table default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]) successfully deleted
I20260811 18:57:22.111052 4994 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.112793 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.113045 3893 raft_consensus.cc:3037] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.117755 3893 raft_consensus.cc:2454] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:22.118992 3824 raft_consensus.cc:2670] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:22.121793 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.122615 4163 ts_tablet_manager.cc:1428] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:22.110244 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:22.123281 4163 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:22.123620 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 6432abd4fc5c4f9ca95971ca4795e1cf deleted and replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
I20260811 18:57:22.124006 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_10f91c342797480e9545a00a8d6be4a9" } new_table_name: "default.Table_26f305827594444882a9b3b84edba403" modify_external_catalogs: true
W20260811 18:57:22.114881 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae has no consensus state
I20260811 18:57:22.125389 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 377) took 4138 ms. Trace:
I20260811 18:57:22.125631 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ace00158b744bd8a97dc1c47fc6049c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.125612 4169 rpcz_store.cc:276] 0811 18:57:17.987057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.001297 (+ 14240us) service_pool.cc:227] Handling call
0811 18:57:22.125070 (+4123773us) ts_tablet_manager.cc:1145] Deleting tablet ebf7720e31d84ae2ab592da1e7a3132a
0811 18:57:22.125082 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.125310 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4118276}
I20260811 18:57:22.126322 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 378) took 4119 ms. Trace:
I20260811 18:57:22.126629 4169 rpcz_store.cc:276] 0811 18:57:18.006401 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.006476 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:22.126067 (+4119591us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:22.126079 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.126204 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4119110}
I20260811 18:57:22.127424 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 379) took 3988 ms. Trace:
I20260811 18:57:22.127642 4169 rpcz_store.cc:276] 0811 18:57:18.139084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.139174 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:22.127211 (+3988037us) ts_tablet_manager.cc:1145] Deleting tablet 38e24f8d4522466d81605ea37fa7064a
0811 18:57:22.127222 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.127355 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3987692}
W20260811 18:57:22.128401 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ebf7720e31d84ae2ab592da1e7a3132a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf7720e31d84ae2ab592da1e7a3132a already in progress: creating tablet
I20260811 18:57:22.128575 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.128782 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f75dc515d7f3473b8e43f20255a70e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f75dc515d7f3473b8e43f20255a70e3e already in progress: creating tablet
I20260811 18:57:22.129158 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 380) took 3905 ms. Trace:
I20260811 18:57:22.116581 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.129386 4169 rpcz_store.cc:276] 0811 18:57:18.223715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.223930 (+ 215us) service_pool.cc:227] Handling call
0811 18:57:22.128831 (+3904901us) ts_tablet_manager.cc:1145] Deleting tablet 4a64df7c295c4d9baded1b63a9980581
0811 18:57:22.128841 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.129033 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3841411}
W20260811 18:57:22.130069 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_056c759cfe3543f7a43de5b2fa3a4160 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.130599 3676 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.131206 4995 raft_consensus.cc:2781] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:22.117180 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 330801031cc74eb2a5bc2caf53a18ead consensus state has no leader
I20260811 18:57:22.132092 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.133272 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8554df7eb5f4fbbbe828834c1d45f25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.119803 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:22.134684 4995 raft_consensus.cc:686] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.135143 4995 consensus_queue.cc:237] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.122771 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9ec4bba841dd446aa3db751358ce7f32
I20260811 18:57:22.136783 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e863da40303e45bda8d9464f8ec9b8a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.137385 4836 raft_consensus.cc:482] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:22.137553 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e863da40303e45bda8d9464f8ec9b8a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.140038 4836 raft_consensus.cc:504] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.141139 4836 leader_election.cc:288] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.142189 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a08e6dbf04e3fa58689ee0cd45db3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.142756 3890 raft_consensus.cc:2454] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.144872 4994 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.145278 3676 leader_election.cc:302] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:22.145298 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4a64df7c295c4d9baded1b63a9980581 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a64df7c295c4d9baded1b63a9980581 already in progress: creating tablet
I20260811 18:57:22.145752 4163 tablet_bootstrap.cc:654] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.148195 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.150566 4994 raft_consensus.cc:2781] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.150919 4994 raft_consensus.cc:482] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.152609 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.153760 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed948e1e9ee14cffae82bcd5517709f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.156536 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 59108us
I20260811 18:57:22.157863 4994 raft_consensus.cc:3037] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
#19 0x7fc7bc95c786 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_5::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:241:13
I20260811 18:57:22.160414 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 381) took 3868 ms. Trace:
I20260811 18:57:22.160796 4169 rpcz_store.cc:276] 0811 18:57:18.261588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.263172 (+ 1584us) service_pool.cc:227] Handling call
0811 18:57:22.129923 (+3866751us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:22.129932 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.130204 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3812639}
I20260811 18:57:22.165436 4990 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:22.167222 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet eb8f5b87062349c4b19ec99fa0175f24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb8f5b87062349c4b19ec99fa0175f24 already in progress: creating tablet
#20 0x7fc7bc95c4b5 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_5>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:22.168682 4990 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.169492 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.170069 3890 raft_consensus.cc:3037] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.176676 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.177359 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.178000 4169 tablet_replica.cc:330] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:22.178133 3678 leader_election.cc:334] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.176s (SENT)
I20260811 18:57:22.178529 3678 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:22.178594 4169 raft_consensus.cc:2229] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.179004 4169 raft_consensus.cc:2258] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:22.179520 3678 raft_consensus.cc:2670] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:22.180414 4169 ts_tablet_manager.cc:1905] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.181392 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.178648 3890 raft_consensus.cc:2454] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.183259 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.183745 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.183602 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.187703 3676 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.188522 4914 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.188766 4914 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.193113 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.193653 3890 raft_consensus.cc:2454] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.199685 4163 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.199971 4163 ts_tablet_manager.cc:1397] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.000s sys 0.006s
I20260811 18:57:22.200810 3676 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.201577 4163 raft_consensus.cc:348] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.202054 4163 raft_consensus.cc:374] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.206391 4163 raft_consensus.cc:729] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.206852 4163 consensus_queue.cc:260] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.212363 4914 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.215111 4163 ts_tablet_manager.cc:1428] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:22.218609 4986 raft_consensus.cc:2781] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.218966 4986 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.222010 4986 raft_consensus.cc:3037] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.223285 4990 raft_consensus.cc:2781] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.226781 4163 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:22.229395 4986 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.234673 4990 raft_consensus.cc:686] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.236732 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.237437 3890 raft_consensus.cc:3037] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:22.246861 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3030c31bc2b24ee296f29b5d5aa402ff has no consensus state
I20260811 18:57:22.247357 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet e14f8717bcd94d6d88295fd8be130ca5 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:22.248013 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:22.248334 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 3263da75f0d84528b7c8cc55dcaeaa46 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.249191 4986 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.249385 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1204e0b30877462982c245f439f0bae4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.250314 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1204e0b30877462982c245f439f0bae4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.250651 4163 tablet_bootstrap.cc:654] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.251312 4167 ts_tablet_manager.cc:1918] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:22.251565 4167 log.cc:1196] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cf285cdf5235451bb1531cb294017101
I20260811 18:57:22.256906 4169 ts_tablet_manager.cc:1918] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.257273 4169 log.cc:1196] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:22.258383 4167 ts_tablet_manager.cc:1939] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:22.259196 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 367) took 4072 ms. Trace:
I20260811 18:57:22.259550 4169 ts_tablet_manager.cc:1939] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.259717 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:22.260839 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 0ace00158b744bd8a97dc1c47fc6049c deleted and replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b
I20260811 18:57:22.260640 4167 rpcz_store.cc:276] 0811 18:57:18.186313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.186591 (+ 278us) service_pool.cc:227] Handling call
0811 18:57:21.704950 (+3518359us) ts_tablet_manager.cc:1145] Deleting tablet cf285cdf5235451bb1531cb294017101
0811 18:57:21.704970 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.761665 (+ 56695us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.176240 (+414575us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.259128 (+ 82888us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3517986}
I20260811 18:57:22.261289 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 368) took 4035 ms. Trace:
I20260811 18:57:22.261536 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a93638c401d94d3d8dad242dd4c9b60a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.263134 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cf285cdf5235451bb1531cb294017101 (table default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]) successfully deleted
I20260811 18:57:22.263170 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c with delete_type TABLET_DATA_DELETED (replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.264717 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a93638c401d94d3d8dad242dd4c9b60a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.264914 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 388) took 3726 ms. Trace:
I20260811 18:57:22.265096 4169 rpcz_store.cc:276] 0811 18:57:18.538208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539790 (+ 1582us) service_pool.cc:227] Handling call
0811 18:57:22.177855 (+3638065us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:22.177881 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.217514 (+ 39633us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.256675 (+ 39161us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.264845 (+ 8170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3632993}
I20260811 18:57:22.271499 4994 raft_consensus.cc:504] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.271963 4169 tablet_replica.cc:330] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:22.273036 4990 consensus_queue.cc:237] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.275806 4169 raft_consensus.cc:2229] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.276712 4169 raft_consensus.cc:2258] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.278296 4169 ts_tablet_manager.cc:1905] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.283082 3890 raft_consensus.cc:2454] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.284286 3676 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:22.284778 3678 leader_election.cc:334] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.750s (SENT)
I20260811 18:57:22.285557 4986 raft_consensus.cc:2781] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.286001 4986 raft_consensus.cc:686] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.286554 4986 consensus_queue.cc:237] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.290823 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619" } modify_external_catalogs: true
I20260811 18:57:22.291208 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619" } modify_external_catalogs: true
I20260811 18:57:22.298524 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a08e6dbf04e3fa58689ee0cd45db3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.298902 3893 raft_consensus.cc:3037] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.299592 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db899a033c3144559d1d5c7bf1d92fbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.300729 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db899a033c3144559d1d5c7bf1d92fbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.301765 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 374c94a12e374d0e82180421c80d92d0 (DEFAULT_TABLE table=default.Table_50769590dc5544e0a551369ba2c9768c [id=c8432eabc0be4256880d9eb6a5afde61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.302614 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 374c94a12e374d0e82180421c80d92d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.303505 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f713f72dfeb48658564074b098a97d9" } new_table_name: "default.Table_5727f2e5abe54ecaaeec9a52f1bd78da" modify_external_catalogs: true
I20260811 18:57:22.304345 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c with delete_type TABLET_DATA_DELETED (replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.304601 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 374c94a12e374d0e82180421c80d92d0 (DEFAULT_TABLE table=default.Table_50769590dc5544e0a551369ba2c9768c [id=c8432eabc0be4256880d9eb6a5afde61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.305176 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 374c94a12e374d0e82180421c80d92d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.308602 4162 tablet_bootstrap.cc:654] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.304575 4167 rpcz_store.cc:276] 0811 18:57:18.225745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.225819 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:22.261082 (+4035263us) ts_tablet_manager.cc:1145] Deleting tablet 4a64df7c295c4d9baded1b63a9980581
0811 18:57:22.261095 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.261249 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4034940}
I20260811 18:57:22.310750 3857 tablet_service.cc:1467] Processing CreateTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 (DEFAULT_TABLE table=default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 [id=77b12e379e234e6f8285ebe4c3c46aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.311409 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff (DEFAULT_TABLE table=default.Table_45b8aa8da0974fe59719c2d87bb95773 [id=98a8126304dd4be4a2575ef1e86cbeed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.311945 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea579c33f8b14e9fa7fa389ce7f45931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.312584 4994 leader_election.cc:288] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.313601 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3030c31bc2b24ee296f29b5d5aa402ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.315258 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8f42fcd065448e5a6332ff5512645ed" } modify_external_catalogs: true
I20260811 18:57:22.315954 3864 tablet_service.cc:1467] Processing CreateTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae (DEFAULT_TABLE table=default.Table_c05597a50f6e4aea9c286eae0cf9a54d [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.316813 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.317701 3870 tablet_service.cc:1467] Processing CreateTablet for tablet c77882ff8d974057a61f037f1943f58f (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.317226 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4a64df7c295c4d9baded1b63a9980581 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a64df7c295c4d9baded1b63a9980581 already in progress: creating tablet
I20260811 18:57:22.318450 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8f42fcd065448e5a6332ff5512645ed" } modify_external_catalogs: true
I20260811 18:57:22.318794 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae (DEFAULT_TABLE table=default.Table_c05597a50f6e4aea9c286eae0cf9a54d [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.319064 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff (DEFAULT_TABLE table=default.Table_45b8aa8da0974fe59719c2d87bb95773 [id=98a8126304dd4be4a2575ef1e86cbeed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.319393 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.321201 3723 tablet_service.cc:1467] Processing CreateTablet for tablet c77882ff8d974057a61f037f1943f58f (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.321859 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77882ff8d974057a61f037f1943f58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.322546 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3030c31bc2b24ee296f29b5d5aa402ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.323518 3722 tablet_service.cc:1467] Processing CreateTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 (DEFAULT_TABLE table=default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 [id=77b12e379e234e6f8285ebe4c3c46aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.324065 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea579c33f8b14e9fa7fa389ce7f45931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.325320 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77882ff8d974057a61f037f1943f58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.331296 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2605007cc2454599868f44c717deb2d2
I20260811 18:57:22.332255 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 515d5b5afbd74ec5ab90a503aef2f4fb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.332175 3555 catalog_manager.cc:5495] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.332913 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:22.333515 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed] alter complete (version 1)
I20260811 18:57:22.334542 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2605007cc2454599868f44c717deb2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.336131 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:22.336944 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 3263da75f0d84528b7c8cc55dcaeaa46 deleted and replaced by tablet 4dd203b7eb8643f493b4642c0396e258
I20260811 18:57:22.337809 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e14f8717bcd94d6d88295fd8be130ca5
I20260811 18:57:22.338024 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.338475 3546 catalog_manager.cc:6565] ReplaceTablet: tablet e14f8717bcd94d6d88295fd8be130ca5 deleted and replaced by tablet 649cb98f553a418b911e307fdaf49917
I20260811 18:57:22.338953 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.339382 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.340051 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.340368 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 2605007cc2454599868f44c717deb2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.342041 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.342504 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 369) took 4048 ms. Trace:
I20260811 18:57:22.342869 4167 rpcz_store.cc:276] 0811 18:57:18.264244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.264479 (+ 235us) service_pool.cc:227] Handling call
0811 18:57:22.310055 (+4045576us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:22.310069 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.313028 (+ 2959us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4045252}
I20260811 18:57:22.343571 4167 tablet_replica.cc:330] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:22.354327 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260811 18:57:22.359824 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7d747f03ad234805a4ea4ef60840d426 consensus state has no leader
I20260811 18:57:22.360096 3858 tablet_service.cc:1467] Processing CreateTablet for tablet c2b283002d0e4316844e482468f97c31 (DEFAULT_TABLE table=default.Table_5fef93c0a9b546199a3217a8b8bd67dd [id=65c745bb0fdf4b85bf0e3451c8a180b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.360225 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.360785 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2b283002d0e4316844e482468f97c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.361845 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a847581412494e34b2c76e605ff11772
W20260811 18:57:22.364388 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eb8f5b87062349c4b19ec99fa0175f24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb8f5b87062349c4b19ec99fa0175f24 already in progress: creating tablet
I20260811 18:57:22.366430 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7d747f03ad234805a4ea4ef60840d426 (DEFAULT_TABLE table=default.Table_4f713f72dfeb48658564074b098a97d9 [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.371897 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e3a2005266804c6c94f0325437caa659"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.372718 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3a2005266804c6c94f0325437caa659 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.374382 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d747f03ad234805a4ea4ef60840d426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.375576 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_815d74faaf044b2f9000790283e989c9" } modify_external_catalogs: true
I20260811 18:57:22.375885 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_815d74faaf044b2f9000790283e989c9" } modify_external_catalogs: true
I20260811 18:57:22.378055 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.379345 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.379755 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7d747f03ad234805a4ea4ef60840d426 (DEFAULT_TABLE table=default.Table_4f713f72dfeb48658564074b098a97d9 [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.380529 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d747f03ad234805a4ea4ef60840d426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.383035 4167 raft_consensus.cc:2229] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.383478 4167 raft_consensus.cc:2258] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.383654 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 97997b0b3a5d46b487a9a33dcc30f9cd (DEFAULT_TABLE table=default.Table_ab9ea53bcc98448384f4fd3d30d2528f [id=e5b7ad7f9abb4ff190a1d4f32e468362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.384392 4167 ts_tablet_manager.cc:1905] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.385047 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97997b0b3a5d46b487a9a33dcc30f9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.386123 3893 raft_consensus.cc:2454] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.386837 3676 leader_election.cc:302] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.387691 4990 raft_consensus.cc:2781] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.389963 4994 consensus_queue.cc:1035] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:22.390314 3676 consensus_peers.cc:487] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f25ff8a9e3a944269e9799995c73ebe0. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.392596 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" } modify_external_catalogs: true
I20260811 18:57:22.392939 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" } modify_external_catalogs: true
I20260811 18:57:22.393704 4990 raft_consensus.cc:686] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.394066 4990 consensus_queue.cc:237] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.396134 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.397171 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e12ffa777d684216bfc19d52f2dad4be" } new_table_name: "default.Table_28fea07a2f434dc1997642a34e6b6e2b" modify_external_catalogs: true
I20260811 18:57:22.399113 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3294569c49f84a2291010174baee10af" } new_table_name: "default.Table_8076b90bff8d49c08328583af352ae7c" modify_external_catalogs: true
I20260811 18:57:22.401755 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c6e9da8dfb4a4c6c9275c092fae7d910
W20260811 18:57:22.409982 3824 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.012s (SENT)
W20260811 18:57:22.410643 3824 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 5.471s (SENT)
I20260811 18:57:22.412942 3824 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:22.413913 4952 raft_consensus.cc:2726] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:22.419685 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:22.427639 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.428550 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.429682 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1f60a286ac024c45bcb845227c5445c9 (DEFAULT_TABLE table=default.Table_7ddb21035302480fbfaf4738e6229f2b [id=466cb5f3f05048f3b2857ba8bf3426bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.430866 4162 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:22.433416 4162 ts_tablet_manager.cc:1397] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.463s user 0.002s sys 0.010s
I20260811 18:57:22.435292 4162 raft_consensus.cc:348] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.435788 4162 raft_consensus.cc:374] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.435977 4162 raft_consensus.cc:729] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.436372 4162 consensus_queue.cc:260] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.437480 4162 ts_tablet_manager.cc:1428] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:22.440332 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f60a286ac024c45bcb845227c5445c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:22.442273 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
I20260811 18:57:22.443634 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 (DEFAULT_TABLE table=default.Table_badc11f1b0554e629ddb3f8a20e151e2 [id=e0494fbbb30942c2aa3fa5f264a3f93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.444283 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4af28c051e8243eaa73d4a5ed1abcea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.446015 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 25266bc7788f49f8af8eb9a948e61c8d (DEFAULT_TABLE table=default.Table_30ac5178633e44aeafd707e48d7c4cba [id=31b3d877b5da424eb8f4b3fd7086cda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.447064 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 838a9f3025a1466296844c218cedf4fa (DEFAULT_TABLE table=default.Table_8c94c9b20d784df0811302a8381a7301 [id=0052bf55538f46c5b17831eaa1cfad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.447579 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.448071 3705 tablet_service.cc:1467] Processing CreateTablet for tablet ea4ad482bde44d6bab5ec809054aaebd (DEFAULT_TABLE table=default.Table_fc78208e540445719638da3757163df5 [id=37f03604188246b2aedfa959d6346824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.448783 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4ad482bde44d6bab5ec809054aaebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.449703 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838a9f3025a1466296844c218cedf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.450552 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25266bc7788f49f8af8eb9a948e61c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.452754 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 97997b0b3a5d46b487a9a33dcc30f9cd (DEFAULT_TABLE table=default.Table_ab9ea53bcc98448384f4fd3d30d2528f [id=e5b7ad7f9abb4ff190a1d4f32e468362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.453414 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97997b0b3a5d46b487a9a33dcc30f9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.454082 3709 tablet_service.cc:1467] Processing CreateTablet for tablet c2b283002d0e4316844e482468f97c31 (DEFAULT_TABLE table=default.Table_5fef93c0a9b546199a3217a8b8bd67dd [id=65c745bb0fdf4b85bf0e3451c8a180b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.454672 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 494939a4eb5a45d6a123d174ad513bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.457214 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_af28562bfbf94af98e28afef24fda608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.458179 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af28562bfbf94af98e28afef24fda608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.461603 4163 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.454926 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2b283002d0e4316844e482468f97c31. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:22.471258 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:22.471983 4162 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:22.472586 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7a06761b8a145519b7bb645e7a73657" } new_table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" modify_external_catalogs: true
I20260811 18:57:22.474908 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_52e514412ff342e3b34ba958e863d4ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.475736 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52e514412ff342e3b34ba958e863d4ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.493422 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.494884 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 07a78361bbc44fcf86985f9f20c5665b (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.472379 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 (DEFAULT_TABLE table=default.Table_badc11f1b0554e629ddb3f8a20e151e2 [id=e0494fbbb30942c2aa3fa5f264a3f93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.474946 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1f60a286ac024c45bcb845227c5445c9 (DEFAULT_TABLE table=default.Table_7ddb21035302480fbfaf4738e6229f2b [id=466cb5f3f05048f3b2857ba8bf3426bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.475319 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 25266bc7788f49f8af8eb9a948e61c8d (DEFAULT_TABLE table=default.Table_30ac5178633e44aeafd707e48d7c4cba [id=31b3d877b5da424eb8f4b3fd7086cda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.484699 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 838a9f3025a1466296844c218cedf4fa (DEFAULT_TABLE table=default.Table_8c94c9b20d784df0811302a8381a7301 [id=0052bf55538f46c5b17831eaa1cfad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.485082 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 494939a4eb5a45d6a123d174ad513bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.486285 3865 tablet_service.cc:1467] Processing CreateTablet for tablet ea4ad482bde44d6bab5ec809054aaebd (DEFAULT_TABLE table=default.Table_fc78208e540445719638da3757163df5 [id=37f03604188246b2aedfa959d6346824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.501452 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4af28c051e8243eaa73d4a5ed1abcea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.502318 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.508996 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a78361bbc44fcf86985f9f20c5665b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.511420 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25266bc7788f49f8af8eb9a948e61c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.513926 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838a9f3025a1466296844c218cedf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.514739 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f60a286ac024c45bcb845227c5445c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.513242 4986 consensus_queue.cc:1035] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260811 18:57:22.525353 3719 tablet_service.cc:1467] Processing CreateTablet for tablet d593aea288e14bb1a4c54360b260cffa (DEFAULT_TABLE table=default.Table_fd82f86a4579497fb4e50639ea1bf581 [id=20f5d41745b94d84a5796ce17ca05376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.526091 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d593aea288e14bb1a4c54360b260cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.526198 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 (DEFAULT_TABLE table=default.Table_ed61380b12e548ce843c5056342085b9 [id=b17ff3c80703482a9bd612ab9dacdd8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.529747 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f2f31331974f42c3a22b3ac21b690f82 (DEFAULT_TABLE table=default.Table_533afa7d5e834d5cb281ddc9619575f8 [id=c8a25582e429459eb7442d0517c93ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.531018 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f31331974f42c3a22b3ac21b690f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.534508 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e3a2005266804c6c94f0325437caa659" } new_table_name: "default.Table_9de9363efd764bd4ac323f86fd70658d" modify_external_catalogs: true
I20260811 18:57:22.534732 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4ad482bde44d6bab5ec809054aaebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.537076 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b7d4eb2831c407a989326cf56a7d8f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.538455 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b7d4eb2831c407a989326cf56a7d8f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.540904 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d18d57b020e046b8a069552ea63ae961"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.541797 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d18d57b020e046b8a069552ea63ae961 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.543820 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4fe6e07876144884a155b97fcbf9ed8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.544704 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fe6e07876144884a155b97fcbf9ed8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.548084 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:22.549309 3555 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0] alter complete (version 1)
I20260811 18:57:22.554701 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 45d6d76629d440129e4509fad38dde43 (DEFAULT_TABLE table=default.Table_99de6d48691743d798f56442d85d5be8 [id=0d95920bcace41f7873200170610c9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.557157 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 07a78361bbc44fcf86985f9f20c5665b (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.557142 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.558444 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_204c26eb8e554700a777796f2df5e7b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.559316 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_204c26eb8e554700a777796f2df5e7b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.562517 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b40da8cb7324da9804e6fdafd8e8266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.567406 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111),
I20260811 18:57:22.567716 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:22.569244 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8" } modify_external_catalogs: true
I20260811 18:57:22.569568 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8" } modify_external_catalogs: true
I20260811 18:57:22.571672 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_655b56a4862144f1afd52a620c3715d3" } new_table_name: "default.Table_609ebcc1b6e4405782a26228865b9460" modify_external_catalogs: true
I20260811 18:57:22.573261 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab47bc9432074c05b97dce613fcbcdae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.574002 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab47bc9432074c05b97dce613fcbcdae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.581557 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4dd203b7eb8643f493b4642c0396e258 has no consensus state
W20260811 18:57:22.582399 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ca801c79fc499ab50cfe76cdb03b00 has no consensus state
I20260811 18:57:22.584630 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_90566ec48eaf42c8be4c06681c3c0268" } modify_external_catalogs: true
I20260811 18:57:22.585047 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_90566ec48eaf42c8be4c06681c3c0268" } modify_external_catalogs: true
I20260811 18:57:22.587095 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.588030 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.591949 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1811e5f62a714f8388460dfcb1717e81 consensus state has no leader
I20260811 18:57:22.593472 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b20872d80c9545e5822bfc82d496ce97
I20260811 18:57:22.610100 4163 ts_tablet_manager.cc:1397] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.286s user 0.011s sys 0.004s
I20260811 18:57:22.617297 3860 tablet_service.cc:1467] Processing CreateTablet for tablet e32f59bdd0a54fcb9c76d7d9132d966c (DEFAULT_TABLE table=default.Table_9dd303f12b584056a04e385cc80faffe [id=25ed995924b2435daa8d514770e952f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.566406 3857 tablet_service.cc:1467] Processing CreateTablet for tablet f2f31331974f42c3a22b3ac21b690f82 (DEFAULT_TABLE table=default.Table_533afa7d5e834d5cb281ddc9619575f8 [id=c8a25582e429459eb7442d0517c93ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.618714 3555 catalog_manager.cc:5495] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.619377 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d6d76629d440129e4509fad38dde43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.620136 3555 catalog_manager.cc:5495] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.561483 3724 tablet_service.cc:1467] Processing CreateTablet for tablet e32f59bdd0a54fcb9c76d7d9132d966c (DEFAULT_TABLE table=default.Table_9dd303f12b584056a04e385cc80faffe [id=25ed995924b2435daa8d514770e952f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.568526 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f (DEFAULT_TABLE table=default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 [id=f1e29c96194e4ca0bc9b14c0f8811a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.569643 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9948c33ed58b4160932be82ef9615ba5 (DEFAULT_TABLE table=default.Table_8214722ffbdc49bf98d5aa539aebed52 [id=bc5efeb04bb64cad932828d746ef6a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.591912 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.596185 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.558146 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 (DEFAULT_TABLE table=default.Table_ed61380b12e548ce843c5056342085b9 [id=b17ff3c80703482a9bd612ab9dacdd8a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.630043 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 10988ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:22.631278 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e32f59bdd0a54fcb9c76d7d9132d966c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.639175 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.655861 3714 tablet_service.cc:1467] Processing CreateTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 (DEFAULT_TABLE table=default.Table_d8f42fcd065448e5a6332ff5512645ed [id=213a95b02ba64f3895f789564b1bc5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.671063 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 45d6d76629d440129e4509fad38dde43 (DEFAULT_TABLE table=default.Table_99de6d48691743d798f56442d85d5be8 [id=0d95920bcace41f7873200170610c9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.661737 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.676569 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d593aea288e14bb1a4c54360b260cffa (DEFAULT_TABLE table=default.Table_fd82f86a4579497fb4e50639ea1bf581 [id=20f5d41745b94d84a5796ce17ca05376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.684582 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a78361bbc44fcf86985f9f20c5665b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.688513 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d593aea288e14bb1a4c54360b260cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.694777 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f (DEFAULT_TABLE table=default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 [id=f1e29c96194e4ca0bc9b14c0f8811a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.693259 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 9948c33ed58b4160932be82ef9615ba5 (DEFAULT_TABLE table=default.Table_8214722ffbdc49bf98d5aa539aebed52 [id=bc5efeb04bb64cad932828d746ef6a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.708067 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6009c9f6bbb0493ebe125120a1a47f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.720587 4167 ts_tablet_manager.cc:1918] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:22.725498 4162 tablet_bootstrap.cc:654] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.726486 4994 consensus_queue.cc:1035] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260811 18:57:22.728178 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b40e44a2f4940af891eddf6c3b6293c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.738647 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e9da8dfb4a4c6c9275c092fae7d910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.703323 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.755769 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 84e152c74f33463e83843b92c0783803 (DEFAULT_TABLE table=default.Table_632b7ba5285744ee8f4bdc969820dae6 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.772378 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 consensus state has no leader
I20260811 18:57:22.785250 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9948c33ed58b4160932be82ef9615ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.795826 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e152c74f33463e83843b92c0783803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.796855 4167 log.cc:1196] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5ae200f3814e431a8288c30fe94bffcf
I20260811 18:57:22.803898 4169 ts_tablet_manager.cc:1918] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.804296 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6009c9f6bbb0493ebe125120a1a47f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.806321 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.812279 4163 raft_consensus.cc:348] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.812422 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" } modify_external_catalogs: true
I20260811 18:57:22.812808 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" } modify_external_catalogs: true
I20260811 18:57:22.815680 4169 log.cc:1196] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:22.819530 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.848317 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91d1c65c57764085a678b08ec4e1938b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.849229 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91d1c65c57764085a678b08ec4e1938b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.852560 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f31331974f42c3a22b3ac21b690f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.853116 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b4ff060dff9a4bf19670e304147315ac" } modify_external_catalogs: true
I20260811 18:57:22.853349 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b4ff060dff9a4bf19670e304147315ac" } modify_external_catalogs: true
I20260811 18:57:22.855033 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4fe6e07876144884a155b97fcbf9ed8d" } new_table_name: "default.Table_6914252b8edc40e79af29a5145201b08" modify_external_catalogs: true
I20260811 18:57:22.860067 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2108d07d209a411da92158a80c5d08ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.861200 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2108d07d209a411da92158a80c5d08ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.862978 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_886df95cce4342f998d27c8661df890e" } new_table_name: "default.Table_0a55918c905f4a4386f6529271fb60dc" modify_external_catalogs: true
I20260811 18:57:22.869374 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b40da8cb7324da9804e6fdafd8e8266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.874363 4994 consensus_queue.cc:1035] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260811 18:57:22.885382 4952 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.891321 4163 raft_consensus.cc:374] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.891268 4952 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.891850 4163 raft_consensus.cc:729] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.892204 4952 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:22.893306 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.893784 3676 consensus_peers.cc:487] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.900017 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d6d76629d440129e4509fad38dde43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.904083 4167 ts_tablet_manager.cc:1939] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:22.916716 3865 tablet_service.cc:1467] Processing CreateTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 (DEFAULT_TABLE table=default.Table_d8f42fcd065448e5a6332ff5512645ed [id=213a95b02ba64f3895f789564b1bc5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.941574 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e32f59bdd0a54fcb9c76d7d9132d966c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.943265 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.953635 4986 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.954298 4169 ts_tablet_manager.cc:1939] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.955518 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 389) took 4417 ms. Trace:
I20260811 18:57:22.955705 4169 rpcz_store.cc:276] 0811 18:57:18.538387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.540022 (+ 1635us) service_pool.cc:227] Handling call
0811 18:57:22.269812 (+3729790us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:22.269829 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.288932 (+ 19103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.709610 (+420678us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.955468 (+245858us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3717478}
I20260811 18:57:22.956228 4169 tablet_replica.cc:330] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:22.956553 4169 raft_consensus.cc:2229] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.956853 4169 raft_consensus.cc:2258] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.957789 4169 ts_tablet_manager.cc:1905] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.957581 4163 consensus_queue.cc:260] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.958631 4163 ts_tablet_manager.cc:1428] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.348s user 0.005s sys 0.001s
W20260811 18:57:22.969825 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260811 18:57:22.936164 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.972826 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e9da8dfb4a4c6c9275c092fae7d910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.985975 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 370) took 4644 ms. Trace:
W20260811 18:57:22.987315 3676 consensus_peers.cc:487] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e215759d48c24d8cb99bbc3fc9009a66. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.993732 4986 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.997413 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 84e152c74f33463e83843b92c0783803 (DEFAULT_TABLE table=default.Table_632b7ba5285744ee8f4bdc969820dae6 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.008214 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b40e44a2f4940af891eddf6c3b6293c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.011576 4167 rpcz_store.cc:276] 0811 18:57:18.264406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.264849 (+ 443us) service_pool.cc:227] Handling call
0811 18:57:22.343352 (+4078503us) ts_tablet_manager.cc:1145] Deleting tablet 5ae200f3814e431a8288c30fe94bffcf
0811 18:57:22.343370 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.470630 (+127260us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.668476 (+197846us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.909320 (+240844us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30158,"tablet-delete.queue_time_us":4078253}
I20260811 18:57:23.016940 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.019747 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08b93745a5f44ce8bc61f090e881f952" } modify_external_catalogs: true
I20260811 18:57:23.020084 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08b93745a5f44ce8bc61f090e881f952" } modify_external_catalogs: true
I20260811 18:57:23.023991 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_598311ac8416435286cfa4a6301380bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.025719 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc16d2310b294ccb8c18945b693c6fce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.026684 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4b6ec419ed7747c3b24dc11992518e88
W20260811 18:57:23.027453 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 335fc15ea75d483990e6ae5a5872682f has no consensus state
I20260811 18:57:23.028103 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d9b7a2cfa85347bf88241286c4bf0083" } new_table_name: "default.Table_d7ed0b81c09744e7af02ac80fb40508c" modify_external_catalogs: true
W20260811 18:57:23.030397 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa95131ce50943e3b0c6b5bdd5a30939 consensus state has no leader
I20260811 18:57:23.031169 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_df08bd98f2ef430eaf0e10876b9e265e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.032850 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1bf717c45ad424daa97f656cabd9db8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.037408 3890 raft_consensus.cc:2454] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:23.038048 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e152c74f33463e83843b92c0783803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.022276 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 21cc1822d4b94081ad86a6177acb3afe
W20260811 18:57:23.039948 3824 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.146s (SENT)
I20260811 18:57:23.041342 5037 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.043227 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d79d187445ee43d6864f03f905400405"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.044379 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d79d187445ee43d6864f03f905400405 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.047528 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_269ccd3c54914a34be3c79acea4eeed3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.048338 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_269ccd3c54914a34be3c79acea4eeed3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.049721 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1bf717c45ad424daa97f656cabd9db8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.050290 4163 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:23.050812 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc16d2310b294ccb8c18945b693c6fce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.041718 5037 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.054595 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28fea07a2f434dc1997642a34e6b6e2b" } new_table_name: "default.Table_db07259b8d8744d49c2c9fc89d4c5506" modify_external_catalogs: true
I20260811 18:57:23.055495 5037 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.056529 3824 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.056701 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df08bd98f2ef430eaf0e10876b9e265e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.057032 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 56770.8us
W20260811 18:57:23.061901 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_598311ac8416435286cfa4a6301380bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.065114 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9948c33ed58b4160932be82ef9615ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.070232 4169 ts_tablet_manager.cc:1918] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:23.077530 3824 leader_election.cc:334] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.438s (SENT)
I20260811 18:57:23.077855 3824 leader_election.cc:302] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:23.079185 5037 raft_consensus.cc:2726] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:23.081707 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 70983.9us
I20260811 18:57:23.084097 4167 tablet_replica.cc:330] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.084391 4167 raft_consensus.cc:2229] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.084607 4167 raft_consensus.cc:2258] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.085219 4167 ts_tablet_manager.cc:1905] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.105540 4169 log.cc:1196] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/33d995d72f8142afaeb57a2c29910fa3
W20260811 18:57:23.107264 3824 raft_consensus.cc:2670] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:23.112247 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 54878.8us
W20260811 18:57:23.112710 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT) [suppressed 35 similar messages]
W20260811 18:57:23.113122 3824 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.002s (SENT)
I20260811 18:57:23.120004 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc1822d4b94081ad86a6177acb3afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.123031 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.142915 4163 tablet_bootstrap.cc:654] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:23.146768 4986 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.147230 3676 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.144701 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:23.150411 4169 ts_tablet_manager.cc:1939] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:23.153740 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 386) took 4615 ms. Trace:
W20260811 18:57:23.154058 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9122ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:23.153995 4169 rpcz_store.cc:276] 0811 18:57:18.537795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539719 (+ 1924us) service_pool.cc:227] Handling call
0811 18:57:22.956124 (+4416405us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:22.956151 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.056324 (+100173us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.070077 (+ 13753us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.153377 (+ 83300us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4401603}
I20260811 18:57:23.154878 3744 raft_consensus.cc:2454] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:23.155999 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 33d995d72f8142afaeb57a2c29910fa3 (table default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]) successfully deleted
I20260811 18:57:23.158985 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:23.162864 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 113111us
I20260811 18:57:23.163964 4169 tablet_replica.cc:330] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:23.164353 4169 raft_consensus.cc:2229] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.165259 4169 raft_consensus.cc:2258] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.166153 4169 ts_tablet_manager.cc:1905] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.168143 3824 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.170444 5037 raft_consensus.cc:2781] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.170657 5037 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.172164 4952 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.172493 4952 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.174948 4952 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:23.175220 5037 raft_consensus.cc:3037] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.180444 4986 raft_consensus.cc:2781] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.184600 4986 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.184628 4162 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.188472 4986 raft_consensus.cc:3037] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.190075 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:23.190613 4162 ts_tablet_manager.cc:1397] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.747s user 0.021s sys 0.018s
W20260811 18:57:23.202183 3824 leader_election.cc:341] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:23.208544 3824 leader_election.cc:341] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:23.217131 4986 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.218995 4162 raft_consensus.cc:348] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.221333 5037 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.223425 4162 raft_consensus.cc:374] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.223733 4162 raft_consensus.cc:729] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:23.227653 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1800e9e2b064a6fb546b9d33cefa68b consensus state has no leader
I20260811 18:57:23.233817 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4efc43ae63254c67aac88d7bd9b6c6a2" } new_table_name: "default.Table_2e5ca40ed9be45ee85dd2681b4523778" modify_external_catalogs: true
I20260811 18:57:23.237129 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:23.237834 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a64df7c295c4d9baded1b63a9980581 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.241137 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd82f86a4579497fb4e50639ea1bf581" } modify_external_catalogs: true
I20260811 18:57:23.241684 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd82f86a4579497fb4e50639ea1bf581" } modify_external_catalogs: true
W20260811 18:57:23.244542 3678 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.999s (SENT)
W20260811 18:57:23.245492 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
W20260811 18:57:23.245843 3678 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.429s (SENT)
W20260811 18:57:23.246255 3678 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.014s (SENT)
W20260811 18:57:23.246640 3678 leader_election.cc:334] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.015s (SENT)
I20260811 18:57:23.246922 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:23.249568 3890 raft_consensus.cc:3037] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.253182 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc1822d4b94081ad86a6177acb3afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.253631 4986 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.254207 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.254524 4162 consensus_queue.cc:260] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.234658 5041 consensus_queue.cc:1035] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260811 18:57:23.263789 3890 raft_consensus.cc:2454] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:23.271030 5041 consensus_queue.cc:1035] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260811 18:57:23.271468 4162 ts_tablet_manager.cc:1428] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.070s user 0.000s sys 0.005s
I20260811 18:57:23.272850 4162 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:23.272796 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:23.273782 3744 raft_consensus.cc:3037] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.285765 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_313ce0e9dd1042e2827c0eebceaf3729"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.288879 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2103ce2744174eec9f603b49604123e4" } new_table_name: "default.Table_c8f7ec32b3654845877bb1de84a57f5f" modify_external_catalogs: true
I20260811 18:57:23.291292 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7075f060dbd942cca40c448af9eb5825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.292259 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2b67d7e59584a2bb877f79bdafed3a1" } new_table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" modify_external_catalogs: true
I20260811 18:57:23.294554 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.294313 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_de17e2e7bbbe429c9dea56a8b734f6b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.303570 5037 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:23.317373 5043 log.cc:924] Time spent T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260811 18:57:23.322865 4163 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:23.323261 4163 ts_tablet_manager.cc:1397] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.364s user 0.005s sys 0.008s
I20260811 18:57:23.325166 4163 raft_consensus.cc:348] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.325539 4163 raft_consensus.cc:374] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.325846 4163 raft_consensus.cc:729] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:23.326871 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.328583 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_313ce0e9dd1042e2827c0eebceaf3729 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.334511 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7075f060dbd942cca40c448af9eb5825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.339001 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:23.340958 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c7a814de93aa4d49a3ab5c7d2b167976"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.342034 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7a814de93aa4d49a3ab5c7d2b167976 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.344544 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5ae200f3814e431a8288c30fe94bffcf (table default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]) successfully deleted
I20260811 18:57:23.353336 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7ddb21035302480fbfaf4738e6229f2b" } modify_external_catalogs: true
I20260811 18:57:23.353755 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7ddb21035302480fbfaf4738e6229f2b" } modify_external_catalogs: true
I20260811 18:57:23.366887 4167 ts_tablet_manager.cc:1918] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:23.367165 4167 log.cc:1196] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:23.326205 4163 consensus_queue.cc:260] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.368067 4167 ts_tablet_manager.cc:1939] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:23.368677 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 371) took 5022 ms. Trace:
I20260811 18:57:23.368830 4167 rpcz_store.cc:276] 0811 18:57:18.346360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.348121 (+ 1761us) service_pool.cc:227] Handling call
0811 18:57:23.012530 (+4664409us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:23.012588 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.229722 (+217134us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.366576 (+136854us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.368632 (+ 2056us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4657527}
I20260811 18:57:23.370015 4167 tablet_replica.cc:330] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.370657 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a300a7aca2814da6b999eb4d865fea66 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:23.370754 4167 raft_consensus.cc:2229] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:23.371121 4167 raft_consensus.cc:2258] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.372146 4167 ts_tablet_manager.cc:1905] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.375403 4169 ts_tablet_manager.cc:1918] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:23.376653 4169 log.cc:1196] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/331396c9962f48ed9ea6dc66f1660902
I20260811 18:57:23.376888 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.379005 4162 tablet_bootstrap.cc:654] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:23.376359 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.379309 4163 ts_tablet_manager.cc:1428] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.052s user 0.004s sys 0.000s
I20260811 18:57:23.386860 4163 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:23.392939 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4059e5907b1e488ab9e1d04e563f96cd consensus state has no leader
I20260811 18:57:23.395211 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:23.396421 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" } modify_external_catalogs: true
I20260811 18:57:23.396742 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" } modify_external_catalogs: true
I20260811 18:57:23.417495 3744 raft_consensus.cc:2454] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:23.419411 3824 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.441s (SENT)
I20260811 18:57:23.423815 3824 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.424887 5037 raft_consensus.cc:2781] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.427224 5037 raft_consensus.cc:686] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:23.427737 5037 consensus_queue.cc:237] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:23.428874 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
W20260811 18:57:23.433243 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 178272us
W20260811 18:57:23.433844 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae7da0868b114052ba421801b76fb33c consensus state has no leader
I20260811 18:57:23.435194 3676 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.435971 5041 raft_consensus.cc:2781] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.436158 5041 raft_consensus.cc:686] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:23.436239 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_016d0800749f4083ad87c2a640e058c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.436589 5041 consensus_queue.cc:237] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:23.437247 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_016d0800749f4083ad87c2a640e058c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.444339 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet b3f0cdcee681460084662e01152ad9dd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.445220 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9693b748bd0490cb416e0cb36a62966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.445621 4162 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.445935 4162 ts_tablet_manager.cc:1397] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.174s user 0.009s sys 0.002s
W20260811 18:57:23.446022 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9693b748bd0490cb416e0cb36a62966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.447520 4162 raft_consensus.cc:348] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.447944 4162 raft_consensus.cc:374] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.448235 4162 raft_consensus.cc:729] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.449206 4162 consensus_queue.cc:260] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.450394 4162 ts_tablet_manager.cc:1428] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 18:57:23.451758 4162 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:23.454017 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_03949bb266e742cbbe858496f37177f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.455173 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_269ccd3c54914a34be3c79acea4eeed3" } new_table_name: "default.Table_4b41b319741e458abbf697fc780af7da" modify_external_catalogs: true
I20260811 18:57:23.456358 4162 tablet_bootstrap.cc:654] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:23.461269 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03949bb266e742cbbe858496f37177f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.498620 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3a85d24a6d8d4b038094e6e7607ddbd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.499656 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.502228 4169 ts_tablet_manager.cc:1939] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:23.502526 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_61f28b085f594e1eaf7d7eb8905a7ba2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.503489 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.505718 4163 tablet_bootstrap.cc:654] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:23.523239 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d11b889e9d044a049d0586382c263951 has no consensus state
I20260811 18:57:23.523269 4990 consensus_queue.cc:1035] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:23.523985 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_084495aab60944cda9e4a3f796132fa2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.524883 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_084495aab60944cda9e4a3f796132fa2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.527148 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8f7ec32b3654845877bb1de84a57f5f" } new_table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" modify_external_catalogs: true
I20260811 18:57:23.536916 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:23.520198 5048 consensus_queue.cc:1035] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260811 18:57:23.539427 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 387) took 4978 ms. Trace:
I20260811 18:57:23.540673 4169 rpcz_store.cc:276] 0811 18:57:18.537974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539755 (+ 1781us) service_pool.cc:227] Handling call
0811 18:57:23.154434 (+4614679us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:23.154449 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.332674 (+178225us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.359210 (+ 26536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.516112 (+156902us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4599887}
I20260811 18:57:23.531931 5052 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.546095 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 385) took 5008 ms. Trace:
I20260811 18:57:23.546159 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 331396c9962f48ed9ea6dc66f1660902 (table default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]) successfully deleted
I20260811 18:57:23.547070 4169 rpcz_store.cc:276] 0811 18:57:18.537581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539682 (+ 2101us) service_pool.cc:227] Handling call
0811 18:57:23.545782 (+5006100us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:23.545796 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.545927 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4972232}
I20260811 18:57:23.548347 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 397) took 4900 ms. Trace:
I20260811 18:57:23.548466 5052 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:23.549167 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8426ccbe530446fcbaa162a03cbb1892 because the tablet was not found. No further retry: Not found: Tablet not found: 8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:23.550566 5052 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.554203 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 335fc15ea75d483990e6ae5a5872682f (DEFAULT_TABLE table=default.Table_6914252b8edc40e79af29a5145201b08 [id=c3840f888f554af48ca56484f725a3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.554728 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 335fc15ea75d483990e6ae5a5872682f (DEFAULT_TABLE table=default.Table_6914252b8edc40e79af29a5145201b08 [id=c3840f888f554af48ca56484f725a3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.555049 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fc15ea75d483990e6ae5a5872682f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.555433 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fc15ea75d483990e6ae5a5872682f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.562233 5059 tablet.cc:1722] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:23.562867 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1" } modify_external_catalogs: true
I20260811 18:57:23.563230 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1" } modify_external_catalogs: true
I20260811 18:57:23.567646 4167 ts_tablet_manager.cc:1918] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:23.566569 4169 rpcz_store.cc:276] 0811 18:57:18.648065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.648160 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:23.548049 (+4899889us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:23.548062 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.548224 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4892991}
I20260811 18:57:23.568053 4167 log.cc:1196] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:23.569067 4167 ts_tablet_manager.cc:1939] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:23.569561 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 62c365f79d4a4201ac323f16bddf3e62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62c365f79d4a4201ac323f16bddf3e62 already in progress: creating tablet
I20260811 18:57:23.569941 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 382) took 5027 ms. Trace:
I20260811 18:57:23.570075 3864 tablet_service.cc:1467] Processing CreateTablet for tablet dbe6f14450cd43129a2387d9de67260a (DEFAULT_TABLE table=default.Table_8b7d4eb2831c407a989326cf56a7d8f9 [id=08ff7cf1de89473ba831e1ce57958694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.571033 4167 rpcz_store.cc:276] 0811 18:57:18.541912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.541995 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:23.369851 (+4827856us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:23.369867 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.507967 (+138100us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.567483 (+ 59516us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.569767 (+ 2284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4827152}
I20260811 18:57:23.571532 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbe6f14450cd43129a2387d9de67260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.573992 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 (DEFAULT_TABLE table=default.Table_52e514412ff342e3b34ba958e863d4ba [id=270f54fa01e044bfbb480b9776cd0b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.574571 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429eb5824f3f4b2b8da6e17ba7c69b92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.578135 3824 leader_election.cc:334] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.050s (SENT)
I20260811 18:57:23.571467 3714 tablet_service.cc:1467] Processing CreateTablet for tablet dbe6f14450cd43129a2387d9de67260a (DEFAULT_TABLE table=default.Table_8b7d4eb2831c407a989326cf56a7d8f9 [id=08ff7cf1de89473ba831e1ce57958694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.578469 3824 leader_election.cc:302] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.578805 3824 raft_consensus.cc:2670] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.579130 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbe6f14450cd43129a2387d9de67260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.580646 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 (DEFAULT_TABLE table=default.Table_52e514412ff342e3b34ba958e863d4ba [id=270f54fa01e044bfbb480b9776cd0b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.581393 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429eb5824f3f4b2b8da6e17ba7c69b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.582773 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f1a55a42d1e64f04ad7f433614a3a1aa (table default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]) successfully deleted
I20260811 18:57:23.595153 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.595552 3890 raft_consensus.cc:2454] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:23.599478 3824 leader_election.cc:334] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.480s (SENT)
W20260811 18:57:23.610904 3824 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.232s (SENT)
I20260811 18:57:23.611382 3824 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.611925 3824 raft_consensus.cc:2670] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.614530 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 6d28c486e408451a990ae3905ef3c2ff (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.615445 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d28c486e408451a990ae3905ef3c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.618561 3676 consensus_peers.cc:487] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33d995d72f8142afaeb57a2c29910fa3. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:23.629997 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 398) took 4965 ms. Trace:
I20260811 18:57:23.631318 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28b2e1b607a34ab2badc82e3c333a386" } new_table_name: "default.Table_c4384a8ba27a4a7696a6051325ddede2" modify_external_catalogs: true
I20260811 18:57:23.631281 4169 rpcz_store.cc:276] 0811 18:57:18.664239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.664303 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:23.570981 (+4906678us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:23.570993 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.629871 (+ 58878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4897385}
I20260811 18:57:23.632068 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 399) took 4927 ms. Trace:
I20260811 18:57:23.632109 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6d28c486e408451a990ae3905ef3c2ff (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.632364 4169 rpcz_store.cc:276] 0811 18:57:18.704762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.739715 (+ 34953us) service_pool.cc:227] Handling call
0811 18:57:23.631770 (+4892055us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:23.631787 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.631941 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4880127}
I20260811 18:57:23.632961 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d28c486e408451a990ae3905ef3c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.633021 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 400) took 4860 ms. Trace:
I20260811 18:57:23.633328 4169 rpcz_store.cc:276] 0811 18:57:18.772467 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.798089 (+ 25622us) service_pool.cc:227] Handling call
0811 18:57:23.632665 (+4834576us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:23.632678 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.632912 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4832278}
I20260811 18:57:23.633827 4169 tablet_replica.cc:330] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:23.634245 4169 raft_consensus.cc:2229] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.637929 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e6a28da9c6fa458d9daa95fdbd30920c" } new_table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" modify_external_catalogs: true
I20260811 18:57:23.638082 3676 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.640931 4986 raft_consensus.cc:2781] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.642753 4986 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.642988 4986 raft_consensus.cc:3037] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:23.644671 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c99ceca57cb245f4a3a6cb4b36f1306d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c99ceca57cb245f4a3a6cb4b36f1306d already in progress: creating tablet
W20260811 18:57:23.645361 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f7d4c26da86b49b8a37f2289f4ef6614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d4c26da86b49b8a37f2289f4ef6614 already in progress: creating tablet
W20260811 18:57:23.645726 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d00d943b778741b4aeec076ba46a9c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d00d943b778741b4aeec076ba46a9c40 already in progress: creating tablet
W20260811 18:57:23.646726 3676 consensus_peers.cc:487] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 331396c9962f48ed9ea6dc66f1660902. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:23.661069 3824 leader_election.cc:334] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.406s (SENT)
I20260811 18:57:23.662849 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 91d19b48908740128914fa5c231dfc04 (DEFAULT_TABLE table=default.Table_e863da40303e45bda8d9464f8ec9b8a0 [id=25706cf458f849498982901790caaaa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.663579 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91d19b48908740128914fa5c231dfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.670519 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 50af4e9b20a74ed983b26281c3c6eda1 (DEFAULT_TABLE table=default.Table_1204e0b30877462982c245f439f0bae4 [id=a260a40d963940198ab9f177c512f356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.671224 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50af4e9b20a74ed983b26281c3c6eda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.671748 5048 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.674314 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 50af4e9b20a74ed983b26281c3c6eda1 (DEFAULT_TABLE table=default.Table_1204e0b30877462982c245f439f0bae4 [id=a260a40d963940198ab9f177c512f356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.676842 5048 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.677634 4162 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.677742 5048 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.682986 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 384) took 4956 ms. Trace:
I20260811 18:57:23.684655 4167 rpcz_store.cc:276] 0811 18:57:18.617796 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.624796 (+ 7000us) service_pool.cc:227] Handling call
0811 18:57:23.573454 (+4948658us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:23.573474 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.573702 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4927387}
I20260811 18:57:23.684710 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 91d19b48908740128914fa5c231dfc04 (DEFAULT_TABLE table=default.Table_e863da40303e45bda8d9464f8ec9b8a0 [id=25706cf458f849498982901790caaaa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.685114 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 71a7eda2248c4f708bdc772ea3598dec (DEFAULT_TABLE table=default.Table_056c759cfe3543f7a43de5b2fa3a4160 [id=673bce4a4bcf4d05b1e83791d99fb0f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.685566 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50af4e9b20a74ed983b26281c3c6eda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.685886 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91d19b48908740128914fa5c231dfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.686272 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71a7eda2248c4f708bdc772ea3598dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.687433 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 385) took 5056 ms. Trace:
I20260811 18:57:23.687819 4167 rpcz_store.cc:276] 0811 18:57:18.631256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.644580 (+ 13324us) service_pool.cc:227] Handling call
0811 18:57:23.685638 (+5041058us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:23.685670 (+ 32us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.687233 (+ 1563us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5025357}
I20260811 18:57:23.688580 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 386) took 4984 ms. Trace:
I20260811 18:57:23.689049 4167 rpcz_store.cc:276] 0811 18:57:18.704084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.704164 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:23.688305 (+4984141us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:23.688317 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.688453 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4983802}
W20260811 18:57:23.694669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 62c365f79d4a4201ac323f16bddf3e62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62c365f79d4a4201ac323f16bddf3e62 already in progress: creating tablet
W20260811 18:57:23.695111 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c99ceca57cb245f4a3a6cb4b36f1306d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c99ceca57cb245f4a3a6cb4b36f1306d already in progress: creating tablet
W20260811 18:57:23.699846 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
I20260811 18:57:23.700554 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_00f2f8bec82f44f3a42420818d7dc6a0" } new_table_name: "default.Table_1074819ead5e452d8a479310453e4b9a" modify_external_catalogs: true
I20260811 18:57:23.703137 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a2ab815e5ecf491ab1ca7f1730ba148c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.703981 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2ab815e5ecf491ab1ca7f1730ba148c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.706084 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1f60a286ac024c45bcb845227c5445c9
I20260811 18:57:23.706691 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:23.707408 3554 catalog_manager.cc:6565] ReplaceTablet: tablet b3f0cdcee681460084662e01152ad9dd deleted and replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0
I20260811 18:57:23.708002 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.708703 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea1ee62f594e4ee798829d582bc2b53f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.710330 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d593aea288e14bb1a4c54360b260cffa
I20260811 18:57:23.711462 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_425203e0bd2944109d8f9ba2f0a7ba6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.712318 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_425203e0bd2944109d8f9ba2f0a7ba6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.713773 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:23.714617 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_69be92ae6ea74daf931e8ccd58a8dd4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.715312 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69be92ae6ea74daf931e8ccd58a8dd4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.716990 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5d495354c0994fb4a74c376078e41a26
I20260811 18:57:23.720752 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_061ae2e2037348cab0a2499a98d273a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.721613 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_061ae2e2037348cab0a2499a98d273a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.724061 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.725394 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.698419 4162 ts_tablet_manager.cc:1397] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.227s user 0.003s sys 0.015s
I20260811 18:57:23.699677 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 78d442dd6e29433ab006e1559c13d06c (DEFAULT_TABLE table=default.Table_a93638c401d94d3d8dad242dd4c9b60a [id=50af5dec6ee3414989f38b8d91db91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.696504 5061 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:23.725548 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.726435 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.728374 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:23.731389 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e8859199c3440ebb2b9664b65e0fb71" } new_table_name: "default.Table_4e412149fc55445bb720ffed59a05618" modify_external_catalogs: true
W20260811 18:57:23.735451 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f7d4c26da86b49b8a37f2289f4ef6614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d4c26da86b49b8a37f2289f4ef6614 already in progress: creating tablet
I20260811 18:57:23.736374 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d593aea288e14bb1a4c54360b260cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.740118 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f60a286ac024c45bcb845227c5445c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.743191 4986 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.744951 4986 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.720104 4169 raft_consensus.cc:2258] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.723758 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.741055 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 71a7eda2248c4f708bdc772ea3598dec (DEFAULT_TABLE table=default.Table_056c759cfe3543f7a43de5b2fa3a4160 [id=673bce4a4bcf4d05b1e83791d99fb0f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.751004 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71a7eda2248c4f708bdc772ea3598dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.745148 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet d593aea288e14bb1a4c54360b260cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.747781 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.747165 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 78d442dd6e29433ab006e1559c13d06c (DEFAULT_TABLE table=default.Table_a93638c401d94d3d8dad242dd4c9b60a [id=50af5dec6ee3414989f38b8d91db91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.754087 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:23.757107 3676 leader_election.cc:341] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:23.752365 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.762567 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.744746 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.745469 5062 tablet.cc:1722] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:23.764351 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 649cb98f553a418b911e307fdaf49917 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.765625 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f60a286ac024c45bcb845227c5445c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.766263 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d442dd6e29433ab006e1559c13d06c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.766635 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edd393d9c4d24aab8a170e1329400656 consensus state has no leader
I20260811 18:57:23.767511 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:23.768764 3887 raft_consensus.cc:3037] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.768776 3708 tablet_service.cc:1467] Processing CreateTablet for tablet ab03996c5c624e5eb17267b124e27f45 (DEFAULT_TABLE table=default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 [id=c89c9365aee249419be03a7bd8b091fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.769943 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
W20260811 18:57:23.770802 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e0747ef88574c68ae074732cb7095b9 consensus state has no leader
I20260811 18:57:23.771531 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.772430 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47326d9f3fac4cb7b76516843cf7bfd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.774112 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae7da0868b114052ba421801b76fb33c (table default.Table_32b9a3e653554115b17152dc38aa0439 [id=c417185e41c94dad8cc5e5f5937adb2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae7da0868b114052ba421801b76fb33c consensus state has no leader
I20260811 18:57:23.778985 4162 raft_consensus.cc:348] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.775082 5059 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38286 (request call id 542) took 1091 ms. Trace:
I20260811 18:57:23.782310 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d442dd6e29433ab006e1559c13d06c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.786782 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.787170 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ae02c9b0b066474894546829aed304e3 (DEFAULT_TABLE table=default.Table_204c26eb8e554700a777796f2df5e7b1 [id=137cef1d67054dc6be05fcef9b22c375]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:23.796768 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 16570ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:23.797506 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.798071 4169 ts_tablet_manager.cc:1905] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.799137 4167 tablet_replica.cc:330] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.800895 4162 raft_consensus.cc:374] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.801090 4162 raft_consensus.cc:729] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.801484 3709 tablet_service.cc:1467] Processing CreateTablet for tablet b16cb4c1796147fbb10109f52071505e (DEFAULT_TABLE table=default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d [id=35e57410bd7f4f1eba55e9abc899cc86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.801891 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649cb98f553a418b911e307fdaf49917. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.804129 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f287040b879642448367abe2b1c73cd9 consensus state has no leader
I20260811 18:57:23.803347 5059 rpcz_store.cc:276] 0811 18:57:22.635397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.635475 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:23.727069 (+1091594us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:22.672761 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:22.672924 (+ 163us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:22.672943 (+ 19us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:22.672953 (+ 10us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:22.673013 (+ 60us) alter_schema_op.cc:126] START. Timestamp: 1786474642672929
0811 18:57:22.673018 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:22.673414 (+ 396us) log.cc:842] Serialized 173 byte log entry
0811 18:57:23.475256 (+801842us) op_driver.cc:444] REPLICATION: finished
0811 18:57:23.561982 (+ 86726us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:23.563498 (+ 1516us) log.cc:842] Serialized 5 byte log entry
0811 18:57:23.720018 (+156520us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.726546 (+ 6528us) log.cc:842] Serialized 14 byte log entry
0811 18:57:23.727011 (+ 465us) alter_schema_op.cc:71] Released schema lock
0811 18:57:23.727021 (+ 10us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":86386,"prepare.queue_time_us":6946,"prepare.run_cpu_time_us":5142,"prepare.run_wall_time_us":82023,"raft.queue_time_us":129,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":346,"replication_time_us":802192,"thread_start_us":664721,"threads_started":3,"wal-append.queue_time_us":116536}]]}
I20260811 18:57:23.810170 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16cb4c1796147fbb10109f52071505e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.811296 4163 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:23.814308 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02c9b0b066474894546829aed304e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.821590 4163 ts_tablet_manager.cc:1397] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.431s user 0.004s sys 0.007s
I20260811 18:57:23.821939 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 649cb98f553a418b911e307fdaf49917 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.825492 3863 tablet_service.cc:1467] Processing CreateTablet for tablet ab03996c5c624e5eb17267b124e27f45 (DEFAULT_TABLE table=default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 [id=c89c9365aee249419be03a7bd8b091fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.825862 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 56d24aff9fd34f3eb05fdf942cc7efc7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.828541 4167 raft_consensus.cc:2229] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:23.834019 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_507c4f42c22847e39b4c700687a1687c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.835005 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_507c4f42c22847e39b4c700687a1687c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.837215 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:23.837962 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.838807 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91dfb6056e5049bb8cca8a06b3a33b7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.843928 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab03996c5c624e5eb17267b124e27f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.851441 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 56d24aff9fd34f3eb05fdf942cc7efc7
I20260811 18:57:23.851991 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 56d24aff9fd34f3eb05fdf942cc7efc7 deleted and replaced by tablet 19d7716739f848d78b9cbbd3975f3d44
I20260811 18:57:23.853453 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a (DEFAULT_TABLE table=default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.855436 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.860581 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 133efffbd3d74cdca61efd8ad0225b41 (DEFAULT_TABLE table=default.Table_d18d57b020e046b8a069552ea63ae961 [id=ed42157134b94a488999a64b5f6b92ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.870934 5072 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:23.851918 4162 consensus_queue.cc:260] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.886518 3887 raft_consensus.cc:2454] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:23.886936 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 133efffbd3d74cdca61efd8ad0225b41 (DEFAULT_TABLE table=default.Table_d18d57b020e046b8a069552ea63ae961 [id=ed42157134b94a488999a64b5f6b92ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.888576 3860 tablet_service.cc:1467] Processing CreateTablet for tablet ae02c9b0b066474894546829aed304e3 (DEFAULT_TABLE table=default.Table_204c26eb8e554700a777796f2df5e7b1 [id=137cef1d67054dc6be05fcef9b22c375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.889775 3856 tablet_service.cc:1467] Processing CreateTablet for tablet bef0f6d56cda40249f12b25a8eb3ab6b (DEFAULT_TABLE table=default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.892341 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b16cb4c1796147fbb10109f52071505e (DEFAULT_TABLE table=default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d [id=35e57410bd7f4f1eba55e9abc899cc86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.892735 3676 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.897114 4162 ts_tablet_manager.cc:1428] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.145s user 0.004s sys 0.000s
I20260811 18:57:23.900486 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133efffbd3d74cdca61efd8ad0225b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.904501 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 844bcd16f06c4788af9c0d7d320464aa (DEFAULT_TABLE table=default.Table_e8554df7eb5f4fbbbe828834c1d45f25 [id=00e172dcefa0491f9f287857d60ba3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.905812 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649cb98f553a418b911e307fdaf49917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.905849 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e4e304a679345cd9289252278ddb281"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.906930 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e4e304a679345cd9289252278ddb281 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.907931 4167 raft_consensus.cc:2258] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.909795 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab03996c5c624e5eb17267b124e27f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.913936 4163 raft_consensus.cc:348] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.914817 4163 raft_consensus.cc:374] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.914978 4163 raft_consensus.cc:729] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.916828 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6317dbacc890428caf5adac65ad7895d" } modify_external_catalogs: true
I20260811 18:57:23.917060 4163 consensus_queue.cc:260] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.917250 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6317dbacc890428caf5adac65ad7895d" } modify_external_catalogs: true
W20260811 18:57:23.917425 3678 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT)
I20260811 18:57:23.917939 4163 ts_tablet_manager.cc:1428] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.096s user 0.004s sys 0.000s
I20260811 18:57:23.917910 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd203b7eb8643f493b4642c0396e258 (DEFAULT_TABLE table=default.Table_f7fd9b1d34c3439c997e0341237d5a82 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.921886 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09" } modify_external_catalogs: true
I20260811 18:57:23.922217 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef0f6d56cda40249f12b25a8eb3ab6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.922308 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09" } modify_external_catalogs: true
I20260811 18:57:23.924433 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_425203e0bd2944109d8f9ba2f0a7ba6d" } new_table_name: "default.Table_71aa50e0de9f470d9c9cceeb3b18cd90" modify_external_catalogs: true
I20260811 18:57:23.924819 5051 raft_consensus.cc:2781] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.925181 5051 raft_consensus.cc:686] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:23.925839 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133efffbd3d74cdca61efd8ad0225b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.927124 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02c9b0b066474894546829aed304e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.898762 3723 tablet_service.cc:1467] Processing CreateTablet for tablet bef0f6d56cda40249f12b25a8eb3ab6b (DEFAULT_TABLE table=default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.928206 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16cb4c1796147fbb10109f52071505e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.929428 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24955fa6843f46dc883be60173049a16" } modify_external_catalogs: true
I20260811 18:57:23.929749 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24955fa6843f46dc883be60173049a16" } modify_external_catalogs: true
I20260811 18:57:23.931586 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d5db1f0452e40eea01177ba93223341" } modify_external_catalogs: true
I20260811 18:57:23.931864 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d5db1f0452e40eea01177ba93223341" } modify_external_catalogs: true
I20260811 18:57:23.935477 4167 ts_tablet_manager.cc:1905] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.938772 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 705605cd929940f8a252b6e95abbeb95 (DEFAULT_TABLE table=default.Table_db899a033c3144559d1d5c7bf1d92fbf [id=5593f1d77ce94befac70ddabfc422ba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.941453 4163 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:23.943341 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 (DEFAULT_TABLE table=default.Table_8dbfb0ccfdef4c018c7eca1d0452375a [id=15fde9b467f04384aa3a2ea431ca352b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.933897 5051 consensus_queue.cc:237] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.947141 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 844bcd16f06c4788af9c0d7d320464aa (DEFAULT_TABLE table=default.Table_e8554df7eb5f4fbbbe828834c1d45f25 [id=00e172dcefa0491f9f287857d60ba3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.947966 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844bcd16f06c4788af9c0d7d320464aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.948779 3824 leader_election.cc:334] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.068s (SENT)
I20260811 18:57:23.949064 3824 leader_election.cc:302] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:23.965447 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 19d7716739f848d78b9cbbd3975f3d44) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.968080 5037 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.968407 5037 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.969038 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd203b7eb8643f493b4642c0396e258 (DEFAULT_TABLE table=default.Table_f7fd9b1d34c3439c997e0341237d5a82 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.969678 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd203b7eb8643f493b4642c0396e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.970749 3872 tablet_service.cc:1467] Processing CreateTablet for tablet f8ca801c79fc499ab50cfe76cdb03b00 (DEFAULT_TABLE table=default.Table_9de9363efd764bd4ac323f86fd70658d [id=7c19e7030e324615ab16dd34a65a2ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.971261 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ca801c79fc499ab50cfe76cdb03b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.975543 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 19d7716739f848d78b9cbbd3975f3d44) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.977188 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 705605cd929940f8a252b6e95abbeb95 (DEFAULT_TABLE table=default.Table_db899a033c3144559d1d5c7bf1d92fbf [id=5593f1d77ce94befac70ddabfc422ba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.977972 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705605cd929940f8a252b6e95abbeb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.980849 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 (DEFAULT_TABLE table=default.Table_8dbfb0ccfdef4c018c7eca1d0452375a [id=15fde9b467f04384aa3a2ea431ca352b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.980873 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 86d690ec09524248bc147ad7408e2ee6 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.981976 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 (DEFAULT_TABLE table=default.Table_ed948e1e9ee14cffae82bcd5517709f8 [id=4275d081d66941b9b9f0a927e7a19eea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:23.982069 3824 raft_consensus.cc:2670] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.982441 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.983625 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa (DEFAULT_TABLE table=default.Table_853572d26c9642b9b8ad616f574301dc [id=cecb2799de544cb392a84211ab0d55d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.993201 5037 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:24.003268 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.004962 4162 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.014982 5048 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.015404 5048 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.017042 5048 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.017581 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e556811ad1643f1baa3bbf4edebd157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.019181 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a (DEFAULT_TABLE table=default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.021219 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f8ca801c79fc499ab50cfe76cdb03b00 (DEFAULT_TABLE table=default.Table_9de9363efd764bd4ac323f86fd70658d [id=7c19e7030e324615ab16dd34a65a2ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.027926 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844bcd16f06c4788af9c0d7d320464aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.028369 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa (DEFAULT_TABLE table=default.Table_853572d26c9642b9b8ad616f574301dc [id=cecb2799de544cb392a84211ab0d55d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.029731 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 (DEFAULT_TABLE table=default.Table_ed948e1e9ee14cffae82bcd5517709f8 [id=4275d081d66941b9b9f0a927e7a19eea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.019429 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d690ec09524248bc147ad7408e2ee6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.022250 3824 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT)
I20260811 18:57:24.022714 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:24.019512 4163 tablet_bootstrap.cc:654] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.035017 3887 raft_consensus.cc:2454] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
#21 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
#22 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
I20260811 18:57:24.030401 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 86d690ec09524248bc147ad7408e2ee6 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.047590 3676 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.048076 5048 raft_consensus.cc:2781] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:24.050132 5048 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:24.050443 5048 raft_consensus.cc:3037] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.051397 3824 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.056273 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.061767 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68808.4us
W20260811 18:57:24.062173 3824 leader_election.cc:334] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.008s (SENT)
I20260811 18:57:24.071333 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:24.071786 3744 raft_consensus.cc:2454] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:24.072433 3824 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.805s (SENT)
I20260811 18:57:24.072763 3824 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.073065 5037 raft_consensus.cc:2726] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:24.073410 3678 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.367s (SENT)
I20260811 18:57:24.076977 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ca801c79fc499ab50cfe76cdb03b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.082751 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.087888 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d690ec09524248bc147ad7408e2ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.055529 5048 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.093858 3824 raft_consensus.cc:2670] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.102583 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705605cd929940f8a252b6e95abbeb95. 1 dirs total, 0 dirs full, 0 dirs failed
#23 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#24 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
W20260811 18:57:24.114430 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46001.9us
#25 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:24.120321 4169 ts_tablet_manager.cc:1918] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:24.123493 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef0f6d56cda40249f12b25a8eb3ab6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.138459 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd203b7eb8643f493b4642c0396e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.142634 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.145408 5048 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.154731 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e556811ad1643f1baa3bbf4edebd157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.179795 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:24.180212 3890 raft_consensus.cc:3037] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.186015 3824 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.187256 5037 raft_consensus.cc:2781] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:24.187489 5037 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:24.187719 5037 raft_consensus.cc:3037] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.198593 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed61380b12e548ce843c5056342085b9" } modify_external_catalogs: true
I20260811 18:57:24.198979 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed61380b12e548ce843c5056342085b9" } modify_external_catalogs: true
I20260811 18:57:24.199896 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_204c26eb8e554700a777796f2df5e7b1" } new_table_name: "default.Table_f6953ae59bab4944a8db87e863735f79" modify_external_catalogs: true
I20260811 18:57:24.203100 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db899a033c3144559d1d5c7bf1d92fbf" } new_table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" modify_external_catalogs: true
W20260811 18:57:24.230533 3678 leader_election.cc:334] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 6.262s (SENT)
I20260811 18:57:24.235040 4162 tablet_bootstrap.cc:654] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.237990 4986 consensus_queue.cc:1035] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260811 18:57:24.240342 3678 leader_election.cc:302] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.255725 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 53891.6us
W20260811 18:57:24.262226 3678 raft_consensus.cc:2670] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.269781 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 53188.7us
I20260811 18:57:24.280339 4163 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:24.280582 4163 ts_tablet_manager.cc:1397] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.362s user 0.004s sys 0.004s
I20260811 18:57:24.289487 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:24.289101 5037 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.301808 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7057b8a7bae249caaed304b5c66e6871
I20260811 18:57:24.303061 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0da5d975ff884b9bb82d239828666af7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.303923 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0da5d975ff884b9bb82d239828666af7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.309926 4163 raft_consensus.cc:348] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:24.310261 4163 raft_consensus.cc:374] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.310150 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:24.310401 4163 raft_consensus.cc:729] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.310465 3744 raft_consensus.cc:3037] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.310793 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 14e34732bda44670ae9da8facc405221
W20260811 18:57:24.310789 3824 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.598s (SENT)
I20260811 18:57:24.311790 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0036c28f59cb4f229da243389602bbd9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.312829 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0036c28f59cb4f229da243389602bbd9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.314566 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 219474fc820d40f49810209ac1c40afe has no consensus state
I20260811 18:57:24.314124 4163 consensus_queue.cc:260] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:24.315009 3890 raft_consensus.cc:2454] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:24.315255 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a65337aa3b94589a51dd41812870aca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.315706 3676 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:24.316001 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a65337aa3b94589a51dd41812870aca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.316505 4986 raft_consensus.cc:2781] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:24.317734 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
I20260811 18:57:24.318749 4986 raft_consensus.cc:686] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:24.318698 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bc171757977f4404bb736a03a87b9ecd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.319123 4986 consensus_queue.cc:237] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.319468 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc171757977f4404bb736a03a87b9ecd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.320904 3824 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.320930 3744 raft_consensus.cc:2454] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:24.321223 3824 raft_consensus.cc:2670] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.321862 3678 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.022s (SENT)
I20260811 18:57:24.322103 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 64b922f2a71544e8a18533aa0a1ccb52
I20260811 18:57:24.322734 3824 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.323385 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.324227 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6ed856dda814eb0bd8eb76c2f83c70b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.324414 3678 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.818s (SENT)
I20260811 18:57:24.325706 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd] alter complete (version 2)
I20260811 18:57:24.326690 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.327564 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.329375 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 705605cd929940f8a252b6e95abbeb95 consensus state has no leader
I20260811 18:57:24.329944 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d1c65c57764085a678b08ec4e1938b" } new_table_name: "default.Table_24cdaf08a18b43e3bf356bb8b04f5017" modify_external_catalogs: true
I20260811 18:57:24.340543 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddcc89fb831040ed9752150dc2ad4b51" } new_table_name: "default.Table_9b7ce6401d2c44b9937b7f442ccbd56d" modify_external_catalogs: true
I20260811 18:57:24.341779 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.342640 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.346796 4167 ts_tablet_manager.cc:1918] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.347262 4167 log.cc:1196] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:24.348335 4167 ts_tablet_manager.cc:1939] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.349295 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 387) took 5595 ms. Trace:
I20260811 18:57:24.323276 5037 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:24.350361 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:24.354892 5037 raft_consensus.cc:2781] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:24.370350 5037 raft_consensus.cc:686] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:24.351842 4162 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.371822 4162 ts_tablet_manager.cc:1397] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.474s user 0.006s sys 0.010s
I20260811 18:57:24.372118 5037 consensus_queue.cc:237] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.373806 4162 raft_consensus.cc:348] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.374285 4162 raft_consensus.cc:374] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.374776 4162 raft_consensus.cc:729] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.375444 4162 consensus_queue.cc:260] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.377140 4169 log.cc:1196] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a443fe209556489abb47f4faa5d696e5
I20260811 18:57:24.379642 4169 ts_tablet_manager.cc:1939] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:24.380388 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 401) took 5590 ms. Trace:
I20260811 18:57:24.358955 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e34732bda44670ae9da8facc405221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.362470 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.381253 4167 rpcz_store.cc:276] 0811 18:57:18.753875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.753950 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:23.727721 (+4973771us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:23.727737 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.240708 (+512971us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.346587 (+105879us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.349243 (+ 2656us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4973160}
I20260811 18:57:24.387840 4162 ts_tablet_manager.cc:1428] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260811 18:57:24.390607 4162 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.388924 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 388) took 5618 ms. Trace:
I20260811 18:57:24.392478 4167 rpcz_store.cc:276] 0811 18:57:18.770410 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.770487 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:24.388540 (+5618053us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:24.388562 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.388774 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5617703}
I20260811 18:57:24.393592 4167 tablet_replica.cc:330] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:24.393703 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d00d943b778741b4aeec076ba46a9c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d00d943b778741b4aeec076ba46a9c40 already in progress: creating tablet
I20260811 18:57:24.397061 4167 raft_consensus.cc:2229] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.399226 4167 raft_consensus.cc:2258] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.405963 4167 ts_tablet_manager.cc:1905] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.418445 4162 tablet_bootstrap.cc:654] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.416026 4169 rpcz_store.cc:276] 0811 18:57:18.789601 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.807351 (+ 17750us) service_pool.cc:227] Handling call
0811 18:57:23.633664 (+4826313us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:23.633676 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.833566 (+199890us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.120110 (+286544us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.380320 (+260210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4813011}
I20260811 18:57:24.450486 4169 tablet_replica.cc:330] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:24.452919 4169 raft_consensus.cc:2229] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:24.456672 4169 raft_consensus.cc:2258] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.457707 4169 ts_tablet_manager.cc:1905] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:24.463194 3678 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.936s (SENT)
I20260811 18:57:24.466076 4162 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.467408 4162 ts_tablet_manager.cc:1397] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.000s
I20260811 18:57:24.475271 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 3263da75f0d84528b7c8cc55dcaeaa46 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.476353 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8d7695d0859045409244bdd77ddd3931"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.477241 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d7695d0859045409244bdd77ddd3931 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.479215 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:24.482249 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c6e0d1eeb63493bbea4cd78dd085a3c from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:24.485603 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffb196eaa8c2483aa71418747adda95a has no consensus state
I20260811 18:57:24.488124 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2bd2a5750a454b88b5bff7b802541c89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.488960 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bd2a5750a454b88b5bff7b802541c89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.489099 4162 raft_consensus.cc:348] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.489670 4162 raft_consensus.cc:374] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.489850 4162 raft_consensus.cc:729] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.490828 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c4130777df64341a269a7954794efd2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.491746 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c4130777df64341a269a7954794efd2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.494040 4162 consensus_queue.cc:260] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.495335 4162 ts_tablet_manager.cc:1428] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:24.495901 4162 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.497817 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0322ec9fd9f24338bb9196b099f56643"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.498610 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0322ec9fd9f24338bb9196b099f56643 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.506059 4167 ts_tablet_manager.cc:1918] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.507120 4167 log.cc:1196] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f25ff8a9e3a944269e9799995c73ebe0
#26 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:24.510157 4167 ts_tablet_manager.cc:1939] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.520504 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 389) took 5742 ms. Trace:
I20260811 18:57:24.526002 4167 rpcz_store.cc:276] 0811 18:57:18.775166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.775403 (+ 237us) service_pool.cc:227] Handling call
0811 18:57:24.393409 (+5618006us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:24.393425 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.440558 (+ 47133us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.504526 (+ 63968us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.517596 (+ 13070us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5617663}
I20260811 18:57:24.532207 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 390) took 5743 ms. Trace:
I20260811 18:57:24.533941 4167 rpcz_store.cc:276] 0811 18:57:18.788588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.788653 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:24.530156 (+5741503us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:24.530178 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.532134 (+ 1956us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5741224}
I20260811 18:57:24.537583 4167 tablet_replica.cc:330] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.538250 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f25ff8a9e3a944269e9799995c73ebe0 (table default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]) successfully deleted
W20260811 18:57:24.538785 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a443fe209556489abb47f4faa5d696e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a443fe209556489abb47f4faa5d696e5 already in progress: creating tablet
I20260811 18:57:24.539345 4167 raft_consensus.cc:2229] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.539816 4167 raft_consensus.cc:2258] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.540827 4167 ts_tablet_manager.cc:1905] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.529095 4162 tablet_bootstrap.cc:654] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.542938 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b40e44a2f4940af891eddf6c3b6293c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.555732 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee" } modify_external_catalogs: true
I20260811 18:57:24.556349 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee" } modify_external_catalogs: true
I20260811 18:57:24.557189 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea0cc7a576d64d1c934364316cae3c70" } modify_external_catalogs: true
I20260811 18:57:24.557552 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea0cc7a576d64d1c934364316cae3c70" } modify_external_catalogs: true
I20260811 18:57:24.558393 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_632b7ba5285744ee8f4bdc969820dae6" } new_table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" modify_external_catalogs: true
I20260811 18:57:24.560575 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e75a55498d4c4072968ee03aaf48b831" } new_table_name: "default.Table_7cabaea271024c3daf3ffc96c5825f1e" modify_external_catalogs: true
I20260811 18:57:24.564586 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b2efb13eaf1468c92c8030667191b84" } new_table_name: "default.Table_34d2346219e14bd58aa02228ad109763" modify_external_catalogs: true
I20260811 18:57:24.566756 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" } modify_external_catalogs: true
I20260811 18:57:24.567185 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" } modify_external_catalogs: true
I20260811 18:57:24.568264 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a33a6c24205421f9fc488087672c6e0" } modify_external_catalogs: true
I20260811 18:57:24.568598 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a33a6c24205421f9fc488087672c6e0" } modify_external_catalogs: true
W20260811 18:57:24.456357 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 129574us
I20260811 18:57:24.584354 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.589056 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e34732bda44670ae9da8facc405221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.589993 4162 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.590276 4162 ts_tablet_manager.cc:1397] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.095s user 0.005s sys 0.004s
I20260811 18:57:24.592965 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.597210 4162 raft_consensus.cc:348] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.597535 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a443fe209556489abb47f4faa5d696e5 (table default.Table_bf28eaf7393d4d508363fbc8dd91686b [id=1a0803466858425cadc33f51f0933a95]) successfully deleted
I20260811 18:57:24.597674 4162 raft_consensus.cc:374] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.597898 4162 raft_consensus.cc:729] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:24.600040 3824 leader_election.cc:334] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.744s (SENT)
I20260811 18:57:24.600302 3824 leader_election.cc:302] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.601029 3824 raft_consensus.cc:2670] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.602990 3824 leader_election.cc:334] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.866s (SENT)
I20260811 18:57:24.603415 3824 leader_election.cc:302] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.603787 3824 raft_consensus.cc:2670] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.604832 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:24.605235 3824 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.275s (SENT)
I20260811 18:57:24.605531 3824 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.606324 5080 raft_consensus.cc:2726] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:24.608431 3824 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.469s (SENT)
I20260811 18:57:24.608700 3824 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.609081 3824 raft_consensus.cc:2670] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.610193 3824 leader_election.cc:334] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.219s (SENT)
I20260811 18:57:24.610616 3824 leader_election.cc:302] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.611001 3824 raft_consensus.cc:2670] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.611495 4162 consensus_queue.cc:260] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.618466 3824 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.386s (SENT)
I20260811 18:57:24.618979 3824 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.619619 5080 raft_consensus.cc:2726] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:24.620155 4162 ts_tablet_manager.cc:1428] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.028s user 0.001s sys 0.003s
I20260811 18:57:24.620772 4162 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.624878 4162 tablet_bootstrap.cc:654] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:24.632856 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84e152c74f33463e83843b92c0783803 consensus state has no leader
W20260811 18:57:24.633586 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
W20260811 18:57:24.634361 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4c7f500a5444bae9a015a5e903c0622 consensus state has no leader
I20260811 18:57:24.636521 4169 ts_tablet_manager.cc:1918] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.636947 4169 log.cc:1196] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/405773849fb94800be61ad8a7a6055a4
I20260811 18:57:24.639906 4169 ts_tablet_manager.cc:1939] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:24.640558 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 403) took 5716 ms. Trace:
I20260811 18:57:24.665561 4986 raft_consensus.cc:482] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.665889 4986 raft_consensus.cc:504] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.668322 4167 ts_tablet_manager.cc:1918] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.668694 4167 log.cc:1196] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d9421bc220774f66b90bf537a47f1e13
I20260811 18:57:24.669412 4167 ts_tablet_manager.cc:1939] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.675680 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 395) took 5740 ms. Trace:
I20260811 18:57:24.677517 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7e8b32170ff47eda203b84328064301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.678375 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7e8b32170ff47eda203b84328064301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.679387 4986 leader_election.cc:288] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.680115 4986 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.680787 4986 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.681689 4986 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.683406 4167 rpcz_store.cc:276] 0811 18:57:18.935117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.935416 (+ 299us) service_pool.cc:227] Handling call
0811 18:57:24.537454 (+5602038us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:24.537474 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.572436 (+ 34962us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.642530 (+ 70094us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.675615 (+ 33085us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5601762}
I20260811 18:57:24.685784 4167 tablet_replica.cc:330] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.685945 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d9421bc220774f66b90bf537a47f1e13 (table default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]) successfully deleted
I20260811 18:57:24.686622 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b654b59f369e429097b918236d8ddd08" } new_table_name: "default.Table_9194817120074797a6f60bf8e24c0636" modify_external_catalogs: true
I20260811 18:57:24.687809 4167 raft_consensus.cc:2229] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.688391 4167 raft_consensus.cc:2258] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.694308 3550 catalog_manager.cc:5495] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:24.698926 4167 ts_tablet_manager.cc:1905] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.668375 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2247194db704f9abd54a8a0df1ca566" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:24.683266 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:24.723671 3676 leader_election.cc:341] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:24.724375 3676 leader_election.cc:341] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:24.724560 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8] alter complete (version 1)
I20260811 18:57:24.725972 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5580d47029684e6a8836966a2eee6853"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.727290 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5580d47029684e6a8836966a2eee6853 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.739465 4162 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.739792 4162 ts_tablet_manager.cc:1397] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.000s sys 0.008s
I20260811 18:57:24.741246 4162 raft_consensus.cc:348] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.741696 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 99b788d7e78c4945bb82fe86597aa634 (DEFAULT_TABLE table=default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a [id=2bc11a14c61347d5bee3feefee097dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.742477 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b788d7e78c4945bb82fe86597aa634. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.744107 3678 leader_election.cc:334] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:24.744511 3678 leader_election.cc:302] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.744832 3678 raft_consensus.cc:2670] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.703991 4163 ts_tablet_manager.cc:1428] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.423s user 0.009s sys 0.013s
I20260811 18:57:24.745451 4163 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:24.741696 4162 raft_consensus.cc:374] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.746361 4162 raft_consensus.cc:729] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.746906 4162 consensus_queue.cc:260] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.749222 4162 ts_tablet_manager.cc:1428] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.003s
I20260811 18:57:24.751175 4162 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.752005 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 (DEFAULT_TABLE table=default.Table_69be92ae6ea74daf931e8ccd58a8dd4f [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.752815 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.711834 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17395ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:24.759400 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 (DEFAULT_TABLE table=default.Table_47326d9f3fac4cb7b76516843cf7bfd0 [id=33ec227ce4d54151bd3b046a7637b160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.759969 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2474ae972e9242b69f488b7a2fdbc481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.769438 4162 tablet_bootstrap.cc:654] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.769582 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0b40da8cb7324da9804e6fdafd8e8266
I20260811 18:57:24.770818 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 7be6d9578e7847c2822278486119390c (DEFAULT_TABLE table=default.Table_ea1ee62f594e4ee798829d582bc2b53f [id=62fea03d455b41d69f9e01c6da7f74fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.771898 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be6d9578e7847c2822278486119390c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.776978 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.777463 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 156781us
W20260811 18:57:24.778173 3824 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.135s (SENT)
I20260811 18:57:24.778730 3824 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.779109 3824 raft_consensus.cc:2670] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.780543 3824 leader_election.cc:334] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.388s (SENT)
I20260811 18:57:24.780781 3824 leader_election.cc:302] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.781785 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 19d7716739f848d78b9cbbd3975f3d44
I20260811 18:57:24.783689 4169 rpcz_store.cc:276] 0811 18:57:18.923927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.977560 (+ 53633us) service_pool.cc:227] Handling call
0811 18:57:24.420645 (+5443085us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:24.420659 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.491698 (+ 71039us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.636303 (+144605us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.640510 (+ 4207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5442379}
I20260811 18:57:24.784251 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.784538 3824 raft_consensus.cc:2670] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.787027 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2d0b59cafb154416952aecaa418e861f
W20260811 18:57:24.790515 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ffb196eaa8c2483aa71418747adda95a (table default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ffb196eaa8c2483aa71418747adda95a consensus state has no leader
I20260811 18:57:24.791738 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0b59cafb154416952aecaa418e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.796197 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb3a9fdc3d594c1a84a5d8efa8490dfa consensus state has no leader
I20260811 18:57:24.797303 4169 tablet_replica.cc:330] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:24.797653 4169 raft_consensus.cc:2229] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:24.797923 4169 raft_consensus.cc:2258] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.800194 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3b40e44a2f4940af891eddf6c3b6293c
I20260811 18:57:24.800733 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 3b40e44a2f4940af891eddf6c3b6293c deleted and replaced by tablet 4da53cef719842f98d2190e487e8d653
I20260811 18:57:24.801088 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 7645a242bc4e4845ad49bfbe20b252f2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.803422 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 92e49846ab714dd294acdd09d49a83bd
I20260811 18:57:24.805277 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.806543 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 99b788d7e78c4945bb82fe86597aa634 (DEFAULT_TABLE table=default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a [id=2bc11a14c61347d5bee3feefee097dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.807631 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:24.807891 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 (DEFAULT_TABLE table=default.Table_69be92ae6ea74daf931e8ccd58a8dd4f [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.808800 4167 ts_tablet_manager.cc:1918] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.808617 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3baa014716584cb184d8a0e4299a8542"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.809523 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3baa014716584cb184d8a0e4299a8542 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.810822 4167 log.cc:1196] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:24.811322 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.811338 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.811604 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c with delete_type TABLET_DATA_DELETED (replaced by tablet 4da53cef719842f98d2190e487e8d653) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.812069 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14b83cad2c2f4713b46f4d1fa3782f90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.812844 4167 ts_tablet_manager.cc:1939] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.815063 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 396) took 5878 ms. Trace:
I20260811 18:57:24.814973 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.815738 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.816385 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e49846ab714dd294acdd09d49a83bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.815199 4167 rpcz_store.cc:276] 0811 18:57:18.935352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.935739 (+ 387us) service_pool.cc:227] Handling call
0811 18:57:24.685525 (+5749786us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:24.685541 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.756663 (+ 71122us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.808548 (+ 51885us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.813355 (+ 4807us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5749599}
I20260811 18:57:24.820698 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.821203 5048 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:24.821593 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.821467 5048 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.822634 5048 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.823609 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_59f9d96102f349498c7f854e48e3d7c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.824237 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:24.824544 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59f9d96102f349498c7f854e48e3d7c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.824983 3676 leader_election.cc:341] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:24.826512 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 403) took 5777 ms. Trace:
I20260811 18:57:24.826738 4167 rpcz_store.cc:276] 0811 18:57:19.048915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.050433 (+ 1518us) service_pool.cc:227] Handling call
0811 18:57:24.826114 (+5775681us) ts_tablet_manager.cc:1145] Deleting tablet 0540e4e43ea34fbfbc66b6035c60684d
0811 18:57:24.826138 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.826289 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5754448}
I20260811 18:57:24.827018 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5fef93c0a9b546199a3217a8b8bd67dd" } new_table_name: "default.Table_a4daf4372b7a44babebcceb2ded88ce5" modify_external_catalogs: true
I20260811 18:57:24.814144 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e8da32e196c4b85943b62957165d308"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.830740 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 7be6d9578e7847c2822278486119390c (DEFAULT_TABLE table=default.Table_ea1ee62f594e4ee798829d582bc2b53f [id=62fea03d455b41d69f9e01c6da7f74fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.831364 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.832188 3549 catalog_manager.cc:5495] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:24.832669 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c with delete_type TABLET_DATA_DELETED (replaced by tablet 4da53cef719842f98d2190e487e8d653) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:24.834255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0540e4e43ea34fbfbc66b6035c60684d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0540e4e43ea34fbfbc66b6035c60684d already in progress: creating tablet
I20260811 18:57:24.834204 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.834587 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:24.834942 4167 tablet_replica.cc:330] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.835361 4167 raft_consensus.cc:2229] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.835879 4167 raft_consensus.cc:2258] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:24.835927 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e8da32e196c4b85943b62957165d308 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.839462 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:24.839915 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 (DEFAULT_TABLE table=default.Table_47326d9f3fac4cb7b76516843cf7bfd0 [id=33ec227ce4d54151bd3b046a7637b160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.840307 4167 ts_tablet_manager.cc:1905] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.840744 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.841363 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0b59cafb154416952aecaa418e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842005 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e49846ab714dd294acdd09d49a83bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842447 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842988 5037 consensus_queue.cc:1035] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260811 18:57:24.843772 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b788d7e78c4945bb82fe86597aa634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.798810 4169 ts_tablet_manager.cc:1905] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.845090 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7f980c9b98994253b590548d86f6d145"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.845799 4163 tablet_bootstrap.cc:654] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:24.846505 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f980c9b98994253b590548d86f6d145 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.850486 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.854662 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be6d9578e7847c2822278486119390c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.855559 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.856287 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.856545 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.858078 3676 consensus_peers.cc:487] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:24.858242 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2474ae972e9242b69f488b7a2fdbc481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.868125 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 1e65c28c0c0b4ae39b8438b4592e777b (DEFAULT_TABLE table=default.Table_a2ab815e5ecf491ab1ca7f1730ba148c [id=4d0cf7429dc94ee2ae7da259499680eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.868948 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e65c28c0c0b4ae39b8438b4592e777b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.902742 3720 tablet_service.cc:1467] Processing CreateTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.904753 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.915068 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.915810 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e120a45539f49bb82ab0fb95eac36f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.917829 4167 ts_tablet_manager.cc:1918] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.918076 4167 log.cc:1196] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:24.919030 4167 ts_tablet_manager.cc:1939] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.921872 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 404) took 5846 ms. Trace:
I20260811 18:57:24.922077 4167 rpcz_store.cc:276] 0811 18:57:19.075439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.075512 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:24.834674 (+5759162us) ts_tablet_manager.cc:1145] Deleting tablet 3744b3d608754de68c4c57b072d1dc15
0811 18:57:24.834690 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.890193 (+ 55503us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.917583 (+ 27390us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.921820 (+ 4237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5758808}
I20260811 18:57:24.923022 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 405) took 5833 ms. Trace:
I20260811 18:57:24.923269 4167 rpcz_store.cc:276] 0811 18:57:19.089852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.091907 (+ 2055us) service_pool.cc:227] Handling call
0811 18:57:24.922662 (+5830755us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:24.922676 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.922887 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5830472}
I20260811 18:57:24.923647 4167 tablet_replica.cc:330] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.924074 4167 raft_consensus.cc:2229] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.924592 4167 raft_consensus.cc:2258] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.924998 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a4328549b8d4ad08d0135327263c811"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.925503 4167 ts_tablet_manager.cc:1905] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:24.925822 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a4328549b8d4ad08d0135327263c811 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.927475 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 27cda53b7ca54346a7a22bf26f5db990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27cda53b7ca54346a7a22bf26f5db990 already in progress: creating tablet
W20260811 18:57:24.929901 3676 consensus_peers.cc:487] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 405773849fb94800be61ad8a7a6055a4. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:24.934046 4162 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.934274 4162 ts_tablet_manager.cc:1397] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.183s user 0.007s sys 0.011s
I20260811 18:57:24.935489 4162 raft_consensus.cc:348] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.936133 4162 raft_consensus.cc:374] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.936420 4162 raft_consensus.cc:729] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.936834 4162 consensus_queue.cc:260] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.938015 4162 ts_tablet_manager.cc:1428] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:24.940471 4162 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.944043 4162 tablet_bootstrap.cc:654] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.949179 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca (DEFAULT_TABLE table=default.Table_03949bb266e742cbbe858496f37177f3 [id=73c7aa5e86eb4c468f6c967b2ac84f9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.949976 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.953562 3711 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4cc4dace944be894fe12690b18ccb (DEFAULT_TABLE table=default.Table_016d0800749f4083ad87c2a640e058c8 [id=c42c64b3e30043eca22cf0aebdf71d0c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:24.953930 3824 leader_election.cc:334] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.131s (SENT)
I20260811 18:57:24.954244 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4cc4dace944be894fe12690b18ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.954109 3824 leader_election.cc:302] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.961906 3824 raft_consensus.cc:2670] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.962877 3824 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.227s (SENT)
I20260811 18:57:24.963039 3824 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.963213 3824 raft_consensus.cc:2670] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.963939 3824 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.258s (SENT)
I20260811 18:57:24.964095 3824 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.964332 3824 raft_consensus.cc:2670] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.967619 3678 leader_election.cc:334] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.049s (SENT)
I20260811 18:57:24.967809 3678 leader_election.cc:302] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.968060 3678 raft_consensus.cc:2670] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.969203 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 (DEFAULT_TABLE table=default.Table_f9693b748bd0490cb416e0cb36a62966 [id=4702ae470cf0451fa2fb675855aa6d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.969890 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37be4f7579fa4ab99ddbacfa80282b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.976966 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ae96e10a2ed4407c9d42a359d82d389e (DEFAULT_TABLE table=default.Table_dc16d2310b294ccb8c18945b693c6fce [id=b28c814fee894ee7a12d05872eea8e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.977598 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae96e10a2ed4407c9d42a359d82d389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.988941 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7bc942d59fd34beda3b8fb9e03e84cb2" } new_table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" modify_external_catalogs: true
W20260811 18:57:24.994097 3678 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.115s (SENT)
I20260811 18:57:24.990023 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61f28b085f594e1eaf7d7eb8905a7ba2" } new_table_name: "default.Table_3e8780219db840aeb566b570452e221b" modify_external_catalogs: true
I20260811 18:57:24.990909 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_609ebcc1b6e4405782a26228865b9460" } new_table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" modify_external_catalogs: true
W20260811 18:57:24.991501 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2b283002d0e4316844e482468f97c31 consensus state has no leader
I20260811 18:57:24.997946 3724 tablet_service.cc:1467] Processing CreateTablet for tablet be9288a76eab4606ba189d7de5813973 (DEFAULT_TABLE table=default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 [id=25867a79fa014eea81b48c95c9570ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.999465 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9288a76eab4606ba189d7de5813973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.011689 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b88a886800d34db0a94ec1fa36757f90 (DEFAULT_TABLE table=default.Table_2108d07d209a411da92158a80c5d08ab [id=6057586eaff54a40a6b1a4f50adf9cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.012494 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88a886800d34db0a94ec1fa36757f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.013417 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_919f7bdbd76f4707813e30d37dfc3767" } modify_external_catalogs: true
I20260811 18:57:25.013870 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_919f7bdbd76f4707813e30d37dfc3767" } modify_external_catalogs: true
I20260811 18:57:25.018965 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" } modify_external_catalogs: true
I20260811 18:57:25.019299 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" } modify_external_catalogs: true
I20260811 18:57:25.021122 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5580d47029684e6a8836966a2eee6853" } modify_external_catalogs: true
I20260811 18:57:25.021524 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5580d47029684e6a8836966a2eee6853" } modify_external_catalogs: true
I20260811 18:57:25.028950 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab47bc9432074c05b97dce613fcbcdae" } new_table_name: "default.Table_540b89ef88904bb6b163c4b18480746e" modify_external_catalogs: true
I20260811 18:57:25.030284 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a0c48e6299546a2921a7a7db03f1032" } modify_external_catalogs: true
I20260811 18:57:25.030793 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a0c48e6299546a2921a7a7db03f1032" } modify_external_catalogs: true
I20260811 18:57:25.032855 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_821a790ed161401bb9758d71a0760916" } modify_external_catalogs: true
I20260811 18:57:25.033110 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_821a790ed161401bb9758d71a0760916" } modify_external_catalogs: true
W20260811 18:57:25.051580 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260811 18:57:25.060714 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ffb196eaa8c2483aa71418747adda95a (DEFAULT_TABLE table=default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.061384 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb196eaa8c2483aa71418747adda95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.008134 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 1e65c28c0c0b4ae39b8438b4592e777b (DEFAULT_TABLE table=default.Table_a2ab815e5ecf491ab1ca7f1730ba148c [id=4d0cf7429dc94ee2ae7da259499680eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.062160 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e65c28c0c0b4ae39b8438b4592e777b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.069067 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8776bb47c4cb891ef13a2a1e36781 (DEFAULT_TABLE table=default.Table_d79d187445ee43d6864f03f905400405 [id=5c5f51b92b6242a3a9cb07db2e8b7daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.022881 4163 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.071064 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8776bb47c4cb891ef13a2a1e36781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.072609 4163 ts_tablet_manager.cc:1397] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.327s user 0.009s sys 0.000s
I20260811 18:57:25.073086 5091 consensus_queue.cc:1035] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:25.027243 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.074913 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.027825 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.078845 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e120a45539f49bb82ab0fb95eac36f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.028280 3854 tablet_service.cc:1467] Processing CreateTablet for tablet be9288a76eab4606ba189d7de5813973 (DEFAULT_TABLE table=default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 [id=25867a79fa014eea81b48c95c9570ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.082895 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9288a76eab4606ba189d7de5813973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.029700 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca (DEFAULT_TABLE table=default.Table_03949bb266e742cbbe858496f37177f3 [id=73c7aa5e86eb4c468f6c967b2ac84f9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.084657 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.086930 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 3195082643264e0d90f70e1007ab9b24 (DEFAULT_TABLE table=default.Table_7075f060dbd942cca40c448af9eb5825 [id=40f69765767d4b23889527bc635f88d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.030217 3856 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4cc4dace944be894fe12690b18ccb (DEFAULT_TABLE table=default.Table_016d0800749f4083ad87c2a640e058c8 [id=c42c64b3e30043eca22cf0aebdf71d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.090323 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4cc4dace944be894fe12690b18ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.030766 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 (DEFAULT_TABLE table=default.Table_f9693b748bd0490cb416e0cb36a62966 [id=4702ae470cf0451fa2fb675855aa6d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.094851 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37be4f7579fa4ab99ddbacfa80282b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.031379 3861 tablet_service.cc:1467] Processing CreateTablet for tablet ae96e10a2ed4407c9d42a359d82d389e (DEFAULT_TABLE table=default.Table_dc16d2310b294ccb8c18945b693c6fce [id=b28c814fee894ee7a12d05872eea8e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.031903 3862 tablet_service.cc:1467] Processing CreateTablet for tablet b88a886800d34db0a94ec1fa36757f90 (DEFAULT_TABLE table=default.Table_2108d07d209a411da92158a80c5d08ab [id=6057586eaff54a40a6b1a4f50adf9cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.099466 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3195082643264e0d90f70e1007ab9b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.101473 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5a184b83373a4c089f2ca0169cbfcf66 (DEFAULT_TABLE table=default.Table_c7a814de93aa4d49a3ab5c7d2b167976 [id=ac945f840b754de791860a2415523df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.101841 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae96e10a2ed4407c9d42a359d82d389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.102393 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88a886800d34db0a94ec1fa36757f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.106592 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a184b83373a4c089f2ca0169cbfcf66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.106573 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.110623 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 810b9bdc330e40619f45c70e57724da0 (DEFAULT_TABLE table=default.Table_313ce0e9dd1042e2827c0eebceaf3729 [id=60db200412ce438194eda77eca5f7257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.117470 4162 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.120872 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d7716739f848d78b9cbbd3975f3d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.123520 3724 tablet_service.cc:1467] Processing CreateTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 (DEFAULT_TABLE table=default.Table_a1bf717c45ad424daa97f656cabd9db8 [id=6d01a59c66564bec9f1714a7a92c87d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.118373 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 219474fc820d40f49810209ac1c40afe (DEFAULT_TABLE table=default.Table_71aa50e0de9f470d9c9cceeb3b18cd90 [id=092f82e43fce4506883f0c8471022cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.132864 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1896700e724204a37439f134a4d988 (DEFAULT_TABLE table=default.Table_061ae2e2037348cab0a2499a98d273a6 [id=d3d44035163a4320bc0b5b79a4354fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.143640 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef (DEFAULT_TABLE table=default.Table_598311ac8416435286cfa4a6301380bd [id=cc4eea8d0a0c454ebb9e93d8ae619066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.143498 4162 ts_tablet_manager.cc:1397] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.006s
I20260811 18:57:25.143675 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810b9bdc330e40619f45c70e57724da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.143723 4167 ts_tablet_manager.cc:1918] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.143592 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ca82f01a33564be489962ef522cc6ed5 (DEFAULT_TABLE table=default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 [id=3c7a820c8c3944dd9d0cd6c8ed701f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.074203 4163 raft_consensus.cc:348] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.147630 4163 raft_consensus.cc:374] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.147787 4163 raft_consensus.cc:729] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.151211 4163 consensus_queue.cc:260] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.154806 4163 ts_tablet_manager.cc:1428] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.082s user 0.002s sys 0.000s
I20260811 18:57:25.155498 3714 tablet_service.cc:1467] Processing CreateTablet for tablet d11b889e9d044a049d0586382c263951 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.156041 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c74abe9dcf642e9b4d40df6ff3639ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.157711 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 125348us
I20260811 18:57:25.161823 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 (DEFAULT_TABLE table=default.Table_a1bf717c45ad424daa97f656cabd9db8 [id=6d01a59c66564bec9f1714a7a92c87d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.162061 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ffb196eaa8c2483aa71418747adda95a (DEFAULT_TABLE table=default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.162328 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21eb02db3de4c97878fa8badccaa6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.165256 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11b889e9d044a049d0586382c263951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.166173 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219474fc820d40f49810209ac1c40afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.162760 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.166498 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb196eaa8c2483aa71418747adda95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.166779 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d7716739f848d78b9cbbd3975f3d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.167340 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1896700e724204a37439f134a4d988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.167834 4167 log.cc:1196] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/331396c9962f48ed9ea6dc66f1660902
I20260811 18:57:25.168484 4167 ts_tablet_manager.cc:1939] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.170306 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 406) took 6079 ms. Trace:
I20260811 18:57:25.163166 3865 tablet_service.cc:1467] Processing CreateTablet for tablet ca82f01a33564be489962ef522cc6ed5 (DEFAULT_TABLE table=default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 [id=3c7a820c8c3944dd9d0cd6c8ed701f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.170884 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca82f01a33564be489962ef522cc6ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.170713 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21eb02db3de4c97878fa8badccaa6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.171839 4162 raft_consensus.cc:348] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.175153 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef (DEFAULT_TABLE table=default.Table_598311ac8416435286cfa4a6301380bd [id=cc4eea8d0a0c454ebb9e93d8ae619066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.175650 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c74abe9dcf642e9b4d40df6ff3639ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.178799 3870 tablet_service.cc:1467] Processing CreateTablet for tablet d11b889e9d044a049d0586382c263951 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.179198 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11b889e9d044a049d0586382c263951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.184165 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 39bfbcfc74834cbd9f4363e8c545d5a8 (DEFAULT_TABLE table=default.Table_ab47bc9432074c05b97dce613fcbcdae [id=3ecc967d0758417cbc1be9002c49d3f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.185226 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bfbcfc74834cbd9f4363e8c545d5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.172212 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca82f01a33564be489962ef522cc6ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.197212 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 39bfbcfc74834cbd9f4363e8c545d5a8 (DEFAULT_TABLE table=default.Table_ab47bc9432074c05b97dce613fcbcdae [id=3ecc967d0758417cbc1be9002c49d3f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.197839 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bfbcfc74834cbd9f4363e8c545d5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.198696 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ca37dd77c42a19146a99452ff87b0 (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.199256 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ca37dd77c42a19146a99452ff87b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.170946 4167 rpcz_store.cc:276] 0811 18:57:19.090018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.092235 (+ 2217us) service_pool.cc:227] Handling call
0811 18:57:24.923523 (+5831288us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:24.923536 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.047717 (+124181us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.143119 (+ 95402us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.169187 (+ 26068us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5831020}
I20260811 18:57:25.205626 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 407) took 6090 ms. Trace:
I20260811 18:57:25.205847 4167 rpcz_store.cc:276] 0811 18:57:19.114792 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.114857 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:25.205104 (+6090247us) ts_tablet_manager.cc:1145] Deleting tablet a0116997a1c64e658045c319f1c45861
0811 18:57:25.205116 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.205426 (+ 310us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6090001}
I20260811 18:57:25.206704 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 408) took 6055 ms. Trace:
I20260811 18:57:25.206915 4167 rpcz_store.cc:276] 0811 18:57:19.151498 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.151928 (+ 430us) service_pool.cc:227] Handling call
0811 18:57:25.206312 (+6054384us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:25.206323 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.206493 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6054128}
I20260811 18:57:25.207482 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 420) took 5659 ms. Trace:
I20260811 18:57:25.207702 4167 rpcz_store.cc:276] 0811 18:57:19.547527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.547609 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:25.207189 (+5659580us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:25.207199 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.207314 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5646190}
I20260811 18:57:25.208552 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 331396c9962f48ed9ea6dc66f1660902 (table default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]) successfully deleted
I20260811 18:57:25.172533 4162 raft_consensus.cc:374] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:25.209010 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a0116997a1c64e658045c319f1c45861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0116997a1c64e658045c319f1c45861 already in progress: creating tablet
W20260811 18:57:25.209389 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ce94d37efd2245bf8cc71fdd90bd9735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce94d37efd2245bf8cc71fdd90bd9735 already in progress: creating tablet
I20260811 18:57:25.214599 4167 tablet_replica.cc:330] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.215000 4167 raft_consensus.cc:2229] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.218410 3678 leader_election.cc:334] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.466s (SENT)
I20260811 18:57:25.218673 3678 leader_election.cc:302] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.220942 3678 raft_consensus.cc:2670] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.221974 5037 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.222167 5037 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:25.223119 5037 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:25.223449 4167 raft_consensus.cc:2258] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.224370 4162 raft_consensus.cc:729] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.226832 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 (DEFAULT_TABLE table=default.Table_df08bd98f2ef430eaf0e10876b9e265e [id=ec725d0625094fc994ee36547333988d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.227882 4167 ts_tablet_manager.cc:1905] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.228789 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb831bb10f47749a2cfd3c88cf3785 (DEFAULT_TABLE table=default.Table_507c4f42c22847e39b4c700687a1687c [id=9648b8cbb8ad49669020fc67e803c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.230820 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2f5ce2b1d14a9a988cff644e196088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.239274 4162 consensus_queue.cc:260] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.242458 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 810b9bdc330e40619f45c70e57724da0 (DEFAULT_TABLE table=default.Table_313ce0e9dd1042e2827c0eebceaf3729 [id=60db200412ce438194eda77eca5f7257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.244415 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810b9bdc330e40619f45c70e57724da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.243260 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 5a184b83373a4c089f2ca0169cbfcf66 (DEFAULT_TABLE table=default.Table_c7a814de93aa4d49a3ab5c7d2b167976 [id=ac945f840b754de791860a2415523df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.246829 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a184b83373a4c089f2ca0169cbfcf66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.243589 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 219474fc820d40f49810209ac1c40afe (DEFAULT_TABLE table=default.Table_71aa50e0de9f470d9c9cceeb3b18cd90 [id=092f82e43fce4506883f0c8471022cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.251423 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219474fc820d40f49810209ac1c40afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.242964 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 3195082643264e0d90f70e1007ab9b24 (DEFAULT_TABLE table=default.Table_7075f060dbd942cca40c448af9eb5825 [id=40f69765767d4b23889527bc635f88d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.252516 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3195082643264e0d90f70e1007ab9b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.244966 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8776bb47c4cb891ef13a2a1e36781 (DEFAULT_TABLE table=default.Table_d79d187445ee43d6864f03f905400405 [id=5c5f51b92b6242a3a9cb07db2e8b7daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.254925 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8776bb47c4cb891ef13a2a1e36781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.259577 4162 ts_tablet_manager.cc:1428] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.107s user 0.002s sys 0.002s
I20260811 18:57:25.243991 4163 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:24.976162 4169 ts_tablet_manager.cc:1918] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.261334 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce521c08754bf391215ab98230fb9d (DEFAULT_TABLE table=default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 [id=d3b4765ced4341a2b9944a39cf19019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.257833 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1896700e724204a37439f134a4d988 (DEFAULT_TABLE table=default.Table_061ae2e2037348cab0a2499a98d273a6 [id=d3d44035163a4320bc0b5b79a4354fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.261690 4169 log.cc:1196] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e7259cf84f5c4c89a1b4510f743c2bc5
I20260811 18:57:25.268878 4169 ts_tablet_manager.cc:1939] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.270115 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1896700e724204a37439f134a4d988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.283921 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 402) took 6360 ms. Trace:
I20260811 18:57:25.284428 4169 rpcz_store.cc:276] 0811 18:57:18.923768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.996147 (+ 72379us) service_pool.cc:227] Handling call
0811 18:57:24.784116 (+5787969us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:24.797034 (+ 12918us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.972019 (+174985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.975986 (+ 3967us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.283890 (+307904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5787643}
I20260811 18:57:25.285118 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 414) took 6234 ms. Trace:
I20260811 18:57:25.262411 4162 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.256119 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b7b48a35acf1486bbb63680ca5da31dd (DEFAULT_TABLE table=default.Table_91dfb6056e5049bb8cca8a06b3a33b7f [id=68958402d14841a494933aa943a07ec0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:25.273311 3678 leader_election.cc:334] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.619s (SENT)
I20260811 18:57:25.285275 4169 rpcz_store.cc:276] 0811 18:57:19.050383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.084405 (+ 34022us) service_pool.cc:227] Handling call
0811 18:57:25.284953 (+6200548us) ts_tablet_manager.cc:1145] Deleting tablet 0540e4e43ea34fbfbc66b6035c60684d
0811 18:57:25.284966 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.285096 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6173607}
I20260811 18:57:25.285491 3678 leader_election.cc:302] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.285738 3678 raft_consensus.cc:2670] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.285750 4169 tablet_replica.cc:330] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.287209 4163 tablet_bootstrap.cc:654] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.283738 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb831bb10f47749a2cfd3c88cf3785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.289793 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cb3c2f76034640b398a94e7536595277"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.291200 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb3c2f76034640b398a94e7536595277 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.294112 4169 raft_consensus.cc:2229] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.294380 4169 raft_consensus.cc:2258] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.298976 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:25.300848 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b48a35acf1486bbb63680ca5da31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.307811 4162 tablet_bootstrap.cc:654] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.307757 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e7259cf84f5c4c89a1b4510f743c2bc5 (table default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]) successfully deleted
W20260811 18:57:25.308302 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0540e4e43ea34fbfbc66b6035c60684d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0540e4e43ea34fbfbc66b6035c60684d already in progress: creating tablet
I20260811 18:57:25.313279 4167 ts_tablet_manager.cc:1918] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:25.313503 4167 log.cc:1196] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.313848 5048 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.313985 4167 ts_tablet_manager.cc:1939] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.314010 5048 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.314706 5048 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.316565 4169 ts_tablet_manager.cc:1905] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.320616 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f389d2b7ff354ac4bab5d68b9ff201c6 consensus state has no leader
W20260811 18:57:25.320475 3824 leader_election.cc:341] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.325280 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
I20260811 18:57:25.326295 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 8304ac7709ca4392baec4cfa78ef5b41 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:25.327014 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:25.327734 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_75b5605dc2a14c9daf6d68e7094bcb39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.328745 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75b5605dc2a14c9daf6d68e7094bcb39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.329684 4162 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.329854 4162 ts_tablet_manager.cc:1397] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.070s user 0.001s sys 0.005s
I20260811 18:57:25.326534 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:25.330861 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1dd4434500a64abcb81ee6eca6398ab8
W20260811 18:57:25.330873 3678 leader_election.cc:334] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.280s (SENT)
I20260811 18:57:25.331104 3678 leader_election.cc:302] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.331347 3678 raft_consensus.cc:2670] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:25.331414 3676 leader_election.cc:341] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.331579 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b012268c924347d8a87ea52350010dc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.332463 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b012268c924347d8a87ea52350010dc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.334453 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
I20260811 18:57:25.335381 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b08e57f933b24cd0aee53441350e6775"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.336102 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b08e57f933b24cd0aee53441350e6775 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.337525 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 (DEFAULT_TABLE table=default.Table_df08bd98f2ef430eaf0e10876b9e265e [id=ec725d0625094fc994ee36547333988d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.337785 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 380c1f53b3ac4509a09b25a2800028af
I20260811 18:57:25.338644 4162 raft_consensus.cc:348] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.338729 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_614252f40e34465080bb490a1a7050ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.338971 4162 raft_consensus.cc:374] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.339037 4162 raft_consensus.cc:729] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:25.339385 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_614252f40e34465080bb490a1a7050ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.340034 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 421) took 5768 ms. Trace:
I20260811 18:57:25.340152 4167 rpcz_store.cc:276] 0811 18:57:19.571590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.571721 (+ 131us) service_pool.cc:227] Handling call
0811 18:57:25.208022 (+5636301us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:25.208033 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.267672 (+ 59639us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.313157 (+ 45485us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.340007 (+ 26850us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3427,"tablet-delete.queue_time_us":5635734}
I20260811 18:57:25.340898 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:25.341837 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be655558b83041fcace57520a599b8a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.342799 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:25.344055 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2f5ce2b1d14a9a988cff644e196088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.343981 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.344779 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.344935 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 422) took 5753 ms. Trace:
I20260811 18:57:25.345795 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb831bb10f47749a2cfd3c88cf3785 (DEFAULT_TABLE table=default.Table_507c4f42c22847e39b4c700687a1687c [id=9648b8cbb8ad49669020fc67e803c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.346187 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb831bb10f47749a2cfd3c88cf3785. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.346289 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1811e5f62a714f8388460dfcb1717e81 consensus state has no leader
I20260811 18:57:25.346890 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e8da32e196c4b85943b62957165d308" } new_table_name: "default.Table_51f392e10eb84f838bd93a0a30fe55f0" modify_external_catalogs: true
I20260811 18:57:25.347272 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ca37dd77c42a19146a99452ff87b0 (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.347611 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ca37dd77c42a19146a99452ff87b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.345054 4167 rpcz_store.cc:276] 0811 18:57:19.591115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.593952 (+ 2837us) service_pool.cc:227] Handling call
0811 18:57:25.340660 (+5746708us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:25.340691 (+ 31us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.344911 (+ 4220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5746058}
I20260811 18:57:25.348593 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 423) took 5753 ms. Trace:
I20260811 18:57:25.348708 4167 rpcz_store.cc:276] 0811 18:57:19.594898 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.597229 (+ 2331us) service_pool.cc:227] Handling call
0811 18:57:25.348308 (+5751079us) ts_tablet_manager.cc:1145] Deleting tablet 8d1e244addf5462499fcadaa3332cba1
0811 18:57:25.348327 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.348555 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5735902}
I20260811 18:57:25.349004 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c2247194db704f9abd54a8a0df1ca566 (table default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]) successfully deleted
W20260811 18:57:25.349418 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fc3f120758c6405dafd2ea0bf54c81d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3f120758c6405dafd2ea0bf54c81d8 already in progress: creating tablet
I20260811 18:57:25.349797 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.349846 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8d1e244addf5462499fcadaa3332cba1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d1e244addf5462499fcadaa3332cba1 already in progress: creating tablet
I20260811 18:57:25.339354 4162 consensus_queue.cc:260] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.352907 5048 consensus_queue.cc:1035] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:25.354095 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.356895 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be655558b83041fcace57520a599b8a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.359321 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 45fde25bcb40493aafb083e161aa6932
I20260811 18:57:25.359882 4162 ts_tablet_manager.cc:1428] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.030s user 0.002s sys 0.002s
I20260811 18:57:25.360321 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d86b05dec10e4f3da6b9c48860fe6d88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.361081 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d86b05dec10e4f3da6b9c48860fe6d88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.360956 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce521c08754bf391215ab98230fb9d (DEFAULT_TABLE table=default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 [id=d3b4765ced4341a2b9944a39cf19019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.361475 4162 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.362720 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 424) took 5708 ms. Trace:
I20260811 18:57:25.363443 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 380c1f53b3ac4509a09b25a2800028af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.362830 4167 rpcz_store.cc:276] 0811 18:57:19.654759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.655356 (+ 597us) service_pool.cc:227] Handling call
0811 18:57:25.349012 (+5693656us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:25.349026 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.362699 (+ 13673us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5693174}
I20260811 18:57:25.368431 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 425) took 5672 ms. Trace:
W20260811 18:57:25.368533 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 515d5b5afbd74ec5ab90a503aef2f4fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 515d5b5afbd74ec5ab90a503aef2f4fb already in progress: creating tablet
I20260811 18:57:25.371400 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.373468 3870 tablet_service.cc:1467] Processing CreateTablet for tablet b7b48a35acf1486bbb63680ca5da31dd (DEFAULT_TABLE table=default.Table_91dfb6056e5049bb8cca8a06b3a33b7f [id=68958402d14841a494933aa943a07ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.374166 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b48a35acf1486bbb63680ca5da31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.374854 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce521c08754bf391215ab98230fb9d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.375669 3824 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.371s (SENT)
I20260811 18:57:25.375785 3824 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.385061 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce521c08754bf391215ab98230fb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.368618 4167 rpcz_store.cc:276] 0811 18:57:19.695936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.706446 (+ 10510us) service_pool.cc:227] Handling call
0811 18:57:25.368174 (+5661728us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:25.368192 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.368401 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5649575}
W20260811 18:57:25.390691 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e18242f762d8407db0e2f8716890feb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e18242f762d8407db0e2f8716890feb4 already in progress: creating tablet
I20260811 18:57:25.390838 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 433) took 5486 ms. Trace:
I20260811 18:57:25.401304 5037 raft_consensus.cc:2726] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:25.405313 4162 tablet_bootstrap.cc:654] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.390942 4167 rpcz_store.cc:276] 0811 18:57:19.904502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.904736 (+ 234us) service_pool.cc:227] Handling call
0811 18:57:25.390683 (+5485947us) ts_tablet_manager.cc:1145] Deleting tablet 0bc6cce2c19746219b8b0fe54b13dcd6
0811 18:57:25.390698 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.390815 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5467517}
W20260811 18:57:25.407397 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bc6cce2c19746219b8b0fe54b13dcd6 already in progress: creating tablet
I20260811 18:57:25.407419 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 435) took 5495 ms. Trace:
I20260811 18:57:25.407543 4167 rpcz_store.cc:276] 0811 18:57:19.911648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.911737 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:25.407239 (+5495502us) ts_tablet_manager.cc:1145] Deleting tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
0811 18:57:25.407256 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.407393 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5484028}
W20260811 18:57:25.408119 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 already in progress: creating tablet
I20260811 18:57:25.408116 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 434) took 5501 ms. Trace:
I20260811 18:57:25.408224 4167 rpcz_store.cc:276] 0811 18:57:19.906407 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.906466 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:25.407987 (+5501521us) ts_tablet_manager.cc:1145] Deleting tablet 7a0e0d139c924f1dafe42da0bedada72
0811 18:57:25.408000 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.408098 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5481021}
I20260811 18:57:25.408797 4167 tablet_replica.cc:330] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:25.408813 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7a0e0d139c924f1dafe42da0bedada72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a0e0d139c924f1dafe42da0bedada72 already in progress: creating tablet
I20260811 18:57:25.409145 4167 raft_consensus.cc:2229] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.409528 4167 raft_consensus.cc:2258] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.419128 4167 ts_tablet_manager.cc:1905] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.424501 4162 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.424700 4162 ts_tablet_manager.cc:1397] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.000s
I20260811 18:57:25.425925 4162 raft_consensus.cc:348] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:25.426247 4162 raft_consensus.cc:374] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.427120 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.427383 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.426319 4162 raft_consensus.cc:729] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.431052 4162 consensus_queue.cc:260] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:25.431288 3824 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.987s (SENT)
I20260811 18:57:25.431703 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.433209 4162 ts_tablet_manager.cc:1428] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260811 18:57:25.433514 4162 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.436091 3678 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.142s (SENT)
I20260811 18:57:25.431425 3824 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:25.443662 3824 raft_consensus.cc:2670] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.446967 4162 tablet_bootstrap.cc:654] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.452055 3678 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.412s (SENT)
I20260811 18:57:25.456877 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 380c1f53b3ac4509a09b25a2800028af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.459035 5048 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.459298 5048 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.460948 5048 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.461354 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:25.462002 3676 leader_election.cc:341] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.464601 4162 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.464778 4162 ts_tablet_manager.cc:1397] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.001s
I20260811 18:57:25.465022 4167 ts_tablet_manager.cc:1918] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.466571 4167 log.cc:1196] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/33d995d72f8142afaeb57a2c29910fa3
I20260811 18:57:25.467876 4167 ts_tablet_manager.cc:1939] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:25.468691 3678 leader_election.cc:334] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.323s (SENT)
I20260811 18:57:25.488426 4163 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.466120 4162 raft_consensus.cc:348] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.488602 4163 ts_tablet_manager.cc:1397] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.245s user 0.006s sys 0.002s
I20260811 18:57:25.488556 4162 raft_consensus.cc:374] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.488669 4162 raft_consensus.cc:729] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.488902 4162 consensus_queue.cc:260] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.489645 4162 ts_tablet_manager.cc:1428] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260811 18:57:25.490007 4162 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.490442 4163 raft_consensus.cc:348] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.490916 4163 raft_consensus.cc:374] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.491030 4163 raft_consensus.cc:729] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.491556 5091 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.491726 5091 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.491972 4163 consensus_queue.cc:260] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.492419 5091 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.488556 3678 leader_election.cc:302] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.492844 3678 raft_consensus.cc:2670] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.493945 4162 tablet_bootstrap.cc:654] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.494302 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 443) took 5222 ms. Trace:
I20260811 18:57:25.494406 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:25.494889 3891 raft_consensus.cc:2454] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:25.495540 3676 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.495978 4167 rpcz_store.cc:276] 0811 18:57:20.271386 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.271453 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:25.408738 (+5137285us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:25.408748 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.432968 (+ 24220us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.464883 (+ 31915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.494278 (+ 29395us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5136635}
I20260811 18:57:25.496764 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 33d995d72f8142afaeb57a2c29910fa3 (table default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]) successfully deleted
I20260811 18:57:25.497190 5048 raft_consensus.cc:2781] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:25.497298 5048 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:25.497382 5048 raft_consensus.cc:3037] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.498492 4167 tablet_replica.cc:330] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.499699 4167 raft_consensus.cc:2229] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.501246 4167 raft_consensus.cc:2258] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.503517 4167 ts_tablet_manager.cc:1905] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.501758 5048 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.517437 5048 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.518134 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:25.518549 3891 raft_consensus.cc:3037] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.537071 4167 ts_tablet_manager.cc:1918] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:25.543530 3678 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.052s (SENT)
I20260811 18:57:25.553838 4167 log.cc:1196] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/405773849fb94800be61ad8a7a6055a4
I20260811 18:57:25.555231 4167 ts_tablet_manager.cc:1939] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.562282 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 444) took 5286 ms. Trace:
I20260811 18:57:25.563211 4167 rpcz_store.cc:276] 0811 18:57:20.272297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.272355 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:25.498436 (+5226081us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:25.498449 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.523359 (+ 24910us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.536930 (+ 13571us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.558314 (+ 21384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5225722}
I20260811 18:57:25.565042 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:25.565503 4167 tablet_replica.cc:330] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.567605 4167 raft_consensus.cc:2229] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.569563 4167 raft_consensus.cc:2258] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.572297 4167 ts_tablet_manager.cc:1905] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.536455 5037 consensus_queue.cc:1035] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:25.582281 4163 ts_tablet_manager.cc:1428] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.094s user 0.004s sys 0.000s
I20260811 18:57:25.582894 4163 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.591195 3891 raft_consensus.cc:2454] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:25.592170 3676 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.592729 5113 raft_consensus.cc:2781] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:25.593003 5113 raft_consensus.cc:686] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:25.593503 5113 consensus_queue.cc:237] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.630128 4167 ts_tablet_manager.cc:1918] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.632937 4167 log.cc:1196] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e7259cf84f5c4c89a1b4510f743c2bc5
I20260811 18:57:25.636049 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91d4fb4116244d9e8c08ce623bea3a2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.637161 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91d4fb4116244d9e8c08ce623bea3a2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.638288 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.638623 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.639629 4167 ts_tablet_manager.cc:1939] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.642669 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 445) took 5369 ms. Trace:
I20260811 18:57:25.645859 4167 rpcz_store.cc:276] 0811 18:57:20.272923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.272984 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:25.565424 (+5292440us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:25.565446 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.598416 (+ 32970us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.629984 (+ 31568us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.642645 (+ 12661us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5279348}
I20260811 18:57:25.647450 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e7259cf84f5c4c89a1b4510f743c2bc5 (table default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]) successfully deleted
I20260811 18:57:25.648360 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 446) took 5361 ms. Trace:
I20260811 18:57:25.649287 4167 rpcz_store.cc:276] 0811 18:57:20.286892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.286958 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:25.647534 (+5360576us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:25.647546 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.648340 (+ 794us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5346867}
W20260811 18:57:25.651006 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6f3060822021431584a494e6941be756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f3060822021431584a494e6941be756 already in progress: creating tablet
I20260811 18:57:25.652355 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 447) took 5340 ms. Trace:
I20260811 18:57:25.652721 4163 tablet_bootstrap.cc:654] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.653440 4167 rpcz_store.cc:276] 0811 18:57:20.311638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.311710 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:25.651119 (+5339409us) ts_tablet_manager.cc:1145] Deleting tablet 7645a242bc4e4845ad49bfbe20b252f2
0811 18:57:25.651136 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.652333 (+ 1197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5338956}
I20260811 18:57:25.655570 4167 tablet_replica.cc:330] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.657433 4167 raft_consensus.cc:2229] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.659276 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7645a242bc4e4845ad49bfbe20b252f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7645a242bc4e4845ad49bfbe20b252f2 already in progress: creating tablet
I20260811 18:57:25.660595 4167 raft_consensus.cc:2258] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.661715 4169 ts_tablet_manager.cc:1918] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.661865 4169 log.cc:1196] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:25.662303 4169 ts_tablet_manager.cc:1939] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.662729 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 415) took 6552 ms. Trace:
I20260811 18:57:25.662839 4169 rpcz_store.cc:276] 0811 18:57:19.110104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.111393 (+ 1289us) service_pool.cc:227] Handling call
0811 18:57:25.285623 (+6174230us) ts_tablet_manager.cc:1145] Deleting tablet 3744b3d608754de68c4c57b072d1dc15
0811 18:57:25.285645 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.472014 (+186369us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.661576 (+189562us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.662704 (+ 1128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6173769}
I20260811 18:57:25.663578 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 417) took 6544 ms. Trace:
I20260811 18:57:25.663684 4169 rpcz_store.cc:276] 0811 18:57:19.118709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.122812 (+ 4103us) service_pool.cc:227] Handling call
0811 18:57:25.663393 (+6540581us) ts_tablet_manager.cc:1145] Deleting tablet a0116997a1c64e658045c319f1c45861
0811 18:57:25.663408 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.663559 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6540238}
I20260811 18:57:25.663854 4162 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:25.664234 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a0116997a1c64e658045c319f1c45861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0116997a1c64e658045c319f1c45861 already in progress: creating tablet
I20260811 18:57:25.664618 4162 ts_tablet_manager.cc:1397] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.175s user 0.013s sys 0.025s
I20260811 18:57:25.664958 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 416) took 6554 ms. Trace:
I20260811 18:57:25.665073 4169 rpcz_store.cc:276] 0811 18:57:19.110255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.150981 (+ 40726us) service_pool.cc:227] Handling call
0811 18:57:25.664812 (+6513831us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:25.664826 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.664937 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6513575}
I20260811 18:57:25.665580 4167 ts_tablet_manager.cc:1905] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.665688 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 27cda53b7ca54346a7a22bf26f5db990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27cda53b7ca54346a7a22bf26f5db990 already in progress: creating tablet
I20260811 18:57:25.666563 4162 raft_consensus.cc:348] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.666931 4162 raft_consensus.cc:374] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.667016 4162 raft_consensus.cc:729] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.667304 4162 consensus_queue.cc:260] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.668197 4162 ts_tablet_manager.cc:1428] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:25.674993 4162 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.683590 4163 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.683766 4163 ts_tablet_manager.cc:1397] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.101s user 0.003s sys 0.002s
W20260811 18:57:25.683666 3678 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.083s (SENT)
I20260811 18:57:25.684933 4163 raft_consensus.cc:348] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.685242 4163 raft_consensus.cc:374] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.685297 4163 raft_consensus.cc:729] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.685539 4163 consensus_queue.cc:260] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.690917 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 418) took 6539 ms. Trace:
I20260811 18:57:25.691075 4169 rpcz_store.cc:276] 0811 18:57:19.151545 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.154980 (+ 3435us) service_pool.cc:227] Handling call
0811 18:57:25.690641 (+6535661us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:25.690660 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.690884 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6535293}
I20260811 18:57:25.691654 4169 tablet_replica.cc:330] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.691890 4169 raft_consensus.cc:2229] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.691980 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ce94d37efd2245bf8cc71fdd90bd9735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce94d37efd2245bf8cc71fdd90bd9735 already in progress: creating tablet
I20260811 18:57:25.692093 4169 raft_consensus.cc:2258] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.692822 4169 ts_tablet_manager.cc:1905] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.699019 4162 tablet_bootstrap.cc:654] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.709841 4163 ts_tablet_manager.cc:1428] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.026s user 0.001s sys 0.002s
I20260811 18:57:25.712801 4163 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:25.715947 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 51839.3us
W20260811 18:57:25.718801 3678 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.094s (SENT)
I20260811 18:57:25.725694 4167 ts_tablet_manager.cc:1918] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:25.727201 4167 log.cc:1196] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:25.728574 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.728914 4167 ts_tablet_manager.cc:1939] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.729993 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 448) took 5417 ms. Trace:
I20260811 18:57:25.730090 4167 rpcz_store.cc:276] 0811 18:57:20.312333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.312382 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:25.655481 (+5343099us) ts_tablet_manager.cc:1145] Deleting tablet 61c8443df21949ff9bdc63fa6aa34108
0811 18:57:25.655494 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.673545 (+ 18051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.710255 (+ 36710us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.729968 (+ 19713us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5342873}
I20260811 18:57:25.738391 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cee8eb42df9b40cd85eb35169133787a" } new_table_name: "default.Table_e8871173d4364a52afb59e9b90e09f2c" modify_external_catalogs: true
I20260811 18:57:25.739435 4167 tablet_replica.cc:330] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.739773 4167 raft_consensus.cc:2229] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.740202 4167 raft_consensus.cc:2258] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.740919 4167 ts_tablet_manager.cc:1905] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.746877 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]) successfully deleted
W20260811 18:57:25.747567 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 has no consensus state
I20260811 18:57:25.748819 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_853572d26c9642b9b8ad616f574301dc" } modify_external_catalogs: true
I20260811 18:57:25.749854 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 86d690ec09524248bc147ad7408e2ee6 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.751068 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8304ac7709ca4392baec4cfa78ef5b41
I20260811 18:57:25.751993 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 8304ac7709ca4392baec4cfa78ef5b41 deleted and replaced by tablet d6c81850cdc74b4ab727bd2279162477
I20260811 18:57:25.753285 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.753690 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.754141 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5f4d9f42e99c454e88964d343262e534" } modify_external_catalogs: true
I20260811 18:57:25.754554 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5f4d9f42e99c454e88964d343262e534" } modify_external_catalogs: true
I20260811 18:57:25.755728 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_86e4bb4e07694c3a96ff484bd7944fd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.756212 4163 tablet_bootstrap.cc:654] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.757504 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e4bb4e07694c3a96ff484bd7944fd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.757967 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eff26ab40dd947859e4f83ff55b793d4" } new_table_name: "default.Table_fe2f572c66cd4b9a8656e0fd439d50c7" modify_external_catalogs: true
I20260811 18:57:25.758445 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6a3985ef18b44138437cd5cd54f34ed" } new_table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" modify_external_catalogs: true
I20260811 18:57:25.759087 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_853572d26c9642b9b8ad616f574301dc" } modify_external_catalogs: true
I20260811 18:57:25.769313 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.769623 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.770677 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_176c852b84c14616bc0ea0f110671639" } modify_external_catalogs: true
I20260811 18:57:25.771175 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_176c852b84c14616bc0ea0f110671639" } modify_external_catalogs: true
I20260811 18:57:25.793570 4169 ts_tablet_manager.cc:1918] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.793751 4169 log.cc:1196] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:25.794250 4169 ts_tablet_manager.cc:1939] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.796452 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c05597a50f6e4aea9c286eae0cf9a54d" } new_table_name: "default.Table_c3c124426696404c9e9cb5200f63055e" modify_external_catalogs: true
I20260811 18:57:25.799937 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.800232 4163 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.802223 4163 ts_tablet_manager.cc:1397] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.090s user 0.002s sys 0.006s
I20260811 18:57:25.803313 4163 raft_consensus.cc:348] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.803599 4163 raft_consensus.cc:374] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.803645 4163 raft_consensus.cc:729] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.803879 4163 consensus_queue.cc:260] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.804417 4162 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.804550 4162 ts_tablet_manager.cc:1397] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.130s user 0.000s sys 0.008s
I20260811 18:57:25.805531 4162 raft_consensus.cc:348] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.805755 4162 raft_consensus.cc:374] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.805809 4162 raft_consensus.cc:729] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:25.805961 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bfb54175c834517b7af291c942333cd consensus state has no leader
W20260811 18:57:25.806041 3824 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.608s (SENT)
I20260811 18:57:25.806174 3824 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.806051 4162 consensus_queue.cc:260] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:25.806321 3824 raft_consensus.cc:2670] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.806710 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f" } modify_external_catalogs: true
I20260811 18:57:25.806991 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f" } modify_external_catalogs: true
I20260811 18:57:25.808039 4163 ts_tablet_manager.cc:1428] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260811 18:57:25.808182 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3b6378d1219433cb724148f1ae74b25 consensus state has no leader
I20260811 18:57:25.808346 4163 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.808727 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fc78208e540445719638da3757163df5" } modify_external_catalogs: true
I20260811 18:57:25.809036 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fc78208e540445719638da3757163df5" } modify_external_catalogs: true
W20260811 18:57:25.809886 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35ebc32eb5d5443eaaeba245b566f98a consensus state has no leader
I20260811 18:57:25.810379 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 84e152c74f33463e83843b92c0783803 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.811923 4162 ts_tablet_manager.cc:1428] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 18:57:25.812242 4162 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.814085 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_7cabaea271024c3daf3ffc96c5825f1e [id=bd974444a6c546959eb6442a80543187], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
I20260811 18:57:25.814782 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:25.816026 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.816223 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d1e244addf5462499fcadaa3332cba1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.816313 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.822225 4167 ts_tablet_manager.cc:1918] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:25.822428 4167 log.cc:1196] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6381a951a966473c80670b0dd1aff062
I20260811 18:57:25.823112 4167 ts_tablet_manager.cc:1939] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.823716 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 450) took 5314 ms. Trace:
I20260811 18:57:25.823808 4167 rpcz_store.cc:276] 0811 18:57:20.509040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.509106 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:25.739355 (+5230249us) ts_tablet_manager.cc:1145] Deleting tablet 6381a951a966473c80670b0dd1aff062
0811 18:57:25.739375 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.758619 (+ 19244us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.762282 (+ 3663us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.823694 (+ 61412us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5225900}
I20260811 18:57:25.824364 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_4da51fec1a044237ab13da18366dde64 [id=263eb849fd614011a9dd641aeb9126d0]) successfully deleted
I20260811 18:57:25.824420 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 463) took 5001 ms. Trace:
I20260811 18:57:25.824501 4167 rpcz_store.cc:276] 0811 18:57:20.822748 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.822839 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:25.824277 (+5001438us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:25.824289 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.824402 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4996681}
W20260811 18:57:25.825165 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 43ad6993b63e4bbd98c8cc52f1a53451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43ad6993b63e4bbd98c8cc52f1a53451 already in progress: creating tablet
I20260811 18:57:25.827071 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 464) took 4993 ms. Trace:
I20260811 18:57:25.827206 4167 rpcz_store.cc:276] 0811 18:57:20.833426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.833568 (+ 142us) service_pool.cc:227] Handling call
0811 18:57:25.826932 (+4993364us) ts_tablet_manager.cc:1145] Deleting tablet 91818d0cbe144a67a458a6648968bf5b
0811 18:57:25.826943 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.827050 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4991372}
I20260811 18:57:25.827718 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 467) took 4960 ms. Trace:
I20260811 18:57:25.827812 4167 rpcz_store.cc:276] 0811 18:57:20.867330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.867586 (+ 256us) service_pool.cc:227] Handling call
0811 18:57:25.827596 (+4960010us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:25.827608 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.827699 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957788}
I20260811 18:57:25.828078 5037 raft_consensus.cc:482] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:25.828289 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 91818d0cbe144a67a458a6648968bf5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91818d0cbe144a67a458a6648968bf5b already in progress: creating tablet
I20260811 18:57:25.828249 5037 raft_consensus.cc:504] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:25.828827 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 059b325df1db4fde9d4b26b559121316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059b325df1db4fde9d4b26b559121316 already in progress: creating tablet
I20260811 18:57:25.829075 5037 leader_election.cc:288] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:25.829492 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae consensus state has no leader
I20260811 18:57:25.830240 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 465) took 4989 ms. Trace:
I20260811 18:57:25.830330 4167 rpcz_store.cc:276] 0811 18:57:20.840379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.840503 (+ 124us) service_pool.cc:227] Handling call
0811 18:57:25.830033 (+4989530us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:25.830043 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.830222 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4959989}
W20260811 18:57:25.830936 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b6f36ab6bb774257b5c2692c64eff63d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6f36ab6bb774257b5c2692c64eff63d already in progress: creating tablet
W20260811 18:57:25.831367 3824 leader_election.cc:341] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.833345 4167 tablet_replica.cc:330] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.833576 4167 raft_consensus.cc:2229] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.833174 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1af5a92fcee4bc4898572e55da333b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.833884 4167 raft_consensus.cc:2258] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.833891 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 419) took 6568 ms. Trace:
I20260811 18:57:25.833976 4169 rpcz_store.cc:276] 0811 18:57:19.265679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.265773 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:25.691601 (+6425828us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:25.691613 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.747207 (+ 55594us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.793439 (+ 46232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.833869 (+ 40430us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6376377}
I20260811 18:57:25.834615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 431) took 6355 ms. Trace:
I20260811 18:57:25.834532 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 828993fb34bb4c62bce2a9322d7c992a (table default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]) successfully deleted
I20260811 18:57:25.834753 4169 rpcz_store.cc:276] 0811 18:57:19.479453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.485654 (+ 6201us) service_pool.cc:227] Handling call
0811 18:57:25.834470 (+6348816us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:25.834486 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.834596 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6322417}
I20260811 18:57:25.835242 4169 tablet_replica.cc:330] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:25.835650 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1af5a92fcee4bc4898572e55da333b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.835927 4169 raft_consensus.cc:2229] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.836140 4169 raft_consensus.cc:2258] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.836872 4169 ts_tablet_manager.cc:1905] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.845615 4162 tablet_bootstrap.cc:654] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.848906 4163 tablet_bootstrap.cc:654] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.855180 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15425ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:25.857146 4167 ts_tablet_manager.cc:1905] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.857858 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 6d28c486e408451a990ae3905ef3c2ff from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:25.873435 3824 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.118s (SENT)
I20260811 18:57:25.873608 3824 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.874076 5037 raft_consensus.cc:2726] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:25.873903 4163 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.876745 4163 ts_tablet_manager.cc:1397] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.068s user 0.006s sys 0.001s
W20260811 18:57:25.885480 3824 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.245s (SENT)
I20260811 18:57:25.885659 3824 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:25.885874 3824 raft_consensus.cc:2670] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.885505 4163 raft_consensus.cc:348] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.887008 4163 raft_consensus.cc:374] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.887087 4163 raft_consensus.cc:729] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.888481 5037 raft_consensus.cc:482] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.888665 5037 raft_consensus.cc:504] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.889678 5037 leader_election.cc:288] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:25.890810 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "283c3fe4f41f4185ab4488d774e46299" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:25.891149 3744 raft_consensus.cc:2454] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:25.891829 3824 leader_election.cc:302] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.891922 4167 ts_tablet_manager.cc:1918] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.892040 4167 log.cc:1196] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:25.892316 5037 raft_consensus.cc:2781] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:25.892396 5037 raft_consensus.cc:482] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:25.892454 5037 raft_consensus.cc:3037] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.892509 4167 ts_tablet_manager.cc:1939] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.892961 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 466) took 5041 ms. Trace:
I20260811 18:57:25.893042 4167 rpcz_store.cc:276] 0811 18:57:20.851895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.852357 (+ 462us) service_pool.cc:227] Handling call
0811 18:57:25.833284 (+4980927us) ts_tablet_manager.cc:1145] Deleting tablet a732c700ff79432aaf1f0d1c1e90aabe
0811 18:57:25.833298 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.861672 (+ 28374us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.891806 (+ 30134us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.892944 (+ 1138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4921218}
I20260811 18:57:25.893671 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a732c700ff79432aaf1f0d1c1e90aabe (table default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]) successfully deleted
I20260811 18:57:25.896575 5037 raft_consensus.cc:504] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.898761 4163 consensus_queue.cc:260] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.899502 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.899844 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
W20260811 18:57:25.900275 3678 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.062s (SENT)
I20260811 18:57:25.901022 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "283c3fe4f41f4185ab4488d774e46299" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:25.901224 3744 raft_consensus.cc:3037] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.902822 4163 ts_tablet_manager.cc:1428] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.026s user 0.001s sys 0.002s
I20260811 18:57:25.903101 4163 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.904909 3744 raft_consensus.cc:2454] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:25.911271 4162 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.911423 4162 ts_tablet_manager.cc:1397] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.099s user 0.001s sys 0.009s
I20260811 18:57:25.916056 4162 raft_consensus.cc:348] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.916365 4162 raft_consensus.cc:374] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.916427 4162 raft_consensus.cc:729] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.916663 4162 consensus_queue.cc:260] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.917419 4162 ts_tablet_manager.cc:1428] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:25.917708 4162 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.918903 3678 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.235s (SENT)
I20260811 18:57:25.919701 3824 leader_election.cc:302] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.922755 5113 raft_consensus.cc:482] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.922921 5113 raft_consensus.cc:504] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.923638 5113 leader_election.cc:288] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:25.925386 3676 leader_election.cc:341] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a443fe209556489abb47f4faa5d696e5
I20260811 18:57:25.927587 4162 tablet_bootstrap.cc:654] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.929455 4167 tablet_replica.cc:330] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.930559 4167 raft_consensus.cc:2229] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.930761 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
I20260811 18:57:25.932545 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.933039 5037 leader_election.cc:288] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:25.933267 4167 raft_consensus.cc:2258] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.933005 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e89523faca1046669de08be2f6b963dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.933763 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e89523faca1046669de08be2f6b963dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.934036 4167 ts_tablet_manager.cc:1905] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.937073 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.944876 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7e8ba858b0c44d588499e1463673957"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.945350 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.920194 5080 raft_consensus.cc:2781] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:25.945657 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7e8ba858b0c44d588499e1463673957 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.945675 5080 raft_consensus.cc:686] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:25.945993 5080 consensus_queue.cc:237] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.947744 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e3a4280bd164399a5346ffaa6f40b8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.948199 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.948426 4163 tablet_bootstrap.cc:654] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.948577 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e3a4280bd164399a5346ffaa6f40b8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.948962 4169 ts_tablet_manager.cc:1918] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:25.949090 4169 log.cc:1196] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.951275 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b069c186694049998175e3d83241b318"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.951694 4169 ts_tablet_manager.cc:1939] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:25.952060 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b069c186694049998175e3d83241b318 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.952776 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 84e152c74f33463e83843b92c0783803
I20260811 18:57:25.953190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 432) took 6416 ms. Trace:
I20260811 18:57:25.953415 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 84e152c74f33463e83843b92c0783803 deleted and replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780
I20260811 18:57:25.953306 4169 rpcz_store.cc:276] 0811 18:57:19.537253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.540139 (+ 2886us) service_pool.cc:227] Handling call
0811 18:57:25.835188 (+6295049us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:25.835202 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.871425 (+ 36223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.948849 (+ 77424us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.953169 (+ 4320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6234725}
I20260811 18:57:25.953824 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 434) took 6353 ms. Trace:
I20260811 18:57:25.953913 4169 rpcz_store.cc:276] 0811 18:57:19.600317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.602562 (+ 2245us) service_pool.cc:227] Handling call
0811 18:57:25.953653 (+6351091us) ts_tablet_manager.cc:1145] Deleting tablet 8d1e244addf5462499fcadaa3332cba1
0811 18:57:25.953667 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.953805 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6350661}
I20260811 18:57:25.954677 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 649cb98f553a418b911e307fdaf49917
I20260811 18:57:25.954918 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e152c74f33463e83843b92c0783803 with delete_type TABLET_DATA_DELETED (replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.956223 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 649cb98f553a418b911e307fdaf49917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.956271 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1cd8ec7574294bc6886f42d65935c0a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.957266 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 86d690ec09524248bc147ad7408e2ee6
I20260811 18:57:25.958748 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 86d690ec09524248bc147ad7408e2ee6 deleted and replaced by tablet b6c70b4163634984bd02dd9b47ffb472
I20260811 18:57:25.959193 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88a784338b5e4928be15e4679b3f3304"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.959474 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d28c486e408451a990ae3905ef3c2ff
I20260811 18:57:25.959724 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e152c74f33463e83843b92c0783803 with delete_type TABLET_DATA_DELETED (replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960297 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d690ec09524248bc147ad7408e2ee6 with delete_type TABLET_DATA_DELETED (replaced by tablet b6c70b4163634984bd02dd9b47ffb472) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960403 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 649cb98f553a418b911e307fdaf49917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.954272 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 433) took 6354 ms. Trace:
I20260811 18:57:25.960629 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d28c486e408451a990ae3905ef3c2ff with delete_type TABLET_DATA_DELETED (replaced by tablet bb1cb45730b14b998a8f547983b83823) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960610 4169 rpcz_store.cc:276] 0811 18:57:19.600151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.607558 (+ 7407us) service_pool.cc:227] Handling call
0811 18:57:25.954166 (+6346608us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:25.954178 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.954257 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6346240}
I20260811 18:57:25.961313 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 435) took 6298 ms. Trace:
I20260811 18:57:25.961428 4169 rpcz_store.cc:276] 0811 18:57:19.663125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.663192 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:25.961075 (+6297883us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:25.961089 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.961292 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6291061}
I20260811 18:57:25.961581 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 6d28c486e408451a990ae3905ef3c2ff deleted and replaced by tablet bb1cb45730b14b998a8f547983b83823
I20260811 18:57:25.961828 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 436) took 6281 ms. Trace:
I20260811 18:57:25.961925 4169 rpcz_store.cc:276] 0811 18:57:19.680169 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.680241 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:25.961712 (+6281471us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:25.961725 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.961811 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6281088}
I20260811 18:57:25.962270 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 442) took 6057 ms. Trace:
I20260811 18:57:25.962373 4169 rpcz_store.cc:276] 0811 18:57:19.904745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.904860 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:25.962162 (+6057302us) ts_tablet_manager.cc:1145] Deleting tablet 0bc6cce2c19746219b8b0fe54b13dcd6
0811 18:57:25.962174 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.962254 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6056381}
I20260811 18:57:25.962741 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 444) took 6054 ms. Trace:
I20260811 18:57:25.962811 4169 rpcz_store.cc:276] 0811 18:57:19.908425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.908510 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:25.962656 (+6054146us) ts_tablet_manager.cc:1145] Deleting tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
0811 18:57:25.962665 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.962729 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6035313}
I20260811 18:57:25.962911 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c2247194db704f9abd54a8a0df1ca566 (table default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]) successfully deleted
I20260811 18:57:25.963037 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 443) took 6055 ms. Trace:
I20260811 18:57:25.963093 4169 rpcz_store.cc:276] 0811 18:57:19.907149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.907988 (+ 839us) service_pool.cc:227] Handling call
0811 18:57:25.962965 (+6054977us) ts_tablet_manager.cc:1145] Deleting tablet 7a0e0d139c924f1dafe42da0bedada72
0811 18:57:25.962973 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963027 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6035112}
W20260811 18:57:25.963307 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8d1e244addf5462499fcadaa3332cba1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d1e244addf5462499fcadaa3332cba1 already in progress: creating tablet
I20260811 18:57:25.963361 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 451) took 5726 ms. Trace:
I20260811 18:57:25.963429 4169 rpcz_store.cc:276] 0811 18:57:20.236628 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.237034 (+ 406us) service_pool.cc:227] Handling call
0811 18:57:25.963252 (+5726218us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:25.963260 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963345 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5706677}
I20260811 18:57:25.963753 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 452) took 5713 ms. Trace:
I20260811 18:57:25.963845 4169 rpcz_store.cc:276] 0811 18:57:20.250514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.250583 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:25.963648 (+5713065us) ts_tablet_manager.cc:1145] Deleting tablet 7645a242bc4e4845ad49bfbe20b252f2
0811 18:57:25.963659 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963737 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5696989}
I20260811 18:57:25.964433 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d690ec09524248bc147ad7408e2ee6 with delete_type TABLET_DATA_DELETED (replaced by tablet b6c70b4163634984bd02dd9b47ffb472) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.964825 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b7b48a35acf1486bbb63680ca5da31dd
I20260811 18:57:25.964954 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d28c486e408451a990ae3905ef3c2ff with delete_type TABLET_DATA_DELETED (replaced by tablet bb1cb45730b14b998a8f547983b83823) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.965935 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b48a35acf1486bbb63680ca5da31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.966748 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fc3f120758c6405dafd2ea0bf54c81d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3f120758c6405dafd2ea0bf54c81d8 already in progress: creating tablet
W20260811 18:57:25.967113 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 515d5b5afbd74ec5ab90a503aef2f4fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 515d5b5afbd74ec5ab90a503aef2f4fb already in progress: creating tablet
I20260811 18:57:25.958441 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a905052284004a77adf260546090e666"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.969015 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a905052284004a77adf260546090e666 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.970794 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cd8ec7574294bc6886f42d65935c0a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.972198 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88a784338b5e4928be15e4679b3f3304 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.973557 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e18242f762d8407db0e2f8716890feb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e18242f762d8407db0e2f8716890feb4 already in progress: creating tablet
W20260811 18:57:25.973994 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bc6cce2c19746219b8b0fe54b13dcd6 already in progress: creating tablet
W20260811 18:57:25.974493 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 already in progress: creating tablet
I20260811 18:57:25.974500 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b48a35acf1486bbb63680ca5da31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:25.974871 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7a0e0d139c924f1dafe42da0bedada72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a0e0d139c924f1dafe42da0bedada72 already in progress: creating tablet
W20260811 18:57:25.975479 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6f3060822021431584a494e6941be756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f3060822021431584a494e6941be756 already in progress: creating tablet
W20260811 18:57:25.975775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7645a242bc4e4845ad49bfbe20b252f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7645a242bc4e4845ad49bfbe20b252f2 already in progress: creating tablet
I20260811 18:57:25.975199 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69be92ae6ea74daf931e8ccd58a8dd4f" } new_table_name: "default.Table_e98a0fba26cb4171bf8eb772249c3e47" modify_external_catalogs: true
W20260811 18:57:25.978710 3678 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.383s (SENT)
I20260811 18:57:25.982561 4169 tablet_replica.cc:330] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.982606 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.982872 4169 raft_consensus.cc:2229] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.983295 4169 raft_consensus.cc:2258] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.983104 3550 catalog_manager.cc:5495] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:25.984115 4169 ts_tablet_manager.cc:1905] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.991371 3678 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.281s (SENT)
I20260811 18:57:25.993602 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a3209bc9f1194850ae8fde67d25c8a0d (DEFAULT_TABLE table=default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d [id=ccecc852942941bc9e47d9af9c07495f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.994124 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3209bc9f1194850ae8fde67d25c8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.001608 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bcb9560450d14ad392e0db4330f60317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.002462 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcb9560450d14ad392e0db4330f60317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.003962 3711 tablet_service.cc:1467] Processing CreateTablet for tablet a3209bc9f1194850ae8fde67d25c8a0d (DEFAULT_TABLE table=default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d [id=ccecc852942941bc9e47d9af9c07495f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.004491 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3209bc9f1194850ae8fde67d25c8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.009420 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 2523fa07587246d994eca722c7032105 (DEFAULT_TABLE table=default.Table_3baa014716584cb184d8a0e4299a8542 [id=35354689f6714dbc929e448b7b00d640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.010004 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2523fa07587246d994eca722c7032105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.011042 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 (DEFAULT_TABLE table=default.Table_51f392e10eb84f838bd93a0a30fe55f0 [id=f1048594851646469a3f7664e216d924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.011536 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.014925 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 (DEFAULT_TABLE table=default.Table_51f392e10eb84f838bd93a0a30fe55f0 [id=f1048594851646469a3f7664e216d924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.015343 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.019209 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2a926bfdbff42feaa7cfed27e0c04a0" } new_table_name: "default.Table_ab9ee42f727a415ab19b78c0e5bd013a" modify_external_catalogs: true
I20260811 18:57:26.021100 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 05cba6b392b642e59321570aac3541d0 (DEFAULT_TABLE table=default.Table_59f9d96102f349498c7f854e48e3d7c5 [id=0b705b356df3452a966e56ef0d684159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.021282 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4b52f563c5f4629a03019c85bfbb653" } new_table_name: "default.Table_f2f1e0bdeb7a45e3b9e6041aa2ac2578" modify_external_catalogs: true
I20260811 18:57:26.021564 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cba6b392b642e59321570aac3541d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.023445 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_771bec99eb584420a45d22e9b038229a" } new_table_name: "default.Table_21287a00ce0a446fb09ba45cacede2c2" modify_external_catalogs: true
I20260811 18:57:26.025233 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_85a94fea83f84948a14090e2deeac956"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.025558 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 05cba6b392b642e59321570aac3541d0 (DEFAULT_TABLE table=default.Table_59f9d96102f349498c7f854e48e3d7c5 [id=0b705b356df3452a966e56ef0d684159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.026042 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cba6b392b642e59321570aac3541d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.026191 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85a94fea83f84948a14090e2deeac956 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.031221 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 2523fa07587246d994eca722c7032105 (DEFAULT_TABLE table=default.Table_3baa014716584cb184d8a0e4299a8542 [id=35354689f6714dbc929e448b7b00d640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.032091 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2523fa07587246d994eca722c7032105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.036257 4162 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.036413 4162 ts_tablet_manager.cc:1397] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.002s sys 0.003s
I20260811 18:57:26.037451 4162 raft_consensus.cc:348] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.037376 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9e2e268d15894407ab276766ff611c86" } new_table_name: "default.Table_102e5367d375402c9800357142894fa0" modify_external_catalogs: true
I20260811 18:57:26.037665 4162 raft_consensus.cc:374] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.037725 4162 raft_consensus.cc:729] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.039409 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 (DEFAULT_TABLE table=default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 [id=c9676ca3a1a8471bb1dfc3dbf19d7861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.040067 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8146f47d9ae48889ac7efc954d4a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.040831 4162 consensus_queue.cc:260] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.042960 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 consensus state has no leader
I20260811 18:57:26.045965 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f13a14bd64704798bf6de4fc05201a31 (DEFAULT_TABLE table=default.Table_14b83cad2c2f4713b46f4d1fa3782f90 [id=1fa3a7f5ea7646428fc8d129dbf153fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.046427 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13a14bd64704798bf6de4fc05201a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.051090 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.053561 4162 ts_tablet_manager.cc:1428] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260811 18:57:26.053997 4162 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.056367 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f13a14bd64704798bf6de4fc05201a31 (DEFAULT_TABLE table=default.Table_14b83cad2c2f4713b46f4d1fa3782f90 [id=1fa3a7f5ea7646428fc8d129dbf153fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.037415 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13" } modify_external_catalogs: true
I20260811 18:57:26.056818 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13a14bd64704798bf6de4fc05201a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.060091 4167 ts_tablet_manager.cc:1918] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:26.060075 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 761902f7366d43ffbde226a0ade2e188 (DEFAULT_TABLE table=default.Table_0036c28f59cb4f229da243389602bbd9 [id=9b456a91de20444dbe8abd84177b3073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.060552 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761902f7366d43ffbde226a0ade2e188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.061918 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 99e9474ef8814514832f0e48c2b3495f (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.058161 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd454d73c8da4bf79e68487caa3dd843"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.062265 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e9474ef8814514832f0e48c2b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.062572 4167 log.cc:1196] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f8eb041f3d704762868a048faec0e0fc
W20260811 18:57:26.062831 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd454d73c8da4bf79e68487caa3dd843 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.063457 5113 raft_consensus.cc:482] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.063632 5113 raft_consensus.cc:504] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.063786 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13" } modify_external_catalogs: true
I20260811 18:57:26.064392 5113 leader_election.cc:288] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.057204 3870 tablet_service.cc:1467] Processing CreateTablet for tablet fcec797e73bf4399aeb2bfd4b02949c0 (DEFAULT_TABLE table=default.Table_9c4130777df64341a269a7954794efd2 [id=c52e87ba38e24c0ea8bc1ed616ba9a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.068409 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21054bdd3d7c400c95b1f41e41bafc79" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.068671 3891 raft_consensus.cc:2454] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:26.069012 3824 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.736s (SENT)
I20260811 18:57:26.069218 3676 leader_election.cc:302] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.069201 3824 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:26.068681 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcec797e73bf4399aeb2bfd4b02949c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.069432 3824 raft_consensus.cc:2670] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:26.070055 3678 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.151s (SENT)
I20260811 18:57:26.070814 5113 raft_consensus.cc:2781] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.070897 5113 raft_consensus.cc:482] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.070957 5113 raft_consensus.cc:3037] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.073770 5113 raft_consensus.cc:504] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.074059 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 99e9474ef8814514832f0e48c2b3495f (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.074460 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e9474ef8814514832f0e48c2b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.075655 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 (DEFAULT_TABLE table=default.Table_0da5d975ff884b9bb82d239828666af7 [id=c082db9ee1db46e2bc2b0ff9d94d8fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.075817 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21054bdd3d7c400c95b1f41e41bafc79" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:26.076004 3891 raft_consensus.cc:3037] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.076040 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de8ce45e514ff291cb9e5c0a4c00d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.078666 5113 leader_election.cc:288] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.082013 3891 raft_consensus.cc:2454] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:26.082558 3676 leader_election.cc:302] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.082973 5113 raft_consensus.cc:2781] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.083244 5113 raft_consensus.cc:686] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:26.083534 5113 consensus_queue.cc:237] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.085542 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ea4ad482bde44d6bab5ec809054aaebd
I20260811 18:57:26.086529 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 761902f7366d43ffbde226a0ade2e188 (DEFAULT_TABLE table=default.Table_0036c28f59cb4f229da243389602bbd9 [id=9b456a91de20444dbe8abd84177b3073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.086994 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761902f7366d43ffbde226a0ade2e188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.087432 5080 raft_consensus.cc:482] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.089002 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4ad482bde44d6bab5ec809054aaebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.089720 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 46505ba544104458888dbc2a37115dcc (DEFAULT_TABLE table=default.Table_1e4e304a679345cd9289252278ddb281 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.090054 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46505ba544104458888dbc2a37115dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.092334 3716 tablet_service.cc:1467] Processing CreateTablet for tablet fcec797e73bf4399aeb2bfd4b02949c0 (DEFAULT_TABLE table=default.Table_9c4130777df64341a269a7954794efd2 [id=c52e87ba38e24c0ea8bc1ed616ba9a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.092792 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcec797e73bf4399aeb2bfd4b02949c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.073770 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 (DEFAULT_TABLE table=default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 [id=c9676ca3a1a8471bb1dfc3dbf19d7861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.096122 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8146f47d9ae48889ac7efc954d4a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.097407 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a255a0922a92419191f18730d8b335bc (DEFAULT_TABLE table=default.Table_7f980c9b98994253b590548d86f6d145 [id=e4ed3f906b884fffbb40dfa89e10ddd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.097774 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a255a0922a92419191f18730d8b335bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.099014 4167 ts_tablet_manager.cc:1939] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.099606 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 471) took 5182 ms. Trace:
W20260811 18:57:26.103856 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:26.087568 5080 raft_consensus.cc:504] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.106555 4167 rpcz_store.cc:276] 0811 18:57:20.917227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.918975 (+ 1748us) service_pool.cc:227] Handling call
0811 18:57:25.929408 (+5010433us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:25.929417 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.965332 (+ 35915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.059979 (+ 94647us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.099587 (+ 39608us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4991598}
I20260811 18:57:26.107110 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 472) took 5124 ms. Trace:
I20260811 18:57:26.107204 4167 rpcz_store.cc:276] 0811 18:57:20.983111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.983861 (+ 750us) service_pool.cc:227] Handling call
0811 18:57:26.106951 (+5123090us) ts_tablet_manager.cc:1145] Deleting tablet 3468b39bd73c4272a84dad37fcfd6f27
0811 18:57:26.106965 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.107078 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5104954}
I20260811 18:57:26.107576 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 473) took 5119 ms. Trace:
I20260811 18:57:26.107645 4167 rpcz_store.cc:276] 0811 18:57:20.988320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.988903 (+ 583us) service_pool.cc:227] Handling call
0811 18:57:26.107457 (+5118554us) ts_tablet_manager.cc:1145] Deleting tablet bf5749d4eebd42319df1b52eeba4c062
0811 18:57:26.107466 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.107560 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5104237}
I20260811 18:57:26.108084 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 475) took 5023 ms. Trace:
I20260811 18:57:26.108170 4167 rpcz_store.cc:276] 0811 18:57:21.084332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.084404 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:26.107935 (+5023531us) ts_tablet_manager.cc:1145] Deleting tablet fdeec0c5672b4b3190d7d7cf341cb118
0811 18:57:26.107952 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108068 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5022582}
I20260811 18:57:26.108279 5080 leader_election.cc:288] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.108547 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 474) took 5024 ms. Trace:
I20260811 18:57:26.108632 4167 rpcz_store.cc:276] 0811 18:57:21.084146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.084900 (+ 754us) service_pool.cc:227] Handling call
0811 18:57:26.108388 (+5023488us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:26.108397 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108529 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5022901}
I20260811 18:57:26.109015 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 476) took 4782 ms. Trace:
I20260811 18:57:26.109112 4167 rpcz_store.cc:276] 0811 18:57:21.326640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.326702 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:26.108866 (+4782164us) ts_tablet_manager.cc:1145] Deleting tablet 289e607f10e14a2983bc6ecc93ce9bbc
0811 18:57:26.108874 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108998 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4781740}
I20260811 18:57:26.109576 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 477) took 4609 ms. Trace:
I20260811 18:57:26.109634 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e63ee9b9fd4b12ae791b9d5bc2e648" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.109650 4167 rpcz_store.cc:276] 0811 18:57:21.499808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.500275 (+ 467us) service_pool.cc:227] Handling call
0811 18:57:26.109323 (+4609048us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:26.109348 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.109561 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4583519}
I20260811 18:57:26.109969 3744 raft_consensus.cc:2454] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.110041 4167 tablet_replica.cc:330] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.110253 4167 raft_consensus.cc:2229] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.105230 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3721f75a96bf418ea7bd86ec90126b52 (DEFAULT_TABLE table=default.Table_8d7695d0859045409244bdd77ddd3931 [id=36fa8d1e4ec84fe286b5b4153857999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.105640 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 40898ceb18cc4f2d9104818798049ae2 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.110610 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f8eb041f3d704762868a048faec0e0fc (table default.Table_45c439603b1140e998b5d9199026da49 [id=32914b6ff485486f843da54878e2d49b]) successfully deleted
I20260811 18:57:26.110772 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40898ceb18cc4f2d9104818798049ae2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.111356 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3468b39bd73c4272a84dad37fcfd6f27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3468b39bd73c4272a84dad37fcfd6f27 already in progress: creating tablet
W20260811 18:57:26.111661 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf5749d4eebd42319df1b52eeba4c062 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf5749d4eebd42319df1b52eeba4c062 already in progress: creating tablet
W20260811 18:57:26.111997 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fdeec0c5672b4b3190d7d7cf341cb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdeec0c5672b4b3190d7d7cf341cb118 already in progress: creating tablet
W20260811 18:57:26.112264 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 42405aec21e3405486b6c658438b5fb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42405aec21e3405486b6c658438b5fb2 already in progress: creating tablet
W20260811 18:57:26.112641 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 289e607f10e14a2983bc6ecc93ce9bbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 289e607f10e14a2983bc6ecc93ce9bbc already in progress: creating tablet
W20260811 18:57:26.113010 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because the tablet was not found. No further retry: Not found: Tablet not found: d3d6daff4b7540309ff58755758e9752
I20260811 18:57:26.110791 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4ad482bde44d6bab5ec809054aaebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.112319 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 46505ba544104458888dbc2a37115dcc (DEFAULT_TABLE table=default.Table_1e4e304a679345cd9289252278ddb281 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.115031 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46505ba544104458888dbc2a37115dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.116647 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 (DEFAULT_TABLE table=default.Table_0da5d975ff884b9bb82d239828666af7 [id=c082db9ee1db46e2bc2b0ff9d94d8fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.117408 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de8ce45e514ff291cb9e5c0a4c00d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.118083 4162 tablet_bootstrap.cc:654] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.120316 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 40898ceb18cc4f2d9104818798049ae2 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.120795 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a255a0922a92419191f18730d8b335bc (DEFAULT_TABLE table=default.Table_7f980c9b98994253b590548d86f6d145 [id=e4ed3f906b884fffbb40dfa89e10ddd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.121196 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a255a0922a92419191f18730d8b335bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.127168 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 3721f75a96bf418ea7bd86ec90126b52 (DEFAULT_TABLE table=default.Table_8d7695d0859045409244bdd77ddd3931 [id=36fa8d1e4ec84fe286b5b4153857999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.127708 3824 leader_election.cc:302] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.128027 5080 raft_consensus.cc:2781] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.128120 5080 raft_consensus.cc:482] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.128183 5080 raft_consensus.cc:3037] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.130656 4169 ts_tablet_manager.cc:1918] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 18:57:26.134735 3678 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.151s (SENT)
I20260811 18:57:26.134733 4169 log.cc:1196] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:26.136425 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40898ceb18cc4f2d9104818798049ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.137502 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3721f75a96bf418ea7bd86ec90126b52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.137848 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e407d34d3c984ecb8c8ffde304f23ce6 consensus state has no leader
W20260811 18:57:26.138543 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bef0f6d56cda40249f12b25a8eb3ab6b consensus state has no leader
W20260811 18:57:26.139528 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
W20260811 18:57:26.140347 3678 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.518s (SENT)
I20260811 18:57:26.142035 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" } modify_external_catalogs: true
I20260811 18:57:26.144870 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3721f75a96bf418ea7bd86ec90126b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.145148 3546 catalog_manager.cc:5495] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:26.145627 3678 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.579s (SENT)
I20260811 18:57:26.145764 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
I20260811 18:57:26.146998 3865 tablet_service.cc:1467] Processing CreateTablet for tablet c282501396154e3088b39691c60cd35b (DEFAULT_TABLE table=default.Table_d7e8b32170ff47eda203b84328064301 [id=f33a076a8b49435d89198c35da0b1831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.147079 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" } modify_external_catalogs: true
I20260811 18:57:26.147485 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c282501396154e3088b39691c60cd35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.150985 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 99ecc0d308d540948de39dfe12199597 (DEFAULT_TABLE table=default.Table_2bd2a5750a454b88b5bff7b802541c89 [id=e659e4ff88894137ba1c37c450230df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.151515 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ecc0d308d540948de39dfe12199597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.155253 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.155521 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.156030 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b (DEFAULT_TABLE table=default.Table_0322ec9fd9f24338bb9196b099f56643 [id=3dac4cc0f73442c9a173f0766e0e9874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.156453 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a5fdab68694761b7493b42a0bf7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.158929 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 (DEFAULT_TABLE table=default.Table_c6ed856dda814eb0bd8eb76c2f83c70b [id=ffd5f1ae63224181bb83a8c11717ad3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.160957 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22abc6f8e9ef4c32ab69af31bdaad872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.162423 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" } modify_external_catalogs: true
I20260811 18:57:26.166998 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" } modify_external_catalogs: true
I20260811 18:57:26.168004 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d" } modify_external_catalogs: true
I20260811 18:57:26.168365 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d" } modify_external_catalogs: true
I20260811 18:57:26.170032 4167 raft_consensus.cc:2258] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.170753 4167 ts_tablet_manager.cc:1905] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.171361 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 4da53cef719842f98d2190e487e8d653 (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.171833 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da53cef719842f98d2190e487e8d653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.176107 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 4da53cef719842f98d2190e487e8d653 (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.176734 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da53cef719842f98d2190e487e8d653. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.178980 3678 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.025s (SENT)
I20260811 18:57:26.180171 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 02bf791926d84522b1099c1242acddcd (DEFAULT_TABLE table=default.Table_bc171757977f4404bb736a03a87b9ecd [id=41452d01375a443ba5c239475692f69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.180780 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02bf791926d84522b1099c1242acddcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.183338 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1d81e93739f44d9e99b6ebc647e1d0e4
I20260811 18:57:26.183838 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 54fa32f8fa1144c4bd9dcd00a550001a
W20260811 18:57:26.184456 3678 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.372s (SENT)
I20260811 18:57:26.185343 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b16cb4c1796147fbb10109f52071505e
I20260811 18:57:26.188802 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.188932 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet b16cb4c1796147fbb10109f52071505e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.194115 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_256aa74bb2884a0d83f0d17b159b9e49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.194435 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 99ecc0d308d540948de39dfe12199597 (DEFAULT_TABLE table=default.Table_2bd2a5750a454b88b5bff7b802541c89 [id=e659e4ff88894137ba1c37c450230df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.194950 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ecc0d308d540948de39dfe12199597. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.195263 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_256aa74bb2884a0d83f0d17b159b9e49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.194177 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9d18547a0114b83bfbed5110e565661"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.196755 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9d18547a0114b83bfbed5110e565661 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.198361 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b (DEFAULT_TABLE table=default.Table_0322ec9fd9f24338bb9196b099f56643 [id=3dac4cc0f73442c9a173f0766e0e9874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.198845 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a5fdab68694761b7493b42a0bf7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.206940 4169 ts_tablet_manager.cc:1939] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.207415 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 453) took 5947 ms. Trace:
I20260811 18:57:26.207545 4169 rpcz_store.cc:276] 0811 18:57:20.260026 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.260121 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:25.982489 (+5722368us) ts_tablet_manager.cc:1145] Deleting tablet 61c8443df21949ff9bdc63fa6aa34108
0811 18:57:25.982508 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.992576 (+ 10068us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.130514 (+137938us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.207388 (+ 76874us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5695530}
I20260811 18:57:26.208176 4169 tablet_replica.cc:330] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:26.208149 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]) successfully deleted
I20260811 18:57:26.208449 4169 raft_consensus.cc:2229] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.208715 4169 raft_consensus.cc:2258] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.209720 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 02bf791926d84522b1099c1242acddcd (DEFAULT_TABLE table=default.Table_bc171757977f4404bb736a03a87b9ecd [id=41452d01375a443ba5c239475692f69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.210182 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02bf791926d84522b1099c1242acddcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.210537 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.210949 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 (DEFAULT_TABLE table=default.Table_c6ed856dda814eb0bd8eb76c2f83c70b [id=ffd5f1ae63224181bb83a8c11717ad3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.211431 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22abc6f8e9ef4c32ab69af31bdaad872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.215492 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e80925854f7a430c93376f6448db3632"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.216524 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e80925854f7a430c93376f6448db3632 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.217891 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.219027 3716 tablet_service.cc:1467] Processing CreateTablet for tablet c282501396154e3088b39691c60cd35b (DEFAULT_TABLE table=default.Table_d7e8b32170ff47eda203b84328064301 [id=f33a076a8b49435d89198c35da0b1831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.219722 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c282501396154e3088b39691c60cd35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.219975 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b5da605dec7141f8aa11be6246b92608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.220729 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5da605dec7141f8aa11be6246b92608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.221266 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb62e05c51bb4f10b1058d170a2202e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.221926 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb62e05c51bb4f10b1058d170a2202e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.226511 5080 raft_consensus.cc:504] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.227257 5080 leader_election.cc:288] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.228148 4169 ts_tablet_manager.cc:1905] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.229704 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e63ee9b9fd4b12ae791b9d5bc2e648" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:26.230060 3744 raft_consensus.cc:3037] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.232908 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b16cb4c1796147fbb10109f52071505e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.234460 3744 raft_consensus.cc:2454] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:26.236297 4163 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:26.236459 4163 ts_tablet_manager.cc:1397] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.333s user 0.007s sys 0.007s
I20260811 18:57:26.240433 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.235340 3824 leader_election.cc:302] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.244968 4162 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.245276 4162 ts_tablet_manager.cc:1397] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.002s
I20260811 18:57:26.246727 4162 raft_consensus.cc:348] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.247138 4162 raft_consensus.cc:374] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.247249 4162 raft_consensus.cc:729] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.247546 4162 consensus_queue.cc:260] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.247987 4167 ts_tablet_manager.cc:1918] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.248127 4167 log.cc:1196] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:26.248600 4167 ts_tablet_manager.cc:1939] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.249148 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 478) took 4749 ms. Trace:
I20260811 18:57:26.249269 4167 rpcz_store.cc:276] 0811 18:57:21.500188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.500471 (+ 283us) service_pool.cc:227] Handling call
0811 18:57:26.109999 (+4609528us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:26.110009 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.203854 (+ 93845us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.247872 (+ 44018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.249125 (+ 1253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4583936}
I20260811 18:57:26.249794 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 480) took 4685 ms. Trace:
I20260811 18:57:26.249900 4167 rpcz_store.cc:276] 0811 18:57:21.564859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.564935 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:26.249606 (+4684671us) ts_tablet_manager.cc:1145] Deleting tablet d8aea8dc31914052832bdc83d638461f
0811 18:57:26.249621 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.249776 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4684217}
I20260811 18:57:26.250073 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 36724a83bf9844b9a27a88ca36a9bf50 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:26.250660 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d8aea8dc31914052832bdc83d638461f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8aea8dc31914052832bdc83d638461f already in progress: creating tablet
W20260811 18:57:26.251204 3678 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.089s (SENT)
I20260811 18:57:26.253587 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7c395f6c4c364b7eb463f78e16675847"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.254989 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c395f6c4c364b7eb463f78e16675847 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.255445 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.256213 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d42d349a68447e39dbdda4dea1dd35b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.256467 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_66121f6795f240ea88c01d7c0ae02e00" } new_table_name: "default.Table_949e95c6313942e5b97401b457b0c160" modify_external_catalogs: true
I20260811 18:57:26.259938 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1811e5f62a714f8388460dfcb1717e81
I20260811 18:57:26.261620 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1811e5f62a714f8388460dfcb1717e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.262630 5080 raft_consensus.cc:2781] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.262789 5080 raft_consensus.cc:686] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:26.263057 5080 consensus_queue.cc:237] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.270849 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 1811e5f62a714f8388460dfcb1717e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.280434 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 481) took 4715 ms. Trace:
I20260811 18:57:26.281366 4167 rpcz_store.cc:276] 0811 18:57:21.565084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.565441 (+ 357us) service_pool.cc:227] Handling call
0811 18:57:26.250236 (+4684795us) ts_tablet_manager.cc:1145] Deleting tablet bf2816cc2dfe45f0815ce50a5a929324
0811 18:57:26.250250 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.280398 (+ 30148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4684261}
W20260811 18:57:26.285741 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf2816cc2dfe45f0815ce50a5a929324 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf2816cc2dfe45f0815ce50a5a929324 already in progress: creating tablet
I20260811 18:57:26.286108 4162 ts_tablet_manager.cc:1428] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.041s user 0.000s sys 0.002s
I20260811 18:57:26.286610 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 482) took 4699 ms. Trace:
I20260811 18:57:26.286805 4167 rpcz_store.cc:276] 0811 18:57:21.587392 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.587482 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:26.282725 (+4695243us) ts_tablet_manager.cc:1145] Deleting tablet ffa8a4a1c0cb4834be501bdd7d468472
0811 18:57:26.286410 (+ 3685us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.286586 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4692253}
I20260811 18:57:26.288249 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 499) took 4237 ms. Trace:
I20260811 18:57:26.288453 4167 rpcz_store.cc:276] 0811 18:57:22.050959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.052357 (+ 1398us) service_pool.cc:227] Handling call
0811 18:57:26.287445 (+4235088us) ts_tablet_manager.cc:1145] Deleting tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
0811 18:57:26.287462 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.288224 (+ 762us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4195274}
W20260811 18:57:26.289140 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ffa8a4a1c0cb4834be501bdd7d468472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa8a4a1c0cb4834be501bdd7d468472 already in progress: creating tablet
I20260811 18:57:26.289398 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 500) took 4238 ms. Trace:
I20260811 18:57:26.289557 4167 rpcz_store.cc:276] 0811 18:57:22.051170 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.052282 (+ 1112us) service_pool.cc:227] Handling call
0811 18:57:26.289071 (+4236789us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:26.289086 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.289374 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4190803}
W20260811 18:57:26.289702 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e already in progress: creating tablet
I20260811 18:57:26.290432 5037 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.290603 5037 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.290807 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7b404e99e9414c67b3cd5aff12203a53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b404e99e9414c67b3cd5aff12203a53 already in progress: creating tablet
I20260811 18:57:26.291397 5037 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.291942 4163 raft_consensus.cc:348] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.292152 4163 raft_consensus.cc:374] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.292227 4163 raft_consensus.cc:729] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.292596 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.293064 3744 raft_consensus.cc:2454] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.293597 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6914252b8edc40e79af29a5145201b08" } modify_external_catalogs: true
I20260811 18:57:26.293696 3824 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.293895 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6914252b8edc40e79af29a5145201b08" } modify_external_catalogs: true
I20260811 18:57:26.294159 5037 raft_consensus.cc:2781] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.294241 5037 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.294329 5037 raft_consensus.cc:3037] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.290208 4167 tablet_replica.cc:330] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.295027 4167 raft_consensus.cc:2229] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.295323 4167 raft_consensus.cc:2258] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.297569 4167 ts_tablet_manager.cc:1905] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.292436 4163 consensus_queue.cc:260] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.304172 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7db24c0702d141f08ea7023227018d81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.304881 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7db24c0702d141f08ea7023227018d81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.307785 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e5ca40ed9be45ee85dd2681b4523778" } new_table_name: "default.Table_fcb655a52f4449839a4919d317bc6cfe" modify_external_catalogs: true
I20260811 18:57:26.300107 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.314455 4162 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.327785 4162 tablet_bootstrap.cc:654] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.338469 5113 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.338699 5113 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.339461 5113 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.344767 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.326717 5037 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.350900 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1ceb39258a049f08a1f48c40df65d8b" } new_table_name: "default.Table_61b2dc398e804cdabeda17315814c849" modify_external_catalogs: true
I20260811 18:57:26.352226 4162 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.352402 4162 ts_tablet_manager.cc:1397] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.005s
I20260811 18:57:26.352735 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e3a4280bd164399a5346ffaa6f40b8f" } new_table_name: "default.Table_8e407f239f804e5cbd2c9ecbc2c170c5" modify_external_catalogs: true
I20260811 18:57:26.354249 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_07499669fc704a31830a06152cd615cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.354869 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07499669fc704a31830a06152cd615cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.356482 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_41c44c966ece466eb232aeab834c5fe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.357012 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41c44c966ece466eb232aeab834c5fe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.358795 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_88a784338b5e4928be15e4679b3f3304" } new_table_name: "default.Table_7309030d82da426ab87ff64346efd25b" modify_external_catalogs: true
W20260811 18:57:26.360303 3676 leader_election.cc:341] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:26.360249 4162 raft_consensus.cc:348] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.365687 4167 ts_tablet_manager.cc:1918] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.370029 4167 log.cc:1196] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:26.371028 4167 ts_tablet_manager.cc:1939] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.373119 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 502) took 4229 ms. Trace:
I20260811 18:57:26.377923 4167 rpcz_store.cc:276] 0811 18:57:22.143771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.148904 (+ 5133us) service_pool.cc:227] Handling call
0811 18:57:26.290151 (+4141247us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:26.290163 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.335909 (+ 45746us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.365536 (+ 29627us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.373094 (+ 7558us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4113185}
I20260811 18:57:26.379417 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 94943cfbe41a47b3a36c81957a3adaeb (table default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]) successfully deleted
I20260811 18:57:26.380623 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 506) took 4233 ms. Trace:
I20260811 18:57:26.381083 4167 rpcz_store.cc:276] 0811 18:57:22.146453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.149317 (+ 2864us) service_pool.cc:227] Handling call
0811 18:57:26.379498 (+4230181us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:26.379518 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.379764 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4201885}
W20260811 18:57:26.382604 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6432abd4fc5c4f9ca95971ca4795e1cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6432abd4fc5c4f9ca95971ca4795e1cf already in progress: creating tablet
I20260811 18:57:26.382905 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 507) took 4236 ms. Trace:
I20260811 18:57:26.383265 5037 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.384517 4167 rpcz_store.cc:276] 0811 18:57:22.146633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.149032 (+ 2399us) service_pool.cc:227] Handling call
0811 18:57:26.382683 (+4233651us) ts_tablet_manager.cc:1145] Deleting tablet 9ec4bba841dd446aa3db751358ce7f32
0811 18:57:26.382701 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.382879 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4200737}
W20260811 18:57:26.385951 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9ec4bba841dd446aa3db751358ce7f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ec4bba841dd446aa3db751358ce7f32 already in progress: creating tablet
I20260811 18:57:26.386157 4167 tablet_replica.cc:330] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.388224 4167 raft_consensus.cc:2229] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.389539 4167 raft_consensus.cc:2258] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.392019 4167 ts_tablet_manager.cc:1905] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:26.392354 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4da53cef719842f98d2190e487e8d653 consensus state has no leader
I20260811 18:57:26.393170 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69803efc857941408d32f7da625acfb6" } modify_external_catalogs: true
I20260811 18:57:26.393602 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69803efc857941408d32f7da625acfb6" } modify_external_catalogs: true
I20260811 18:57:26.395334 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" } modify_external_catalogs: true
I20260811 18:57:26.395859 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" } modify_external_catalogs: true
W20260811 18:57:26.397795 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4059e5907b1e488ab9e1d04e563f96cd consensus state has no leader
I20260811 18:57:26.398934 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_709cd55123504b35877d469f96354d28" } modify_external_catalogs: true
I20260811 18:57:26.399261 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_709cd55123504b35877d469f96354d28" } modify_external_catalogs: true
I20260811 18:57:26.401479 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 335fc15ea75d483990e6ae5a5872682f
I20260811 18:57:26.405035 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fc15ea75d483990e6ae5a5872682f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.410542 5113 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.410754 5113 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.411655 5113 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.412736 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.413908 3676 leader_election.cc:341] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:26.416000 4167 ts_tablet_manager.cc:1918] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:26.428194 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85140a3800f34a23b716703fd4e73d17 consensus state has no leader
I20260811 18:57:26.419934 5113 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.428850 4167 log.cc:1196] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e51a08e6dbf04e3fa58689ee0cd45db3
W20260811 18:57:26.428872 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9af9fb059984c9680e78b68120b3726 has no consensus state
I20260811 18:57:26.428884 5113 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.429467 4167 ts_tablet_manager.cc:1939] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.429769 5113 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.434713 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 505) took 4288 ms. Trace:
I20260811 18:57:26.433909 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.434991 3891 raft_consensus.cc:2454] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:26.435751 3676 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.437335 5113 raft_consensus.cc:2781] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.437441 5113 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.437521 5113 raft_consensus.cc:3037] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.439190 4167 rpcz_store.cc:276] 0811 18:57:22.146278 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.155537 (+ 9259us) service_pool.cc:227] Handling call
0811 18:57:26.386083 (+4230546us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:26.386100 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.412228 (+ 26128us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.415879 (+ 3651us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.434675 (+ 18796us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4202584}
I20260811 18:57:26.440658 4167 tablet_replica.cc:330] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.440934 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e51a08e6dbf04e3fa58689ee0cd45db3 (table default.Table_acb61e38f2a0442688865698892cc5c9 [id=3c7daa88956747719fb78e9860e7039f]) successfully deleted
I20260811 18:57:26.441514 4167 raft_consensus.cc:2229] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.442142 4167 raft_consensus.cc:2258] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.442044 5113 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.444883 5113 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.445005 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:26.445283 3891 raft_consensus.cc:3037] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.445746 4167 ts_tablet_manager.cc:1905] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.455417 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 705605cd929940f8a252b6e95abbeb95
I20260811 18:57:26.456539 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 705605cd929940f8a252b6e95abbeb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:26.456740 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c3ac1bc93404622a9b6a74aea8b7c1c has no consensus state
I20260811 18:57:26.469676 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" } modify_external_catalogs: true
I20260811 18:57:26.470062 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" } modify_external_catalogs: true
I20260811 18:57:26.471695 5113 consensus_queue.cc:1035] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.473007 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
I20260811 18:57:26.474332 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.484606 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5eb58df7cc2042f69300d10a7999719b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.485841 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5eb58df7cc2042f69300d10a7999719b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.489389 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17690ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:26.489215 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f1e69da8480425682a8348cad2ffa51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.489935 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f1e69da8480425682a8348cad2ffa51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#27 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
I20260811 18:57:26.506853 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f6cd18a5e464333bbdc36290a29eaea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.508033 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f6cd18a5e464333bbdc36290a29eaea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.509892 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9df3de8d1ff649e8a41bff8b3de9d92f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.510593 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9df3de8d1ff649e8a41bff8b3de9d92f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.513814 4162 raft_consensus.cc:374] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.513944 4162 raft_consensus.cc:729] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.514268 4162 consensus_queue.cc:260] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.516850 4162 ts_tablet_manager.cc:1428] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.164s user 0.023s sys 0.000s
I20260811 18:57:26.517251 4162 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.518878 4162 tablet_bootstrap.cc:654] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.520721 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.523128 4167 ts_tablet_manager.cc:1918] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.523415 4167 log.cc:1196] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:26.523973 4167 ts_tablet_manager.cc:1939] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:26.524390 3678 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.244s (SENT)
I20260811 18:57:26.525079 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 503) took 4379 ms. Trace:
I20260811 18:57:26.525264 4167 rpcz_store.cc:276] 0811 18:57:22.145857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.155446 (+ 9589us) service_pool.cc:227] Handling call
0811 18:57:26.440604 (+4285158us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:26.440617 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.467831 (+ 27214us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.523003 (+ 55172us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.525057 (+ 2054us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4256681}
I20260811 18:57:26.526090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 501) took 4388 ms. Trace:
I20260811 18:57:26.526257 4167 rpcz_store.cc:276] 0811 18:57:22.137769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.148492 (+ 10723us) service_pool.cc:227] Handling call
0811 18:57:26.525957 (+4377465us) ts_tablet_manager.cc:1145] Deleting tablet 7991dabe263b44f6b7b7d1d43b96ee44
0811 18:57:26.525969 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.526073 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4339300}
I20260811 18:57:26.526683 4167 tablet_replica.cc:330] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.526985 4167 raft_consensus.cc:2229] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.527402 4167 raft_consensus.cc:2258] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.527645 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 828993fb34bb4c62bce2a9322d7c992a (table default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]) successfully deleted
W20260811 18:57:26.528101 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7991dabe263b44f6b7b7d1d43b96ee44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7991dabe263b44f6b7b7d1d43b96ee44 already in progress: creating tablet
#28 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
I20260811 18:57:26.538668 3891 raft_consensus.cc:2454] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:26.538980 4163 ts_tablet_manager.cc:1428] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.302s user 0.003s sys 0.012s
I20260811 18:57:26.542469 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_035060c2fcfd476196e06c23978f156f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.545209 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b50aff06b9214a069e8b74b23089e9bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.539453 3676 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:26.551908 3678 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.456s (SENT)
I20260811 18:57:26.553133 4163 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:26.553191 5091 raft_consensus.cc:2781] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.553332 5091 raft_consensus.cc:686] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:26.553565 5091 consensus_queue.cc:237] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.553999 4167 ts_tablet_manager.cc:1905] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:26.576776 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b50aff06b9214a069e8b74b23089e9bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.577690 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_035060c2fcfd476196e06c23978f156f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.591111 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 242915us
I20260811 18:57:26.593034 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:26.593281 3744 raft_consensus.cc:3037] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.597357 3744 raft_consensus.cc:2454] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:26.601552 4162 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.601728 4162 ts_tablet_manager.cc:1397] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.000s
I20260811 18:57:26.601831 4163 tablet_bootstrap.cc:654] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.599820 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fc15ea75d483990e6ae5a5872682f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.602551 4162 raft_consensus.cc:348] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.602802 4162 raft_consensus.cc:374] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.602867 4162 raft_consensus.cc:729] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.603081 4162 consensus_queue.cc:260] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.603827 4162 ts_tablet_manager.cc:1428] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:26.604130 4162 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.605494 4162 tablet_bootstrap.cc:654] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#29 0x7fc7af50f61e in clone (/lib/x86_64-linux-gnu/libc.so.6+0x12161e)
0x6190000bf988 is located 8 bytes inside of 1048-byte region [0x6190000bf980,0x6190000bfd98)
freed by thread T137 (tablet-delete [) here:
I20260811 18:57:26.613170 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a795d5395cc64155b84057a53c701064" } new_table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" modify_external_catalogs: true
I20260811 18:57:26.620214 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1204e0b30877462982c245f439f0bae4" } new_table_name: "default.Table_e425950244124732b380187898a2b68f" modify_external_catalogs: true
#0 0x8c4680 in operator delete(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:160:3
I20260811 18:57:26.641196 4162 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.641372 4162 ts_tablet_manager.cc:1397] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.001s
#1 0x7fc7bfa20d2c in std::_Sp_counted_ptr<kudu::tablet::Tablet*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() ../../../include/c++/8/bits/shared_ptr_base.h:377:9
I20260811 18:57:26.642488 4162 raft_consensus.cc:348] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.642930 4162 raft_consensus.cc:374] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.643013 4162 raft_consensus.cc:729] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.643146 4167 ts_tablet_manager.cc:1918] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:26.643335 4167 log.cc:1196] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:26.643287 4162 consensus_queue.cc:260] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.643896 4167 ts_tablet_manager.cc:1939] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.644846 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 508) took 4378 ms. Trace:
I20260811 18:57:26.644991 4167 rpcz_store.cc:276] 0811 18:57:22.266101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.267499 (+ 1398us) service_pool.cc:227] Handling call
0811 18:57:26.526620 (+4259121us) ts_tablet_manager.cc:1145] Deleting tablet 0ace00158b744bd8a97dc1c47fc6049c
0811 18:57:26.526645 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.590419 (+ 63774us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.643028 (+ 52609us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.644819 (+ 1791us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4222084}
I20260811 18:57:26.643901 4162 ts_tablet_manager.cc:1428] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:26.646808 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0f271e4958af4bfbb65af29a053ac2d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.649590 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_558f1f69ec7547c3b029b4672f04c6e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.650563 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_558f1f69ec7547c3b029b4672f04c6e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.652010 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_102e5367d375402c9800357142894fa0 [id=a067d27ecf9c402db2631f2df5f0fea4]) successfully deleted
I20260811 18:57:26.656530 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_540b89ef88904bb6b163c4b18480746e" } new_table_name: "default.Table_661f1ebd9f4c4cb9be591c5e1c203a86" modify_external_catalogs: true
W20260811 18:57:26.657572 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f271e4958af4bfbb65af29a053ac2d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.664049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 514) took 4324 ms. Trace:
I20260811 18:57:26.664165 4167 rpcz_store.cc:276] 0811 18:57:22.339980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.340052 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:26.645488 (+4305436us) ts_tablet_manager.cc:1145] Deleting tablet 2605007cc2454599868f44c717deb2d2
0811 18:57:26.663844 (+ 18356us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.664018 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4304931}
I20260811 18:57:26.666200 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_365efecf4f8e4f4993ba5579614a3049" } modify_external_catalogs: true
I20260811 18:57:26.666594 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_365efecf4f8e4f4993ba5579614a3049" } modify_external_catalogs: true
W20260811 18:57:26.667969 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2605007cc2454599868f44c717deb2d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2605007cc2454599868f44c717deb2d2 already in progress: creating tablet
W20260811 18:57:26.668823 3678 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.472s (SENT)
I20260811 18:57:26.672760 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 516) took 4331 ms. Trace:
I20260811 18:57:26.672923 4167 rpcz_store.cc:276] 0811 18:57:22.341791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.341838 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:26.664619 (+4322781us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:26.672573 (+ 7954us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.672734 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4322392}
W20260811 18:57:26.673599 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3263da75f0d84528b7c8cc55dcaeaa46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3263da75f0d84528b7c8cc55dcaeaa46 already in progress: creating tablet
I20260811 18:57:26.677662 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 517) took 4317 ms. Trace:
I20260811 18:57:26.677857 4167 rpcz_store.cc:276] 0811 18:57:22.360023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.360094 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:26.673435 (+4313341us) ts_tablet_manager.cc:1145] Deleting tablet e14f8717bcd94d6d88295fd8be130ca5
0811 18:57:26.677502 (+ 4067us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.677640 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4313044}
I20260811 18:57:26.678563 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 521) took 4251 ms. Trace:
I20260811 18:57:26.678632 4162 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.678727 4167 rpcz_store.cc:276] 0811 18:57:22.426942 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.427011 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:26.678455 (+4251444us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:26.678465 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.678545 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4250551}
W20260811 18:57:26.680333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a847581412494e34b2c76e605ff11772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a847581412494e34b2c76e605ff11772 already in progress: creating tablet
I20260811 18:57:26.680465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 522) took 4252 ms. Trace:
I20260811 18:57:26.680546 4167 rpcz_store.cc:276] 0811 18:57:22.428289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.428347 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:26.680351 (+4252004us) ts_tablet_manager.cc:1145] Deleting tablet c6e9da8dfb4a4c6c9275c092fae7d910
0811 18:57:26.680363 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.680450 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4251628}
I20260811 18:57:26.681011 4167 tablet_replica.cc:330] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.681052 4162 tablet_bootstrap.cc:654] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.681212 4167 raft_consensus.cc:2229] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:26.681274 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c6e9da8dfb4a4c6c9275c092fae7d910 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6e9da8dfb4a4c6c9275c092fae7d910 already in progress: creating tablet
I20260811 18:57:26.681500 4167 raft_consensus.cc:2258] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.682171 4167 ts_tablet_manager.cc:1905] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.684201 5113 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.684404 5113 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.685559 5113 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.686478 5113 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.686669 5113 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.688719 5113 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:26.697396 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50af4e9b20a74ed983b26281c3c6eda1 consensus state has no leader
I20260811 18:57:26.698294 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b9115c298dfa4ecb970e76c7de2cd8e9" } new_table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" modify_external_catalogs: true
W20260811 18:57:26.700102 3678 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.746s (SENT)
W20260811 18:57:26.700119 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6d988ef7e9dd43158d107998ea4d82b9 consensus state has no leader
I20260811 18:57:26.700949 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4f7f1249bb248359c032a4e20f7c9b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.703142 4162 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.703315 4162 ts_tablet_manager.cc:1397] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.005s
I20260811 18:57:26.704293 4162 raft_consensus.cc:348] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.704573 4162 raft_consensus.cc:374] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.704628 4162 raft_consensus.cc:729] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.705623 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9693b748bd0490cb416e0cb36a62966" } modify_external_catalogs: true
I20260811 18:57:26.706027 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9693b748bd0490cb416e0cb36a62966" } modify_external_catalogs: true
I20260811 18:57:26.707093 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.707391 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:26.702199 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4f7f1249bb248359c032a4e20f7c9b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.707391 4162 consensus_queue.cc:260] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.708518 4162 ts_tablet_manager.cc:1428] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260811 18:57:26.717991 4162 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.719295 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.720160 4162 tablet_bootstrap.cc:654] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.720217 3824 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.720789 5133 raft_consensus.cc:2781] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.720966 5133 raft_consensus.cc:686] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:26.721496 5133 consensus_queue.cc:237] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.727643 3678 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:26.728232 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 705605cd929940f8a252b6e95abbeb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.729420 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.729898 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.731333 3676 leader_election.cc:341] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.731593 3676 leader_election.cc:341] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.748888 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
I20260811 18:57:26.752213 4167 ts_tablet_manager.cc:1918] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.752380 4167 log.cc:1196] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:26.752607 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fb62e05c51bb4f10b1058d170a2202e6" } new_table_name: "default.Table_0790be2c84c14fb3aeb2add67667646e" modify_external_catalogs: true
I20260811 18:57:26.752840 4167 ts_tablet_manager.cc:1939] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.757105 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2962915ef36940f8b6dee3a420d74c59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.758023 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2962915ef36940f8b6dee3a420d74c59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.759935 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e695533c42647128860db601dcc0b5f" } modify_external_catalogs: true
I20260811 18:57:26.760254 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e695533c42647128860db601dcc0b5f" } modify_external_catalogs: true
W20260811 18:57:26.760957 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6009c9f6bbb0493ebe125120a1a47f3f consensus state has no leader
I20260811 18:57:26.761456 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_69f270a4181547b2a2cc604a814c82c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.762213 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69f270a4181547b2a2cc604a814c82c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.777087 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 524) took 4334 ms. Trace:
I20260811 18:57:26.777261 4167 rpcz_store.cc:276] 0811 18:57:22.442551 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.454522 (+ 11971us) service_pool.cc:227] Handling call
0811 18:57:26.680963 (+4226441us) ts_tablet_manager.cc:1145] Deleting tablet 494939a4eb5a45d6a123d174ad513bcb
0811 18:57:26.680974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.722214 (+ 41240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.752053 (+ 29839us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.777047 (+ 24994us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4225294}
I20260811 18:57:26.784061 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdd7e4b15384ec8844f2e31a5496d8a (DEFAULT_TABLE table=default.Table_85a94fea83f84948a14090e2deeac956 [id=c844a598a3c1437cbcdafa534b77ede7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.784583 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdd7e4b15384ec8844f2e31a5496d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.784811 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 494939a4eb5a45d6a123d174ad513bcb (table default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]) successfully deleted
I20260811 18:57:26.784898 3715 tablet_service.cc:1467] Processing CreateTablet for tablet c4ffd3f7c4274466bb5bde604ecf720a (DEFAULT_TABLE table=default.Table_bcb9560450d14ad392e0db4330f60317 [id=24c96cb40e7541b2a65a2b94553f009b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.785286 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ffd3f7c4274466bb5bde604ecf720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.786670 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 09366eccf38a4876840cc5ddddbbb103 (DEFAULT_TABLE table=default.Table_d7e8ba858b0c44d588499e1463673957 [id=307c31b5346949adb9b0c0db5da22ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.787081 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09366eccf38a4876840cc5ddddbbb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.788575 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.788932 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.789505 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:26.790771 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 (DEFAULT_TABLE table=default.Table_e89523faca1046669de08be2f6b963dd [id=10d38692f62648acb9248d2668bdc173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.791206 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c50dcffac54c7bac210eeaed340ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.792320 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 538) took 4224 ms. Trace:
I20260811 18:57:26.792423 4167 rpcz_store.cc:276] 0811 18:57:22.567767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.569699 (+ 1932us) service_pool.cc:227] Handling call
0811 18:57:26.792174 (+4222475us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:26.792186 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.792296 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164385}
I20260811 18:57:26.792907 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f24c8db84f04484584ad1ba5d97f96d4 (DEFAULT_TABLE table=default.Table_a905052284004a77adf260546090e666 [id=dd14415dd4a448178cb65d59d9deabc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.793244 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f24c8db84f04484584ad1ba5d97f96d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.794644 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b9af9fb059984c9680e78b68120b3726 (DEFAULT_TABLE table=default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.795013 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9af9fb059984c9680e78b68120b3726. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.796340 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 452ee450778e4d33adfa7e2745b5d970 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 452ee450778e4d33adfa7e2745b5d970 already in progress: creating tablet
I20260811 18:57:26.797175 4167 tablet_replica.cc:330] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.797398 4167 raft_consensus.cc:2229] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.798632 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f2255ff17edd4b81b18423473cc52218 (DEFAULT_TABLE table=default.Table_1cd8ec7574294bc6886f42d65935c0a5 [id=5b789b53abce4da480695c1f1c0562c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.799150 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2255ff17edd4b81b18423473cc52218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.802552 4167 raft_consensus.cc:2258] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.802778 3722 tablet_service.cc:1467] Processing CreateTablet for tablet fd2baeed833d4a62adc2b89e71050c79 (DEFAULT_TABLE table=default.Table_86e4bb4e07694c3a96ff484bd7944fd3 [id=6ae4c8c6cd59421ab00af37326d7e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.803195 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2baeed833d4a62adc2b89e71050c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.803267 4167 ts_tablet_manager.cc:1905] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.806926 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 00645f522efa4076832758b48fe566f3 (DEFAULT_TABLE table=default.Table_b08e57f933b24cd0aee53441350e6775 [id=c920e237cc6c4dfeac73235ddcff872a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.807353 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00645f522efa4076832758b48fe566f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.807575 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 69059a0667234a408cc1c09585df63bc (DEFAULT_TABLE table=default.Table_614252f40e34465080bb490a1a7050ed [id=63acd1bb983b453e910b79f7479cf2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.807871 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69059a0667234a408cc1c09585df63bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.810621 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f (DEFAULT_TABLE table=default.Table_91d4fb4116244d9e8c08ce623bea3a2e [id=e638fe436b5941f5a0633046786e2da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.810725 3706 tablet_service.cc:1467] Processing CreateTablet for tablet a135bd92ab5b4f76bad9b3eedcfada29 (DEFAULT_TABLE table=default.Table_75b5605dc2a14c9daf6d68e7094bcb39 [id=df10702558ae46c3b03adaa4d1af4896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.784487 3724 tablet_service.cc:1467] Processing CreateTablet for tablet a49b8b344177487fb5bd096ee9cdea2f (DEFAULT_TABLE table=default.Table_bd454d73c8da4bf79e68487caa3dd843 [id=654fb49529b84dcab699903b1b482115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.811097 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a135bd92ab5b4f76bad9b3eedcfada29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.811249 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49b8b344177487fb5bd096ee9cdea2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.814723 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 (DEFAULT_TABLE table=default.Table_e1af5a92fcee4bc4898572e55da333b5 [id=a0bf7b47be7e440789b03a33e06b8b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.815097 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c944d5cfd9b3405b8151abfc9f2fd998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.815762 3865 tablet_service.cc:1467] Processing CreateTablet for tablet a49b8b344177487fb5bd096ee9cdea2f (DEFAULT_TABLE table=default.Table_bd454d73c8da4bf79e68487caa3dd843 [id=654fb49529b84dcab699903b1b482115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.816315 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49b8b344177487fb5bd096ee9cdea2f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.818732 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6009c9f6bbb0493ebe125120a1a47f3f (table default.Table_6cc21ee738d3430ead80d93044f49334 [id=f1e29c96194e4ca0bc9b14c0f8811a98], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6009c9f6bbb0493ebe125120a1a47f3f consensus state has no leader
I20260811 18:57:26.819007 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 5d8cf245bdc9410ab460f455bc7f7943 (DEFAULT_TABLE table=default.Table_cb3c2f76034640b398a94e7536595277 [id=118bf5f196cb4e37bfa10e1cf1608d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.819355 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d8cf245bdc9410ab460f455bc7f7943. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.822451 3678 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.658s (SENT)
I20260811 18:57:26.823436 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b38d3b9570467a9128bd7fe73acf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.827379 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdd7e4b15384ec8844f2e31a5496d8a (DEFAULT_TABLE table=default.Table_85a94fea83f84948a14090e2deeac956 [id=c844a598a3c1437cbcdafa534b77ede7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.828016 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdd7e4b15384ec8844f2e31a5496d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.831393 3863 tablet_service.cc:1467] Processing CreateTablet for tablet c4ffd3f7c4274466bb5bde604ecf720a (DEFAULT_TABLE table=default.Table_bcb9560450d14ad392e0db4330f60317 [id=24c96cb40e7541b2a65a2b94553f009b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.831892 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ffd3f7c4274466bb5bde604ecf720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.836920 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 (DEFAULT_TABLE table=default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.837468 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd4ec30acde4585a1c3451927217cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.840221 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.840703 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.842761 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f24c8db84f04484584ad1ba5d97f96d4 (DEFAULT_TABLE table=default.Table_a905052284004a77adf260546090e666 [id=dd14415dd4a448178cb65d59d9deabc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.843148 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f24c8db84f04484584ad1ba5d97f96d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.845225 5113 raft_consensus.cc:482] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.845367 5113 raft_consensus.cc:504] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.846112 5113 leader_election.cc:288] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.846727 3864 tablet_service.cc:1467] Processing CreateTablet for tablet b9af9fb059984c9680e78b68120b3726 (DEFAULT_TABLE table=default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.847244 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9af9fb059984c9680e78b68120b3726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.847286 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08436f33db043b1bc6395cd92102e59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.847930 3676 leader_election.cc:341] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.848366 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 105595us
I20260811 18:57:26.850804 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f2255ff17edd4b81b18423473cc52218 (DEFAULT_TABLE table=default.Table_1cd8ec7574294bc6886f42d65935c0a5 [id=5b789b53abce4da480695c1f1c0562c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.851320 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2255ff17edd4b81b18423473cc52218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.853627 4163 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:26.853816 4163 ts_tablet_manager.cc:1397] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.301s user 0.001s sys 0.005s
I20260811 18:57:26.855120 4163 raft_consensus.cc:348] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.855399 4163 raft_consensus.cc:374] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.855553 4163 raft_consensus.cc:729] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.855862 4163 consensus_queue.cc:260] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.863601 3888 raft_consensus.cc:1261] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:26.864413 5140 consensus_queue.cc:1035] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.868844 4163 ts_tablet_manager.cc:1428] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.000s sys 0.003s
I20260811 18:57:26.869308 4163 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:26.871563 4163 tablet_bootstrap.cc:654] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.871830 4169 ts_tablet_manager.cc:1918] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:26.871973 4169 log.cc:1196] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6381a951a966473c80670b0dd1aff062
I20260811 18:57:26.872426 4169 ts_tablet_manager.cc:1939] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.873077 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 09366eccf38a4876840cc5ddddbbb103 (DEFAULT_TABLE table=default.Table_d7e8ba858b0c44d588499e1463673957 [id=307c31b5346949adb9b0c0db5da22ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.873631 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09366eccf38a4876840cc5ddddbbb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.875691 5080 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.875810 5080 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.876657 5080 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.885032 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 (DEFAULT_TABLE table=default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.885553 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd4ec30acde4585a1c3451927217cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.890614 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f (DEFAULT_TABLE table=default.Table_91d4fb4116244d9e8c08ce623bea3a2e [id=e638fe436b5941f5a0633046786e2da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.891170 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b38d3b9570467a9128bd7fe73acf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.898882 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 00645f522efa4076832758b48fe566f3 (DEFAULT_TABLE table=default.Table_b08e57f933b24cd0aee53441350e6775 [id=c920e237cc6c4dfeac73235ddcff872a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.899422 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00645f522efa4076832758b48fe566f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.899526 4162 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.899683 4162 ts_tablet_manager.cc:1397] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.182s user 0.005s sys 0.001s
I20260811 18:57:26.900435 4162 raft_consensus.cc:348] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.900640 4162 raft_consensus.cc:374] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.900697 4162 raft_consensus.cc:729] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.901047 4162 consensus_queue.cc:260] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.903030 3864 tablet_service.cc:1467] Processing CreateTablet for tablet fd2baeed833d4a62adc2b89e71050c79 (DEFAULT_TABLE table=default.Table_86e4bb4e07694c3a96ff484bd7944fd3 [id=6ae4c8c6cd59421ab00af37326d7e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.903558 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2baeed833d4a62adc2b89e71050c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.904505 4162 ts_tablet_manager.cc:1428] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:26.904899 4162 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.910420 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 (DEFAULT_TABLE table=default.Table_e89523faca1046669de08be2f6b963dd [id=10d38692f62648acb9248d2668bdc173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.911217 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c50dcffac54c7bac210eeaed340ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.911217 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 454) took 6404 ms. Trace:
I20260811 18:57:26.911461 4169 rpcz_store.cc:276] 0811 18:57:20.507042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.507116 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:26.208113 (+5700997us) ts_tablet_manager.cc:1145] Deleting tablet 6381a951a966473c80670b0dd1aff062
0811 18:57:26.208130 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.646569 (+438439us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.871715 (+225146us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.911197 (+ 39482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5700613}
I20260811 18:57:26.912091 4169 tablet_replica.cc:330] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:26.912343 4169 raft_consensus.cc:2229] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.912533 4169 raft_consensus.cc:2258] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.913123 4169 ts_tablet_manager.cc:1905] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.920068 5152 tablet.cc:1722] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:26.920595 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a135bd92ab5b4f76bad9b3eedcfada29 (DEFAULT_TABLE table=default.Table_75b5605dc2a14c9daf6d68e7094bcb39 [id=df10702558ae46c3b03adaa4d1af4896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.921324 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a135bd92ab5b4f76bad9b3eedcfada29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.928146 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 5d8cf245bdc9410ab460f455bc7f7943 (DEFAULT_TABLE table=default.Table_cb3c2f76034640b398a94e7536595277 [id=118bf5f196cb4e37bfa10e1cf1608d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.930784 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.931048 3744 raft_consensus.cc:2454] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.933046 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_4da51fec1a044237ab13da18366dde64 [id=263eb849fd614011a9dd641aeb9126d0]) successfully deleted
I20260811 18:57:26.933918 3856 tablet_service.cc:1467] Processing CreateTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 (DEFAULT_TABLE table=default.Table_e1af5a92fcee4bc4898572e55da333b5 [id=a0bf7b47be7e440789b03a33e06b8b5f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:26.935763 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT) [suppressed 57 similar messages]
W20260811 18:57:26.954841 3678 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.756s (SENT)
I20260811 18:57:26.960258 4162 tablet_bootstrap.cc:654] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.966490 5091 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.966706 5091 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.967590 5091 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
#2 0x8e30e6 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() ../../../include/c++/8/bits/shared_ptr_base.h:155:6
W20260811 18:57:26.974805 3824 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.272s (SENT)
I20260811 18:57:26.974977 3824 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:26.975286 5133 raft_consensus.cc:2726] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:26.975831 3824 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.975905 5133 consensus_queue.cc:1035] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.976492 5133 raft_consensus.cc:2781] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.976426 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 69059a0667234a408cc1c09585df63bc (DEFAULT_TABLE table=default.Table_614252f40e34465080bb490a1a7050ed [id=63acd1bb983b453e910b79f7479cf2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.976593 5133 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.976682 5133 raft_consensus.cc:3037] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.980188 4169 ts_tablet_manager.cc:1918] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.980994 5133 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.982237 5133 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.982544 4169 log.cc:1196] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:26.987035 4169 ts_tablet_manager.cc:1939] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.987249 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.990272 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.989645 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d8cf245bdc9410ab460f455bc7f7943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.994751 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c944d5cfd9b3405b8151abfc9f2fd998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.995030 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da (DEFAULT_TABLE table=default.Table_8a4328549b8d4ad08d0135327263c811 [id=7668053328524584b72ecd40ed55ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.995522 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9608d6ddcad749cd9e72dd6f265db0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.998736 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69059a0667234a408cc1c09585df63bc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.000442 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_120c3bf5084e4fd1a967d4564e474f6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.001859 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2053e3676584227a3ea13c4f912e315"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.003607 4162 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.003826 4162 ts_tablet_manager.cc:1397] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.099s user 0.002s sys 0.002s
I20260811 18:57:27.004252 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.004457 3743 raft_consensus.cc:3037] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.004776 4162 raft_consensus.cc:348] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.004993 4162 raft_consensus.cc:374] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.005074 4162 raft_consensus.cc:729] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.005312 4162 consensus_queue.cc:260] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.005893 4162 ts_tablet_manager.cc:1428] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:27.006202 4162 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.008111 3743 raft_consensus.cc:2454] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.011809 3824 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.012329 5133 raft_consensus.cc:2781] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.012667 5133 raft_consensus.cc:686] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.013010 5133 consensus_queue.cc:237] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.014002 3676 leader_election.cc:341] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.016059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 459) took 6423 ms. Trace:
#3 0x8e2fc1 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() ../../../include/c++/8/bits/shared_ptr_base.h:728:11
I20260811 18:57:27.017678 4169 rpcz_store.cc:276] 0811 18:57:20.593000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.593064 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:26.912040 (+6318976us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:26.912052 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.942202 (+ 30150us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.980061 (+ 37859us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.016035 (+ 35974us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6317618}
W20260811 18:57:27.021130 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2053e3676584227a3ea13c4f912e315 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.018632 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da (DEFAULT_TABLE table=default.Table_8a4328549b8d4ad08d0135327263c811 [id=7668053328524584b72ecd40ed55ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.023488 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9608d6ddcad749cd9e72dd6f265db0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.023794 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b6c70b4163634984bd02dd9b47ffb472 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.024326 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c70b4163634984bd02dd9b47ffb472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.026428 4162 tablet_bootstrap.cc:654] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.032001 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 469) took 6321 ms. Trace:
I20260811 18:57:27.032783 4169 rpcz_store.cc:276] 0811 18:57:20.710381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.717598 (+ 7217us) service_pool.cc:227] Handling call
0811 18:57:27.031461 (+6313863us) ts_tablet_manager.cc:1145] Deleting tablet 91818d0cbe144a67a458a6648968bf5b
0811 18:57:27.031784 (+ 323us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.031964 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6299852}
I20260811 18:57:27.036720 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 (DEFAULT_TABLE table=default.Table_3652770b3e134affa80ff22fdfc1a0a1 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.037312 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27f9dbb52f34687a33c2ebf20e8f780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.034255 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 468) took 6324 ms. Trace:
I20260811 18:57:27.040589 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 36724a83bf9844b9a27a88ca36a9bf50 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:27.041687 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 91818d0cbe144a67a458a6648968bf5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91818d0cbe144a67a458a6648968bf5b already in progress: creating tablet
I20260811 18:57:27.041945 5091 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.042199 5091 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.042853 5091 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.035193 5153 tablet.cc:1722] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:27.044554 5133 consensus_queue.cc:1035] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.045842 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.046069 3678 leader_election.cc:334] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.950s (SENT)
I20260811 18:57:27.046198 3678 leader_election.cc:302] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:27.046396 3678 raft_consensus.cc:2670] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:27.046613 3676 leader_election.cc:341] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.047138 3865 tablet_service.cc:1467] Processing CreateTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 (DEFAULT_TABLE table=default.Table_3652770b3e134affa80ff22fdfc1a0a1 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.047616 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27f9dbb52f34687a33c2ebf20e8f780. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.048398 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15600ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:27.060597 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 37be4f7579fa4ab99ddbacfa80282b42
I20260811 18:57:27.061815 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.065145 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0aee6beb2e70421083841520733087f8
I20260811 18:57:27.066094 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9722572533de436cb9149e6230570382
W20260811 18:57:27.066143 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4f21761b37b42dbb814bd54cca89452 has no consensus state
I20260811 18:57:27.066504 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7251204325dd424192a6f58e18e71c34
I20260811 18:57:27.068069 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 28af2d2c61254b0ca399dddb05e9ee70
I20260811 18:57:27.068181 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.068414 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 9722572533de436cb9149e6230570382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.068292 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69" } new_table_name: "default.Table_de8a2d61a3cc411a97ad8e8df11c7d89" modify_external_catalogs: true
I20260811 18:57:27.071062 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_706c784347b146089c032d699f6a9abe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.071468 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.071852 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.070199 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_035060c2fcfd476196e06c23978f156f" } new_table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" modify_external_catalogs: true
W20260811 18:57:27.072567 3824 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.412s (SENT)
W20260811 18:57:27.072613 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_706c784347b146089c032d699f6a9abe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.072697 3824 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.073504 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.073644 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.073961 4162 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.074046 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.074103 4162 ts_tablet_manager.cc:1397] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.002s
I20260811 18:57:27.074610 5133 raft_consensus.cc:2726] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.061641 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_863d848a099c49edae9fa42c7b7db8ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.075229 4162 raft_consensus.cc:348] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.075507 4162 raft_consensus.cc:374] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.075568 4162 raft_consensus.cc:729] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:27.075578 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_863d848a099c49edae9fa42c7b7db8ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.075805 4162 consensus_queue.cc:260] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.076531 4162 ts_tablet_manager.cc:1428] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:57:27.076844 4162 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.079164 3856 tablet_service.cc:1467] Processing CreateTablet for tablet b6c70b4163634984bd02dd9b47ffb472 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.079731 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c70b4163634984bd02dd9b47ffb472. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.080583 3678 leader_election.cc:334] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.136s (SENT)
I20260811 18:57:27.080796 3678 leader_election.cc:302] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:27.080982 3678 raft_consensus.cc:2670] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.080912 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_068dc550d67d413c916d1384c3cb811c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.081610 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_068dc550d67d413c916d1384c3cb811c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.038028 4169 rpcz_store.cc:276] 0811 18:57:20.710084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.710744 (+ 660us) service_pool.cc:227] Handling call
0811 18:57:27.034030 (+6323286us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:27.034044 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.034228 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6299001}
I20260811 18:57:27.090950 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 470) took 6380 ms. Trace:
I20260811 18:57:27.091176 4169 rpcz_store.cc:276] 0811 18:57:20.710641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.717651 (+ 7010us) service_pool.cc:227] Handling call
0811 18:57:27.090780 (+6373129us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:27.090798 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.090926 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6354950}
I20260811 18:57:27.091490 4169 tablet_replica.cc:330] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:27.091719 4169 raft_consensus.cc:2229] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:27.091918 4169 raft_consensus.cc:2258] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.092648 4169 ts_tablet_manager.cc:1905] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.094589 4163 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.094753 4163 ts_tablet_manager.cc:1397] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.226s user 0.004s sys 0.000s
I20260811 18:57:27.095631 4163 raft_consensus.cc:348] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.095968 4163 raft_consensus.cc:374] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.096045 4163 raft_consensus.cc:729] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.096606 4163 consensus_queue.cc:260] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.097600 5091 raft_consensus.cc:482] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.097779 5091 raft_consensus.cc:504] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.097702 4163 ts_tablet_manager.cc:1428] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:27.098542 5091 leader_election.cc:288] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.100220 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.100862 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 9722572533de436cb9149e6230570382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.103632 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.104163 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9621bbd0f94223a5d36bc01487f615. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.104121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 43ad6993b63e4bbd98c8cc52f1a53451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43ad6993b63e4bbd98c8cc52f1a53451 already in progress: creating tablet
W20260811 18:57:27.104086 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 44107.8us
W20260811 18:57:27.104530 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b6f36ab6bb774257b5c2692c64eff63d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6f36ab6bb774257b5c2692c64eff63d already in progress: creating tablet
I20260811 18:57:27.109311 4167 ts_tablet_manager.cc:1918] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.109548 4167 log.cc:1196] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/125757b8c1a6448b841c63339f392339
I20260811 18:57:27.114672 4167 ts_tablet_manager.cc:1939] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.114806 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 90576c068ce4468a9c572307dcb79b10 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.116416 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbf6ce167fb348be8d61372791f0cbca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.117192 3676 leader_election.cc:341] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.119216 4163 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.121418 4163 tablet_bootstrap.cc:654] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.122555 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3cc27811ad29451daaebbf31fb3ad198" } new_table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" modify_external_catalogs: true
W20260811 18:57:27.128909 3678 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.058s (SENT)
I20260811 18:57:27.130849 4162 tablet_bootstrap.cc:654] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.132226 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 3c142d8ef40947e29f3c94e1e8e4c517 (DEFAULT_TABLE table=default.Table_b012268c924347d8a87ea52350010dc3 [id=de9e6027bb194adebeba8a4b3253ff9b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.132730 3824 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT)
I20260811 18:57:27.132858 3824 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:27.133042 3824 raft_consensus.cc:2670] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:27.133659 3824 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.104s (SENT)
I20260811 18:57:27.133762 3824 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.134828 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c142d8ef40947e29f3c94e1e8e4c517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.134760 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3c142d8ef40947e29f3c94e1e8e4c517 (DEFAULT_TABLE table=default.Table_b012268c924347d8a87ea52350010dc3 [id=de9e6027bb194adebeba8a4b3253ff9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.135195 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c142d8ef40947e29f3c94e1e8e4c517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.137662 5080 raft_consensus.cc:2726] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.144893 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 539) took 4576 ms. Trace:
I20260811 18:57:27.145567 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.148464 3719 tablet_service.cc:1467] Processing CreateTablet for tablet d6c81850cdc74b4ab727bd2279162477 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.147527 4167 rpcz_store.cc:276] 0811 18:57:22.567943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.569754 (+ 1811us) service_pool.cc:227] Handling call
0811 18:57:26.797127 (+4227373us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:26.797139 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.971057 (+173918us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.109192 (+138135us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.144866 (+ 35674us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4169147}
I20260811 18:57:27.148913 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c81850cdc74b4ab727bd2279162477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.153705 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9621bbd0f94223a5d36bc01487f615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.155922 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bb1cb45730b14b998a8f547983b83823 (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.156378 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1cb45730b14b998a8f547983b83823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.166441 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c (DEFAULT_TABLE table=default.Table_7309030d82da426ab87ff64346efd25b [id=6e7f157b26ec44fba47381cbcaff1f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.167044 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.148437 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 98ad733726594d2a8087f857f155dbe2 (DEFAULT_TABLE table=default.Table_be655558b83041fcace57520a599b8a0 [id=411f1ba1a39c4f10b9aaea7b10799e0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.169090 3856 tablet_service.cc:1467] Processing CreateTablet for tablet bb1cb45730b14b998a8f547983b83823 (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.170117 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1cb45730b14b998a8f547983b83823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.169457 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ad733726594d2a8087f857f155dbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.173800 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 98ad733726594d2a8087f857f155dbe2 (DEFAULT_TABLE table=default.Table_be655558b83041fcace57520a599b8a0 [id=411f1ba1a39c4f10b9aaea7b10799e0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.175377 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c (DEFAULT_TABLE table=default.Table_7309030d82da426ab87ff64346efd25b [id=6e7f157b26ec44fba47381cbcaff1f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.175817 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.176421 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 540) took 4576 ms. Trace:
I20260811 18:57:27.177809 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ad733726594d2a8087f857f155dbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.182492 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3453632ca8274e459b096036508fbb8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.182530 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5d93073a69c143ff9460248848a108d4 has no consensus state
W20260811 18:57:27.184194 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3453632ca8274e459b096036508fbb8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.184851 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:27.185807 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.186546 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a35f3caa50e849abac2fc4d5711ea9fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.188371 3824 consensus_peers.cc:487] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94943cfbe41a47b3a36c81957a3adaeb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:27.192803 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fc31b7c8d824f879807f105af51df48" } new_table_name: "default.Table_fb8dd203bf3941fbacfd7dad1418e566" modify_external_catalogs: true
I20260811 18:57:27.192848 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.195218 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bea4820fb5f4e65acb2a1a36332ac2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.197731 4162 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.197901 4162 ts_tablet_manager.cc:1397] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.121s user 0.003s sys 0.004s
I20260811 18:57:27.199178 4162 raft_consensus.cc:348] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.199437 4162 raft_consensus.cc:374] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.199527 4162 raft_consensus.cc:729] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.199858 4162 consensus_queue.cc:260] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.200672 4162 ts_tablet_manager.cc:1428] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:27.176538 4167 rpcz_store.cc:276] 0811 18:57:22.599852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.613269 (+ 13417us) service_pool.cc:227] Handling call
0811 18:57:27.176267 (+4562998us) ts_tablet_manager.cc:1145] Deleting tablet b20872d80c9545e5822bfc82d496ce97
0811 18:57:27.176280 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.176400 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4536703}
W20260811 18:57:27.205539 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b20872d80c9545e5822bfc82d496ce97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b20872d80c9545e5822bfc82d496ce97 already in progress: creating tablet
W20260811 18:57:27.206488 3824 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.822s (SENT)
I20260811 18:57:27.206656 3824 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.207158 5080 raft_consensus.cc:2726] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.218881 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 90576c068ce4468a9c572307dcb79b10
W20260811 18:57:27.219532 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfea0cd619024204b9bff1e2dd62b7b0 consensus state has no leader
I20260811 18:57:27.219730 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 90576c068ce4468a9c572307dcb79b10 deleted and replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331
W20260811 18:57:27.222589 3678 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.710s (SENT)
I20260811 18:57:27.226778 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 90576c068ce4468a9c572307dcb79b10 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.227815 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet b16cb4c1796147fbb10109f52071505e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.232724 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 90576c068ce4468a9c572307dcb79b10 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.177947 5133 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.233183 5133 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.234059 5133 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.177938 3864 tablet_service.cc:1467] Processing CreateTablet for tablet d6c81850cdc74b4ab727bd2279162477 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.234901 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c81850cdc74b4ab727bd2279162477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.235172 4163 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.235352 4163 ts_tablet_manager.cc:1397] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.116s user 0.002s sys 0.001s
I20260811 18:57:27.236455 4163 raft_consensus.cc:348] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.236718 4163 raft_consensus.cc:374] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.236784 4163 raft_consensus.cc:729] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.240697 4162 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.242202 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.242571 3744 raft_consensus.cc:2454] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.243520 3824 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.243961 5133 raft_consensus.cc:2781] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.244055 5133 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.244168 5133 raft_consensus.cc:3037] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.246183 4162 tablet_bootstrap.cc:654] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.246537 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 545) took 4019 ms. Trace:
I20260811 18:57:27.246634 4167 rpcz_store.cc:276] 0811 18:57:23.227176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.227241 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:27.205980 (+3978739us) ts_tablet_manager.cc:1145] Deleting tablet 21cc1822d4b94081ad86a6177acb3afe
0811 18:57:27.246383 (+ 40403us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.246516 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3952564}
W20260811 18:57:27.247275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 21cc1822d4b94081ad86a6177acb3afe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21cc1822d4b94081ad86a6177acb3afe already in progress: creating tablet
I20260811 18:57:27.251029 5133 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.251963 5133 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:27.253041 3678 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.332s (SENT)
I20260811 18:57:27.280051 4163 consensus_queue.cc:260] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.281265 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.281569 3744 raft_consensus.cc:3037] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.285290 3744 raft_consensus.cc:2454] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.286269 4163 ts_tablet_manager.cc:1428] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.051s user 0.002s sys 0.002s
I20260811 18:57:27.287370 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 546) took 4039 ms. Trace:
I20260811 18:57:27.287967 4167 rpcz_store.cc:276] 0811 18:57:23.248237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.248325 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:27.247672 (+3999347us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:27.281126 (+ 33454us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.287346 (+ 6220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3993288}
I20260811 18:57:27.288729 4167 tablet_replica.cc:330] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:27.288821 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4b6ec419ed7747c3b24dc11992518e88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b6ec419ed7747c3b24dc11992518e88 already in progress: creating tablet
I20260811 18:57:27.289067 4167 raft_consensus.cc:2229] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Raft consensus shutting down.
I20260811 18:57:27.289479 4167 raft_consensus.cc:2258] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.290518 4167 ts_tablet_manager.cc:1905] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.287140 3824 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.291846 5133 raft_consensus.cc:2781] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.292058 5133 raft_consensus.cc:686] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.292330 5133 consensus_queue.cc:237] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.301827 4163 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.329123 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_863d848a099c49edae9fa42c7b7db8ec" } new_table_name: "default.Table_0df46efea56e41b79fbb3e4dd034f84e" modify_external_catalogs: true
I20260811 18:57:27.331370 4162 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.331573 4162 ts_tablet_manager.cc:1397] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
W20260811 18:57:27.333405 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a811d4488d0e4541808a5e1aa44d663b consensus state has no leader
I20260811 18:57:27.333621 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f64f05d1ee44029680c5423fd0d81b" } modify_external_catalogs: true
I20260811 18:57:27.333940 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_82ea06e136674461909baae7b6d4e8cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.334944 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82ea06e136674461909baae7b6d4e8cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.334244 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1bf717c45ad424daa97f656cabd9db8" } modify_external_catalogs: true
I20260811 18:57:27.335729 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1bf717c45ad424daa97f656cabd9db8" } modify_external_catalogs: true
I20260811 18:57:27.336709 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f64f05d1ee44029680c5423fd0d81b" } modify_external_catalogs: true
I20260811 18:57:27.337568 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea791cf775eb426ba289b61dfbaf8b0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.337960 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8" } modify_external_catalogs: true
I20260811 18:57:27.338327 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8" } modify_external_catalogs: true
W20260811 18:57:27.338737 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea791cf775eb426ba289b61dfbaf8b0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.339053 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd454d73c8da4bf79e68487caa3dd843" } modify_external_catalogs: true
W20260811 18:57:27.339351 3824 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.114s (SENT)
I20260811 18:57:27.339345 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd454d73c8da4bf79e68487caa3dd843" } modify_external_catalogs: true
I20260811 18:57:27.339514 3824 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:27.339735 3824 raft_consensus.cc:2670] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.341593 5133 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.341811 5133 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.342902 5091 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.343091 5091 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.343955 5091 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.345072 4167 ts_tablet_manager.cc:1918] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.5
I20260811 18:57:27.345352 4167 log.cc:1196] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:27.345228 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:27.346323 4167 ts_tablet_manager.cc:1939] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.349501 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" } modify_external_catalogs: true
I20260811 18:57:27.349859 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" } modify_external_catalogs: true
I20260811 18:57:27.288821 4169 ts_tablet_manager.cc:1918] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.350476 4169 log.cc:1196] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:27.351034 4169 ts_tablet_manager.cc:1939] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:27.352057 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 471) took 6616 ms. Trace:
I20260811 18:57:27.359103 4169 rpcz_store.cc:276] 0811 18:57:20.736065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.736434 (+ 369us) service_pool.cc:227] Handling call
0811 18:57:27.091443 (+6355009us) ts_tablet_manager.cc:1145] Deleting tablet a732c700ff79432aaf1f0d1c1e90aabe
0811 18:57:27.091454 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.232484 (+141030us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.288707 (+ 56223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.352035 (+ 63328us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6354638}
I20260811 18:57:27.353370 5133 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.352907 4163 tablet_bootstrap.cc:654] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.361496 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 472) took 6625 ms. Trace:
I20260811 18:57:27.361493 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a732c700ff79432aaf1f0d1c1e90aabe (table default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]) successfully deleted
I20260811 18:57:27.363900 4169 rpcz_store.cc:276] 0811 18:57:20.736358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.736861 (+ 503us) service_pool.cc:227] Handling call
0811 18:57:27.361341 (+6624480us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:27.361355 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.361474 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6623988}
W20260811 18:57:27.346760 3676 leader_election.cc:341] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.353053 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.358752 4162 raft_consensus.cc:348] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.364318 4162 raft_consensus.cc:374] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.364337 3744 raft_consensus.cc:2454] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.364442 4162 raft_consensus.cc:729] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.364775 4162 consensus_queue.cc:260] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.367753 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 547) took 4005 ms. Trace:
I20260811 18:57:27.367908 4167 rpcz_store.cc:276] 0811 18:57:23.362368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.362503 (+ 135us) service_pool.cc:227] Handling call
0811 18:57:27.288669 (+3926166us) ts_tablet_manager.cc:1145] Deleting tablet 9de9acdb33a2438eb4378a5eac844c35
0811 18:57:27.288685 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.311115 (+ 22430us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.342524 (+ 31409us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.367729 (+ 25205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3911552}
I20260811 18:57:27.368649 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 560) took 3643 ms. Trace:
I20260811 18:57:27.368813 4167 rpcz_store.cc:276] 0811 18:57:23.725273 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725327 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:27.368519 (+3643192us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:27.368531 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.368631 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3640473}
I20260811 18:57:27.369323 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]) successfully deleted
I20260811 18:57:27.369415 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 559) took 3644 ms. Trace:
I20260811 18:57:27.369480 4167 rpcz_store.cc:276] 0811 18:57:23.725115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725510 (+ 395us) service_pool.cc:227] Handling call
0811 18:57:27.369289 (+3643779us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:27.369314 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.369398 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3641196}
W20260811 18:57:27.369719 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5d495354c0994fb4a74c376078e41a26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d495354c0994fb4a74c376078e41a26 already in progress: creating tablet
I20260811 18:57:27.369931 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 558) took 3645 ms. Trace:
W20260811 18:57:27.370011 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe203387bc9c45ba82afe6b058aa5e5f already in progress: creating tablet
I20260811 18:57:27.370015 4167 rpcz_store.cc:276] 0811 18:57:23.724895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.726121 (+ 1226us) service_pool.cc:227] Handling call
0811 18:57:27.369835 (+3643714us) ts_tablet_manager.cc:1145] Deleting tablet d593aea288e14bb1a4c54360b260cffa
0811 18:57:27.369844 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.369915 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3633256}
I20260811 18:57:27.370513 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 561) took 3635 ms. Trace:
I20260811 18:57:27.370654 4167 rpcz_store.cc:276] 0811 18:57:23.734880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.734942 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:27.370411 (+3635469us) ts_tablet_manager.cc:1145] Deleting tablet 1f60a286ac024c45bcb845227c5445c9
0811 18:57:27.370422 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.370498 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3630080}
I20260811 18:57:27.371163 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 562) took 3624 ms. Trace:
W20260811 18:57:27.371359 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d593aea288e14bb1a4c54360b260cffa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d593aea288e14bb1a4c54360b260cffa already in progress: creating tablet
I20260811 18:57:27.371333 4167 rpcz_store.cc:276] 0811 18:57:23.747449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.747729 (+ 280us) service_pool.cc:227] Handling call
0811 18:57:27.371061 (+3623332us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:27.371073 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.371145 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3616788}
W20260811 18:57:27.371685 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1f60a286ac024c45bcb845227c5445c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f60a286ac024c45bcb845227c5445c9 already in progress: creating tablet
I20260811 18:57:27.371809 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 557) took 3647 ms. Trace:
I20260811 18:57:27.371882 4167 rpcz_store.cc:276] 0811 18:57:23.724741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725922 (+ 1181us) service_pool.cc:227] Handling call
0811 18:57:27.371711 (+3645789us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:27.371722 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.371794 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3614059}
W20260811 18:57:27.371978 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 122b7898fea44ef78f269b1505d8ddc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 122b7898fea44ef78f269b1505d8ddc4 already in progress: creating tablet
W20260811 18:57:27.372400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b3f0cdcee681460084662e01152ad9dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3f0cdcee681460084662e01152ad9dd already in progress: creating tablet
I20260811 18:57:27.372680 4162 ts_tablet_manager.cc:1428] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.041s user 0.001s sys 0.001s
I20260811 18:57:27.364857 3824 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.374969 5133 raft_consensus.cc:2781] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.375121 5133 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.375276 5133 raft_consensus.cc:3037] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.379112 5133 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.380900 5133 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.381639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 476) took 6390 ms. Trace:
W20260811 18:57:27.382087 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 059b325df1db4fde9d4b26b559121316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059b325df1db4fde9d4b26b559121316 already in progress: creating tablet
I20260811 18:57:27.380669 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
W20260811 18:57:27.382951 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13788f40714847ff8ce6a810ac53e398 has no consensus state
I20260811 18:57:27.383076 3744 raft_consensus.cc:3037] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.383757 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1a2751cf08eb4ce0b354486cf5678c1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.384568 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a2751cf08eb4ce0b354486cf5678c1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.386356 3744 raft_consensus.cc:2454] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.382169 4167 tablet_replica.cc:330] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:27.387329 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet f287040b879642448367abe2b1c73cd9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.387460 4167 raft_consensus.cc:2229] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:27.393335 4162 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.386943 3824 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.396147 4169 rpcz_store.cc:276] 0811 18:57:20.990897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.993112 (+ 2215us) service_pool.cc:227] Handling call
0811 18:57:27.381502 (+6388390us) ts_tablet_manager.cc:1145] Deleting tablet bf5749d4eebd42319df1b52eeba4c062
0811 18:57:27.381516 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.381620 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6337970}
I20260811 18:57:27.396518 5133 raft_consensus.cc:2781] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:27.397317 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bf5749d4eebd42319df1b52eeba4c062 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf5749d4eebd42319df1b52eeba4c062 already in progress: creating tablet
I20260811 18:57:27.397709 5133 raft_consensus.cc:686] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.398013 5133 consensus_queue.cc:237] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.404011 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 475) took 6422 ms. Trace:
I20260811 18:57:27.404363 4167 raft_consensus.cc:2258] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.405507 4167 ts_tablet_manager.cc:1905] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.405797 4169 rpcz_store.cc:276] 0811 18:57:20.981891 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.984828 (+ 2937us) service_pool.cc:227] Handling call
0811 18:57:27.403863 (+6419035us) ts_tablet_manager.cc:1145] Deleting tablet 3468b39bd73c4272a84dad37fcfd6f27
0811 18:57:27.403876 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.403983 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6358806}
W20260811 18:57:27.410946 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3468b39bd73c4272a84dad37fcfd6f27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3468b39bd73c4272a84dad37fcfd6f27 already in progress: creating tablet
I20260811 18:57:27.395829 4163 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.411491 4163 ts_tablet_manager.cc:1397] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.110s user 0.005s sys 0.000s
I20260811 18:57:27.413087 4163 raft_consensus.cc:348] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.409641 5091 consensus_queue.cc:1035] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:27.431411 3678 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.028s (SENT)
I20260811 18:57:27.432832 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f2f1e0bdeb7a45e3b9e6041aa2ac2578" } new_table_name: "default.Table_710d4f427cd6437a908b1109b4a140a1" modify_external_catalogs: true
I20260811 18:57:27.459793 4162 tablet_bootstrap.cc:654] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:27.466661 3678 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.919s (SENT)
I20260811 18:57:27.466831 3678 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:27.466735 4163 raft_consensus.cc:374] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:27.467003 3678 raft_consensus.cc:2670] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.467099 4163 raft_consensus.cc:729] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.467427 4163 consensus_queue.cc:260] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.411939 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 478) took 6315 ms. Trace:
W20260811 18:57:27.440452 3824 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.044s (SENT)
I20260811 18:57:27.473762 3824 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.474330 4169 rpcz_store.cc:276] 0811 18:57:21.096333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.099622 (+ 3289us) service_pool.cc:227] Handling call
0811 18:57:27.411797 (+6312175us) ts_tablet_manager.cc:1145] Deleting tablet fdeec0c5672b4b3190d7d7cf341cb118
0811 18:57:27.411811 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.411920 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6288008}
I20260811 18:57:27.475227 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 477) took 6380 ms. Trace:
I20260811 18:57:27.475459 4169 rpcz_store.cc:276] 0811 18:57:21.094693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.100189 (+ 5496us) service_pool.cc:227] Handling call
0811 18:57:27.475085 (+6374896us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:27.475099 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.475209 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6341728}
I20260811 18:57:27.476116 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 479) took 6152 ms. Trace:
I20260811 18:57:27.476294 4169 rpcz_store.cc:276] 0811 18:57:21.323309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.341308 (+ 17999us) service_pool.cc:227] Handling call
0811 18:57:27.475984 (+6134676us) ts_tablet_manager.cc:1145] Deleting tablet 289e607f10e14a2983bc6ecc93ce9bbc
0811 18:57:27.475996 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.476101 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6133907}
I20260811 18:57:27.477494 5133 raft_consensus.cc:2726] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.479900 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 481) took 5920 ms. Trace:
I20260811 18:57:27.480049 4169 rpcz_store.cc:276] 0811 18:57:21.559061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.559158 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:27.477212 (+5918054us) ts_tablet_manager.cc:1145] Deleting tablet bf2816cc2dfe45f0815ce50a5a929324
0811 18:57:27.477223 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.479880 (+ 2657us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5917583}
I20260811 18:57:27.480603 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 480) took 5921 ms. Trace:
I20260811 18:57:27.480733 4169 rpcz_store.cc:276] 0811 18:57:21.558766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.561626 (+ 2860us) service_pool.cc:227] Handling call
0811 18:57:27.480508 (+5918882us) ts_tablet_manager.cc:1145] Deleting tablet d8aea8dc31914052832bdc83d638461f
0811 18:57:27.480518 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.480590 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5918263}
I20260811 18:57:27.481249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 482) took 5844 ms. Trace:
I20260811 18:57:27.482044 4169 rpcz_store.cc:276] 0811 18:57:21.636943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.637036 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:27.481087 (+5844051us) ts_tablet_manager.cc:1145] Deleting tablet ffa8a4a1c0cb4834be501bdd7d468472
0811 18:57:27.481165 (+ 78us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.481233 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5826083}
I20260811 18:57:27.481779 4163 ts_tablet_manager.cc:1428] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.070s user 0.002s sys 0.000s
W20260811 18:57:27.487529 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
I20260811 18:57:27.488324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47c0e5602f6b4e1793b88727c8bfdaee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.489042 4163 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:27.489658 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47c0e5602f6b4e1793b88727c8bfdaee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.491775 4163 tablet_bootstrap.cc:654] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.493721 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 499) took 5447 ms. Trace:
I20260811 18:57:27.493813 4169 rpcz_store.cc:276] 0811 18:57:22.046379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.046480 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:27.493542 (+5447062us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:27.493552 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.493684 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5430510}
I20260811 18:57:27.489086 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8ad9f329a9c449e4ade20157c2601ee8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.495448 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ad9f329a9c449e4ade20157c2601ee8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.496174 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fdeec0c5672b4b3190d7d7cf341cb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdeec0c5672b4b3190d7d7cf341cb118 already in progress: creating tablet
W20260811 18:57:27.496631 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 42405aec21e3405486b6c658438b5fb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42405aec21e3405486b6c658438b5fb2 already in progress: creating tablet
W20260811 18:57:27.496976 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 289e607f10e14a2983bc6ecc93ce9bbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 289e607f10e14a2983bc6ecc93ce9bbc already in progress: creating tablet
W20260811 18:57:27.497303 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bf2816cc2dfe45f0815ce50a5a929324 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf2816cc2dfe45f0815ce50a5a929324 already in progress: creating tablet
W20260811 18:57:27.497661 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d8aea8dc31914052832bdc83d638461f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8aea8dc31914052832bdc83d638461f already in progress: creating tablet
W20260811 18:57:27.498029 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ffa8a4a1c0cb4834be501bdd7d468472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa8a4a1c0cb4834be501bdd7d468472 already in progress: creating tablet
W20260811 18:57:27.499025 3678 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.960s (SENT)
I20260811 18:57:27.499531 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 419824ebd14e415f90072b8c335f1ad1 (DEFAULT_TABLE table=default.Table_a4f7f1249bb248359c032a4e20f7c9b4 [id=7087e984854645998d78661c89e4503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.499450 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 18c599d945044e0088bc66c8472d2495 (DEFAULT_TABLE table=default.Table_120c3bf5084e4fd1a967d4564e474f6c [id=c6b3d432c45c440f80a484b8220cbef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.500123 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 419824ebd14e415f90072b8c335f1ad1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.502425 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7b404e99e9414c67b3cd5aff12203a53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b404e99e9414c67b3cd5aff12203a53 already in progress: creating tablet
I20260811 18:57:27.503237 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5193d57970f2486795256698cdf51e6a (DEFAULT_TABLE table=default.Table_2962915ef36940f8b6dee3a420d74c59 [id=c69988bcd74d4ac5b0ed3b8ce97ffec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503497 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 18c599d945044e0088bc66c8472d2495 (DEFAULT_TABLE table=default.Table_120c3bf5084e4fd1a967d4564e474f6c [id=c6b3d432c45c440f80a484b8220cbef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503255 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 06519254820241f1bd4bbceb3faedca3 (DEFAULT_TABLE table=default.Table_0f271e4958af4bfbb65af29a053ac2d4 [id=2d9b9e4757454293a104889e19de1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503953 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5193d57970f2486795256698cdf51e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504163 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06519254820241f1bd4bbceb3faedca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504087 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 06519254820241f1bd4bbceb3faedca3 (DEFAULT_TABLE table=default.Table_0f271e4958af4bfbb65af29a053ac2d4 [id=2d9b9e4757454293a104889e19de1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.504513 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18c599d945044e0088bc66c8472d2495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504488 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5193d57970f2486795256698cdf51e6a (DEFAULT_TABLE table=default.Table_2962915ef36940f8b6dee3a420d74c59 [id=c69988bcd74d4ac5b0ed3b8ce97ffec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.514492 3555 catalog_manager.cc:5495] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:27.515898 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4] alter complete (version 4)
I20260811 18:57:27.516661 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9ee42f727a415ab19b78c0e5bd013a" } new_table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" modify_external_catalogs: true
I20260811 18:57:27.520280 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
I20260811 18:57:27.520936 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 419824ebd14e415f90072b8c335f1ad1 (DEFAULT_TABLE table=default.Table_a4f7f1249bb248359c032a4e20f7c9b4 [id=7087e984854645998d78661c89e4503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.521224 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_988443c8d6bd4b06b90d4e7f95a0998c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.522020 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_988443c8d6bd4b06b90d4e7f95a0998c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.524413 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:27.529649 3678 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.161s (SENT)
I20260811 18:57:27.534832 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a49b8b344177487fb5bd096ee9cdea2f
I20260811 18:57:27.535581 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a" } new_table_name: "default.Table_744e4342189a4b29b424eebd6ad7542f" modify_external_catalogs: true
I20260811 18:57:27.538208 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet a49b8b344177487fb5bd096ee9cdea2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.541198 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c21eb02db3de4c97878fa8badccaa6e3
I20260811 18:57:27.542238 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1af253ec487e4e30b462dc9cb3b1b3cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.543176 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1af253ec487e4e30b462dc9cb3b1b3cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.545106 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:27.546130 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 4059e5907b1e488ab9e1d04e563f96cd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.549124 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.549515 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.550599 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.550864 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a49b8b344177487fb5bd096ee9cdea2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.551124 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.551435 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.556875 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 9ccceeb89da34853a768df69e1298457 (DEFAULT_TABLE table=default.Table_4f6cd18a5e464333bbdc36290a29eaea [id=edb6b1a8db364ec089c07b27e22da454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.557524 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ccceeb89da34853a768df69e1298457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.559947 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 78868ff1f8024163ae1d782151100cd4 (DEFAULT_TABLE table=default.Table_9d42d349a68447e39dbdda4dea1dd35b [id=45b2ec435ce94becafdb869c7437640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.560461 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78868ff1f8024163ae1d782151100cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.560506 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 114d22f22d3d4dc09563aa1108e1f509 (DEFAULT_TABLE table=default.Table_256aa74bb2884a0d83f0d17b159b9e49 [id=4ec9375cd8b64a2ba89d8759b6a789b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.560997 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 053d923618464b0c919563b4be9bc118 (DEFAULT_TABLE table=default.Table_e80925854f7a430c93376f6448db3632 [id=d8411ab33e004d1ebe3c9aa92a1d4721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.561676 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 5d93073a69c143ff9460248848a108d4 (DEFAULT_TABLE table=default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562132 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b (DEFAULT_TABLE table=default.Table_8f1e69da8480425682a8348cad2ffa51 [id=79f4456b3f5442c98f7d361d2b6be7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562548 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 78868ff1f8024163ae1d782151100cd4 (DEFAULT_TABLE table=default.Table_9d42d349a68447e39dbdda4dea1dd35b [id=45b2ec435ce94becafdb869c7437640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562734 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 114d22f22d3d4dc09563aa1108e1f509 (DEFAULT_TABLE table=default.Table_256aa74bb2884a0d83f0d17b159b9e49 [id=4ec9375cd8b64a2ba89d8759b6a789b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562937 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9ccceeb89da34853a768df69e1298457 (DEFAULT_TABLE table=default.Table_4f6cd18a5e464333bbdc36290a29eaea [id=edb6b1a8db364ec089c07b27e22da454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.563246 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114d22f22d3d4dc09563aa1108e1f509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.563409 3861 tablet_service.cc:1467] Processing CreateTablet for tablet dd80633f926942e88f54b32d9a03ea3a (DEFAULT_TABLE table=default.Table_f9d18547a0114b83bfbed5110e565661 [id=199e295bad4d4c80b2887b8e9386b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.564203 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18c599d945044e0088bc66c8472d2495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.566243 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 053d923618464b0c919563b4be9bc118 (DEFAULT_TABLE table=default.Table_e80925854f7a430c93376f6448db3632 [id=d8411ab33e004d1ebe3c9aa92a1d4721]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.566538 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17247ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:27.566645 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06519254820241f1bd4bbceb3faedca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.566751 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053d923618464b0c919563b4be9bc118. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.569689 3678 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.894s (SENT)
I20260811 18:57:27.570926 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5193d57970f2486795256698cdf51e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.570832 3722 tablet_service.cc:1467] Processing CreateTablet for tablet dd80633f926942e88f54b32d9a03ea3a (DEFAULT_TABLE table=default.Table_f9d18547a0114b83bfbed5110e565661 [id=199e295bad4d4c80b2887b8e9386b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.571333 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd80633f926942e88f54b32d9a03ea3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.574668 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111),
I20260811 18:57:27.575019 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:27.575610 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114d22f22d3d4dc09563aa1108e1f509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.578735 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053d923618464b0c919563b4be9bc118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.582653 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d93073a69c143ff9460248848a108d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.586529 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1342b8cd65b4c31a47b27a79c678d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.591790 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78868ff1f8024163ae1d782151100cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.594761 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ccceeb89da34853a768df69e1298457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.598587 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd80633f926942e88f54b32d9a03ea3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.598500 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5ab42c67414f6699e48987979dfa68 (DEFAULT_TABLE table=default.Table_b5da605dec7141f8aa11be6246b92608 [id=f641203ddd0348a39947fb306dd07cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.599032 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5ab42c67414f6699e48987979dfa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.602828 3723 tablet_service.cc:1467] Processing CreateTablet for tablet e7d75d8468fc4c938fde0b2951d84562 (DEFAULT_TABLE table=default.Table_07499669fc704a31830a06152cd615cf [id=f9c87a8d5bca46369a27d34a7cdea6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.603379 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d75d8468fc4c938fde0b2951d84562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.606778 3709 tablet_service.cc:1467] Processing CreateTablet for tablet c80955bd879c4c1eb16b07ad38096259 (DEFAULT_TABLE table=default.Table_69f270a4181547b2a2cc604a814c82c7 [id=74ef6dbec821458e88361f3cea2175e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.607095 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5ab42c67414f6699e48987979dfa68 (DEFAULT_TABLE table=default.Table_b5da605dec7141f8aa11be6246b92608 [id=f641203ddd0348a39947fb306dd07cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.607448 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80955bd879c4c1eb16b07ad38096259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.608086 3862 tablet_service.cc:1467] Processing CreateTablet for tablet e7d75d8468fc4c938fde0b2951d84562 (DEFAULT_TABLE table=default.Table_07499669fc704a31830a06152cd615cf [id=f9c87a8d5bca46369a27d34a7cdea6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.608865 3858 tablet_service.cc:1467] Processing CreateTablet for tablet c80955bd879c4c1eb16b07ad38096259 (DEFAULT_TABLE table=default.Table_69f270a4181547b2a2cc604a814c82c7 [id=74ef6dbec821458e88361f3cea2175e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.609658 3872 tablet_service.cc:1467] Processing CreateTablet for tablet d0ff928a633e4cae9d49229b989216d8 (DEFAULT_TABLE table=default.Table_5eb58df7cc2042f69300d10a7999719b [id=1fedf260324e464b9e390a637d62deeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.622602 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5ab42c67414f6699e48987979dfa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.623242 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ff928a633e4cae9d49229b989216d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.623867 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80955bd879c4c1eb16b07ad38096259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.624408 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d75d8468fc4c938fde0b2951d84562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.630573 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1f25b2427bf4b23b84761e4fb0dba55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.631086 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f287040b879642448367abe2b1c73cd9
W20260811 18:57:27.631528 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f25b2427bf4b23b84761e4fb0dba55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.631645 3549 catalog_manager.cc:6565] ReplaceTablet: tablet f287040b879642448367abe2b1c73cd9 deleted and replaced by tablet 482c2c39beb14880b40a600214c5efe4
I20260811 18:57:27.632066 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.632720 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.633397 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:27.633601 5091 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.633850 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 14e34732bda44670ae9da8facc405221 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.633756 5091 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.634449 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2be6762e326448d7b677a5641e2b3103
I20260811 18:57:27.634570 5091 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.635011 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db07259b8d8744d49c2c9fc89d4c5506" } new_table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" modify_external_catalogs: true
I20260811 18:57:27.635479 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.636520 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cafc7915ae7e4280a4d71e3bce4ca57c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.639606 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 419824ebd14e415f90072b8c335f1ad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.641537 5161 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.641980 5161 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.643637 5161 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.645130 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet f287040b879642448367abe2b1c73cd9 with delete_type TABLET_DATA_DELETED (replaced by tablet 482c2c39beb14880b40a600214c5efe4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.645607 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be6762e326448d7b677a5641e2b3103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.647060 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.647225 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f287040b879642448367abe2b1c73cd9 with delete_type TABLET_DATA_DELETED (replaced by tablet 482c2c39beb14880b40a600214c5efe4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.647858 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be6762e326448d7b677a5641e2b3103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.649755 3716 tablet_service.cc:1467] Processing CreateTablet for tablet d0ff928a633e4cae9d49229b989216d8 (DEFAULT_TABLE table=default.Table_5eb58df7cc2042f69300d10a7999719b [id=1fedf260324e464b9e390a637d62deeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.654752 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.654881 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.655246 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.655746 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.656515 3718 tablet_service.cc:1467] Processing CreateTablet for tablet b4f21761b37b42dbb814bd54cca89452 (DEFAULT_TABLE table=default.Table_0790be2c84c14fb3aeb2add67667646e [id=d9452934b441409d9bf9065826e30176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.657295 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 5d93073a69c143ff9460248848a108d4 (DEFAULT_TABLE table=default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.658229 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.658998 3676 leader_election.cc:341] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.660377 3851 tablet_service.cc:1467] Processing CreateTablet for tablet b4f21761b37b42dbb814bd54cca89452 (DEFAULT_TABLE table=default.Table_0790be2c84c14fb3aeb2add67667646e [id=d9452934b441409d9bf9065826e30176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.660959 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f21761b37b42dbb814bd54cca89452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.661613 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b (DEFAULT_TABLE table=default.Table_8f1e69da8480425682a8348cad2ffa51 [id=79f4456b3f5442c98f7d361d2b6be7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.662968 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 38a4783168554053aa953dba5e893b6b (DEFAULT_TABLE table=default.Table_7c395f6c4c364b7eb463f78e16675847 [id=17f081f75cf9467fa9c7b46b110403cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.665107 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6880ad27a3eb4599b407edeb2969a08f (DEFAULT_TABLE table=default.Table_41c44c966ece466eb232aeab834c5fe6 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.666021 3720 tablet_service.cc:1467] Processing CreateTablet for tablet b62ef754b8264afc8df6b4fd4bdb3fee (DEFAULT_TABLE table=default.Table_9df3de8d1ff649e8a41bff8b3de9d92f [id=f39457130ce54643a89d0928d8624350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.667583 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 261daa79df704a019f567bedb1046a15 (DEFAULT_TABLE table=default.Table_7db24c0702d141f08ea7023227018d81 [id=e814d7c224214dd3a051525c5ac34391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.670409 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 6880ad27a3eb4599b407edeb2969a08f (DEFAULT_TABLE table=default.Table_41c44c966ece466eb232aeab834c5fe6 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.670832 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880ad27a3eb4599b407edeb2969a08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.675889 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 498) took 5629 ms. Trace:
I20260811 18:57:27.676919 4162 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.677627 4162 ts_tablet_manager.cc:1397] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.284s user 0.000s sys 0.006s
I20260811 18:57:27.681614 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.681937 3744 raft_consensus.cc:2454] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.670111 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 261daa79df704a019f567bedb1046a15 (DEFAULT_TABLE table=default.Table_7db24c0702d141f08ea7023227018d81 [id=e814d7c224214dd3a051525c5ac34391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.667222 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b62ef754b8264afc8df6b4fd4bdb3fee (DEFAULT_TABLE table=default.Table_9df3de8d1ff649e8a41bff8b3de9d92f [id=f39457130ce54643a89d0928d8624350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.684167 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a4783168554053aa953dba5e893b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.684207 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 38a4783168554053aa953dba5e893b6b (DEFAULT_TABLE table=default.Table_7c395f6c4c364b7eb463f78e16675847 [id=17f081f75cf9467fa9c7b46b110403cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.685299 3824 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.685791 5161 raft_consensus.cc:2781] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.685882 5161 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.685947 5161 raft_consensus.cc:3037] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.687215 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 60f202e2f0934241be03ce67399d4b9e (DEFAULT_TABLE table=default.Table_b50aff06b9214a069e8b74b23089e9bb [id=b840acbfd40e46e689a8bbe29b24aa9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.688657 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a4783168554053aa953dba5e893b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.694600 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1342b8cd65b4c31a47b27a79c678d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.695405 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62ef754b8264afc8df6b4fd4bdb3fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.697974 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261daa79df704a019f567bedb1046a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.705035 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9fc9077de3204866a090b8d53ea5bd1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.707137 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fc9077de3204866a090b8d53ea5bd1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.708843 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4059e5907b1e488ab9e1d04e563f96cd
I20260811 18:57:27.710568 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 4059e5907b1e488ab9e1d04e563f96cd deleted and replaced by tablet 99bb8e934e6c44ca88640715b11e6bef
W20260811 18:57:27.711737 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 302cf69ea7c14fdaac72f6a9a9aff113 consensus state has no leader
I20260811 18:57:27.713613 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.714170 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ce5b9945b46348149fb08dafb33f61b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.715090 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce5b9945b46348149fb08dafb33f61b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.715878 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.716853 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7db24c0702d141f08ea7023227018d81" } new_table_name: "default.Table_0f3752c97bf04c169b9d733d20b26e0c" modify_external_catalogs: true
I20260811 18:57:27.723501 4162 raft_consensus.cc:348] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.728647 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15380ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:27.676672 4169 rpcz_store.cc:276] 0811 18:57:22.046042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.046676 (+ 634us) service_pool.cc:227] Handling call
0811 18:57:27.498647 (+5451971us) ts_tablet_manager.cc:1145] Deleting tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
0811 18:57:27.498666 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.675858 (+177192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5389662}
I20260811 18:57:27.730245 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" } modify_external_catalogs: true
I20260811 18:57:27.730567 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" } modify_external_catalogs: true
W20260811 18:57:27.731217 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e already in progress: creating tablet
I20260811 18:57:27.732960 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0a55918c905f4a4386f6529271fb60dc" } new_table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" modify_external_catalogs: true
I20260811 18:57:27.735206 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9743ba51a07b4fca8aaf1a188a692aac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.736282 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9743ba51a07b4fca8aaf1a188a692aac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.740190 3854 tablet_service.cc:1467] Processing CreateTablet for tablet e42f184431074846820dfd1939148025 (DEFAULT_TABLE table=default.Table_558f1f69ec7547c3b029b4672f04c6e6 [id=c5608ce4470d487fbf252c1890491b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.745851 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" } modify_external_catalogs: true
I20260811 18:57:27.746189 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" } modify_external_catalogs: true
I20260811 18:57:27.749226 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.750303 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd92de7863b94ff38b2c402c27ddf26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.765596 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 500) took 5645 ms. Trace:
I20260811 18:57:27.765736 4169 rpcz_store.cc:276] 0811 18:57:22.120313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.121411 (+ 1098us) service_pool.cc:227] Handling call
0811 18:57:27.731606 (+5610195us) ts_tablet_manager.cc:1145] Deleting tablet 7991dabe263b44f6b7b7d1d43b96ee44
0811 18:57:27.731625 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.765568 (+ 33943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5609644}
W20260811 18:57:27.767853 4863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:27.768594 4863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:27.768962 4863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:27.769973 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7991dabe263b44f6b7b7d1d43b96ee44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7991dabe263b44f6b7b7d1d43b96ee44 already in progress: creating tablet
I20260811 18:57:27.742138 5161 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.772598 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
I20260811 18:57:27.773463 4162 raft_consensus.cc:374] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.773594 4162 raft_consensus.cc:729] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.773831 4162 consensus_queue.cc:260] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.775045 4162 ts_tablet_manager.cc:1428] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.096s user 0.000s sys 0.003s
I20260811 18:57:27.729636 5091 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.777395 5091 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.778266 5091 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.782727 4162 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.783205 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de" } modify_external_catalogs: true
I20260811 18:57:27.783645 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de" } modify_external_catalogs: true
W20260811 18:57:27.786643 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 261daa79df704a019f567bedb1046a15 consensus state has no leader
I20260811 18:57:27.787351 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" } modify_external_catalogs: true
I20260811 18:57:27.787633 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" } modify_external_catalogs: true
I20260811 18:57:27.787597 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.791512 5161 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.794493 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4093564a88264ed59c945523c9ddc9b3
I20260811 18:57:27.803488 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.804867 3676 leader_election.cc:341] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.808858 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.745083 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 60f202e2f0934241be03ce67399d4b9e (DEFAULT_TABLE table=default.Table_b50aff06b9214a069e8b74b23089e9bb [id=b840acbfd40e46e689a8bbe29b24aa9d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.753824 3678 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.830s (SENT)
W20260811 18:57:27.815521 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 61734.1us
I20260811 18:57:27.816162 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f184431074846820dfd1939148025. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.816223 3678 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.082s (SENT)
I20260811 18:57:27.758087 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880ad27a3eb4599b407edeb2969a08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.818792 4162 tablet_bootstrap.cc:654] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.822821 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.823105 3743 raft_consensus.cc:3037] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.828079 5161 consensus_queue.cc:1035] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.829948 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 501) took 5701 ms. Trace:
I20260811 18:57:27.831219 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f21761b37b42dbb814bd54cca89452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.832101 4169 rpcz_store.cc:276] 0811 18:57:22.128214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.128272 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:27.766133 (+5637861us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:27.802390 (+ 36257us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.829920 (+ 27530us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":65,"tablet-delete.queue_time_us":5637435}
I20260811 18:57:27.833480 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62ef754b8264afc8df6b4fd4bdb3fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.834383 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261daa79df704a019f567bedb1046a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.835412 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 4093564a88264ed59c945523c9ddc9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.838807 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ff928a633e4cae9d49229b989216d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.840569 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 502) took 5695 ms. Trace:
I20260811 18:57:27.840692 4169 rpcz_store.cc:276] 0811 18:57:22.145264 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.147934 (+ 2670us) service_pool.cc:227] Handling call
0811 18:57:27.835963 (+5688029us) ts_tablet_manager.cc:1145] Deleting tablet 9ec4bba841dd446aa3db751358ce7f32
0811 18:57:27.835978 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.840540 (+ 4562us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5687576}
I20260811 18:57:27.841042 4169 tablet_replica.cc:330] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:27.841277 4169 raft_consensus.cc:2229] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:27.841953 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9ec4bba841dd446aa3db751358ce7f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ec4bba841dd446aa3db751358ce7f32 already in progress: creating tablet
W20260811 18:57:27.843356 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5d93073a69c143ff9460248848a108d4 (table default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5d93073a69c143ff9460248848a108d4 consensus state has no leader
I20260811 18:57:27.845286 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03949bb266e742cbbe858496f37177f3" } modify_external_catalogs: true
I20260811 18:57:27.845705 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03949bb266e742cbbe858496f37177f3" } modify_external_catalogs: true
I20260811 18:57:27.847832 4162 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.847991 4162 ts_tablet_manager.cc:1397] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.005s
I20260811 18:57:27.848529 4169 raft_consensus.cc:2258] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.849015 4162 raft_consensus.cc:348] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.849258 4162 raft_consensus.cc:374] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.849326 4162 raft_consensus.cc:729] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.849525 4162 consensus_queue.cc:260] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.850303 4169 ts_tablet_manager.cc:1905] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.850803 4162 ts_tablet_manager.cc:1428] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:27.851181 3743 raft_consensus.cc:2454] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.852286 4162 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:27.858757 3824 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.078s (SENT)
I20260811 18:57:27.859010 3824 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.859848 5161 raft_consensus.cc:2726] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.863358 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e4e304a679345cd9289252278ddb281" } new_table_name: "default.Table_deb475a5050c4f3dbaece0eb1e4ac618" modify_external_catalogs: true
I20260811 18:57:27.865917 4162 tablet_bootstrap.cc:654] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.879050 4167 ts_tablet_manager.cc:1918] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.880165 3824 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.881134 5080 raft_consensus.cc:2781] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.884706 4163 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.887353 4163 ts_tablet_manager.cc:1397] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.398s user 0.001s sys 0.005s
W20260811 18:57:27.890555 3678 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.384s (SENT)
I20260811 18:57:27.892999 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d93073a69c143ff9460248848a108d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.900417 5080 raft_consensus.cc:686] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.900826 5080 consensus_queue.cc:237] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.901572 4163 raft_consensus.cc:348] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.906292 4163 raft_consensus.cc:374] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.907658 4163 raft_consensus.cc:729] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:27.909238 4863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:27.911353 4863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260811 18:57:27.912150 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f202e2f0934241be03ce67399d4b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.910483 4163 consensus_queue.cc:260] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.913576 4863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:27.916630 4863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:27.919866 4163 ts_tablet_manager.cc:1428] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.031s user 0.003s sys 0.002s
I20260811 18:57:27.922415 4167 log.cc:1196] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:27.923076 4167 ts_tablet_manager.cc:1939] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.925951 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 567) took 4144 ms. Trace:
I20260811 18:57:27.926441 5091 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.926721 5091 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.927513 5091 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.927577 4163 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.928997 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.932124 3676 leader_election.cc:341] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.944914 4162 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.945241 4162 ts_tablet_manager.cc:1397] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.010s
I20260811 18:57:27.946885 4162 raft_consensus.cc:348] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.947436 4162 raft_consensus.cc:374] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.947706 4162 raft_consensus.cc:729] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.949133 4162 consensus_queue.cc:260] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.926599 4167 rpcz_store.cc:276] 0811 18:57:23.781728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.781793 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:27.382114 (+3600321us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:27.382127 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.457664 (+ 75537us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.878588 (+420924us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.925875 (+ 47287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3595130}
I20260811 18:57:27.955050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 575) took 4055 ms. Trace:
I20260811 18:57:27.955185 4167 rpcz_store.cc:276] 0811 18:57:23.899666 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.899799 (+ 133us) service_pool.cc:227] Handling call
0811 18:57:27.954849 (+4055050us) ts_tablet_manager.cc:1145] Deleting tablet 56d24aff9fd34f3eb05fdf942cc7efc7
0811 18:57:27.954863 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955027 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3988968}
I20260811 18:57:27.955593 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 586) took 3614 ms. Trace:
I20260811 18:57:27.955683 4167 rpcz_store.cc:276] 0811 18:57:24.341367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.341434 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:27.955470 (+3614036us) ts_tablet_manager.cc:1145] Deleting tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
0811 18:57:27.955481 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955576 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3613490}
I20260811 18:57:27.955989 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 587) took 3613 ms. Trace:
I20260811 18:57:27.956094 4167 rpcz_store.cc:276] 0811 18:57:24.342289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.342371 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:27.955888 (+3613517us) ts_tablet_manager.cc:1145] Deleting tablet 64b922f2a71544e8a18533aa0a1ccb52
0811 18:57:27.955898 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955972 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3612931}
I20260811 18:57:27.956429 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 585) took 3615 ms. Trace:
I20260811 18:57:27.956511 4167 rpcz_store.cc:276] 0811 18:57:24.341129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.358587 (+ 17458us) service_pool.cc:227] Handling call
0811 18:57:27.956320 (+3597733us) ts_tablet_manager.cc:1145] Deleting tablet 14e34732bda44670ae9da8facc405221
0811 18:57:27.956333 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.956413 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3575451}
I20260811 18:57:27.956818 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 584) took 3615 ms. Trace:
I20260811 18:57:27.956898 4167 rpcz_store.cc:276] 0811 18:57:24.340950 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.362104 (+ 21154us) service_pool.cc:227] Handling call
0811 18:57:27.956718 (+3594614us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:27.956730 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.956801 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3575701}
I20260811 18:57:27.957511 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 592) took 3181 ms. Trace:
I20260811 18:57:27.957615 4167 rpcz_store.cc:276] 0811 18:57:24.775962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.776026 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:27.957386 (+3181360us) ts_tablet_manager.cc:1145] Deleting tablet 0b40da8cb7324da9804e6fdafd8e8266
0811 18:57:27.957398 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.957490 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3180227}
I20260811 18:57:27.958002 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 593) took 3174 ms. Trace:
I20260811 18:57:27.958051 4162 ts_tablet_manager.cc:1428] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.002s
I20260811 18:57:27.986377 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:27.984453 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 4093564a88264ed59c945523c9ddc9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.989532 4162 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.991221 4162 tablet_bootstrap.cc:654] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.991117 5080 consensus_queue.cc:1035] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.997785 5080 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.998557 5080 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.999388 5080 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.002673 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.004063 3743 raft_consensus.cc:2454] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.004938 3824 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.005405 5133 raft_consensus.cc:2781] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.005538 5133 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.005708 5133 raft_consensus.cc:3037] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.766605 3710 tablet_service.cc:1467] Processing CreateTablet for tablet e42f184431074846820dfd1939148025 (DEFAULT_TABLE table=default.Table_558f1f69ec7547c3b029b4672f04c6e6 [id=c5608ce4470d487fbf252c1890491b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.007361 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f184431074846820dfd1939148025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.008888 5133 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.009588 5133 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.010672 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.010969 3743 raft_consensus.cc:3037] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.013794 3743 raft_consensus.cc:2454] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.014654 3824 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.015126 5080 raft_consensus.cc:2781] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.015324 5080 raft_consensus.cc:686] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.015674 5080 consensus_queue.cc:237] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.023059 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 already in progress: creating tablet
W20260811 18:57:28.023401 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 64b922f2a71544e8a18533aa0a1ccb52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64b922f2a71544e8a18533aa0a1ccb52 already in progress: creating tablet
W20260811 18:57:28.023643 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 14e34732bda44670ae9da8facc405221 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e34732bda44670ae9da8facc405221 already in progress: creating tablet
W20260811 18:57:28.023895 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7057b8a7bae249caaed304b5c66e6871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7057b8a7bae249caaed304b5c66e6871 already in progress: creating tablet
W20260811 18:57:28.024220 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0b40da8cb7324da9804e6fdafd8e8266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b40da8cb7324da9804e6fdafd8e8266 already in progress: creating tablet
I20260811 18:57:28.025066 4162 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.025421 4162 ts_tablet_manager.cc:1397] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.005s
I20260811 18:57:27.986701 4167 rpcz_store.cc:276] 0811 18:57:24.783914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.783994 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:27.957893 (+3173899us) ts_tablet_manager.cc:1145] Deleting tablet 19d7716739f848d78b9cbbd3975f3d44
0811 18:57:27.957905 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.957986 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3173449}
I20260811 18:57:28.027493 4162 raft_consensus.cc:348] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.028028 4162 raft_consensus.cc:374] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.028167 4162 raft_consensus.cc:729] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.029667 4162 consensus_queue.cc:260] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.040555 4162 ts_tablet_manager.cc:1428] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260811 18:57:28.043388 4162 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.039984 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 594) took 3247 ms. Trace:
W20260811 18:57:28.050719 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa95131ce50943e3b0c6b5bdd5a30939 consensus state has no leader
I20260811 18:57:28.054049 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5296710afead477eb67f356c9afbc4a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.047101 4167 rpcz_store.cc:276] 0811 18:57:24.791459 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.791526 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:28.031472 (+3239946us) ts_tablet_manager.cc:1145] Deleting tablet 2d0b59cafb154416952aecaa418e861f
0811 18:57:28.032551 (+ 1079us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.038486 (+ 5935us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3234512}
I20260811 18:57:28.061789 4162 tablet_bootstrap.cc:654] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.067469 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 595) took 3260 ms. Trace:
I20260811 18:57:28.069347 4167 rpcz_store.cc:276] 0811 18:57:24.807343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.808450 (+ 1107us) service_pool.cc:227] Handling call
0811 18:57:28.067332 (+3258882us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:28.067348 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.067444 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3255769}
W20260811 18:57:28.082361 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5296710afead477eb67f356c9afbc4a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.089263 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3de36d9c6d4542be9ee2c873f0bfcfca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.094739 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3de36d9c6d4542be9ee2c873f0bfcfca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.091617 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 596) took 3283 ms. Trace:
W20260811 18:57:28.089655 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 19d7716739f848d78b9cbbd3975f3d44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d7716739f848d78b9cbbd3975f3d44 already in progress: creating tablet
W20260811 18:57:28.096432 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2d0b59cafb154416952aecaa418e861f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d0b59cafb154416952aecaa418e861f already in progress: creating tablet
W20260811 18:57:28.097126 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because the tablet was not found. No further retry: Not found: Tablet not found: f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:28.096150 4167 rpcz_store.cc:276] 0811 18:57:24.807533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.808090 (+ 557us) service_pool.cc:227] Handling call
0811 18:57:28.086420 (+3278330us) ts_tablet_manager.cc:1145] Deleting tablet 3b40e44a2f4940af891eddf6c3b6293c
0811 18:57:28.087134 (+ 714us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.090898 (+ 3764us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3273901}
I20260811 18:57:28.103229 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.104060 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3b40e44a2f4940af891eddf6c3b6293c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b40e44a2f4940af891eddf6c3b6293c already in progress: creating tablet
I20260811 18:57:28.112339 5091 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.112490 5091 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.111179 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 597) took 3302 ms. Trace:
I20260811 18:57:28.113220 5091 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.114197 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.115180 3676 leader_election.cc:341] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.117940 4162 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.118297 4162 ts_tablet_manager.cc:1397] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.076s user 0.010s sys 0.000s
I20260811 18:57:28.113262 4167 rpcz_store.cc:276] 0811 18:57:24.807782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.807943 (+ 161us) service_pool.cc:227] Handling call
0811 18:57:28.107494 (+3299551us) ts_tablet_manager.cc:1145] Deleting tablet 92e49846ab714dd294acdd09d49a83bd
0811 18:57:28.107516 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.110611 (+ 3095us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3290085}
I20260811 18:57:28.120453 4162 raft_consensus.cc:348] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.121004 4162 raft_consensus.cc:374] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.121213 4162 raft_consensus.cc:729] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:28.122646 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 92e49846ab714dd294acdd09d49a83bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e49846ab714dd294acdd09d49a83bd already in progress: creating tablet
I20260811 18:57:28.123067 4162 consensus_queue.cc:260] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.129915 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ae7da0868b114052ba421801b76fb33c
I20260811 18:57:28.130918 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.131693 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6284ad4471b4b7095b11fe7a8f304e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.133697 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.132688 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 598) took 3298 ms. Trace:
W20260811 18:57:28.134420 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.135720 4162 ts_tablet_manager.cc:1428] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.017s user 0.001s sys 0.003s
I20260811 18:57:28.136022 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d988ef7e9dd43158d107998ea4d82b9
I20260811 18:57:28.136771 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c281ac8dd4a444c08084edb70426f847" } new_table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" modify_external_catalogs: true
I20260811 18:57:28.139567 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7da0868b114052ba421801b76fb33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.139850 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.139505 4162 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.134773 4167 rpcz_store.cc:276] 0811 18:57:24.833863 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833933 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:28.125902 (+3291969us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:28.126930 (+ 1028us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.132334 (+ 5404us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3290291}
W20260811 18:57:28.144157 3678 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.278s (SENT)
I20260811 18:57:28.144526 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b6c70b4163634984bd02dd9b47ffb472
I20260811 18:57:28.145512 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c70b4163634984bd02dd9b47ffb472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.146193 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e79b6e054e914fd39ca0a6282eb0ce17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.146971 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e79b6e054e914fd39ca0a6282eb0ce17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.149354 4167 tablet_replica.cc:330] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.153517 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3baa014716584cb184d8a0e4299a8542" } new_table_name: "default.Table_3b056c714ed947198f75f2c7802ae4dd" modify_external_catalogs: true
I20260811 18:57:28.161612 4167 raft_consensus.cc:2229] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.167603 4163 tablet_bootstrap.cc:654] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.169294 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
I20260811 18:57:28.172438 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f202e2f0934241be03ce67399d4b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.172438 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.178174 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b908c7d655a467bbda1f9d2fdc1b99a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.181114 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46505ba544104458888dbc2a37115dcc consensus state has no leader
I20260811 18:57:28.181901 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_988443c8d6bd4b06b90d4e7f95a0998c" } new_table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" modify_external_catalogs: true
I20260811 18:57:28.183058 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9914a933914f4377ba70e0b57c70edff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.184628 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2523fa07587246d994eca722c7032105 consensus state has no leader
I20260811 18:57:28.185086 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7" } modify_external_catalogs: true
I20260811 18:57:28.185426 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7" } modify_external_catalogs: true
W20260811 18:57:28.193670 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b908c7d655a467bbda1f9d2fdc1b99a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.196206 4167 raft_consensus.cc:2258] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.196944 4167 ts_tablet_manager.cc:1905] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#4 0xa686e1 in std::__shared_ptr<kudu::tablet::Tablet, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() ../../../include/c++/8/bits/shared_ptr_base.h:1167:31
I20260811 18:57:28.198510 4162 tablet_bootstrap.cc:654] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.202971 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9914a933914f4377ba70e0b57c70edff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.203598 4863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=40953
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
#5 0x7fc7bfa75e66 in std::__shared_ptr<kudu::tablet::Tablet, (__gnu_cxx::_Lock_policy)2>::reset() ../../../include/c++/8/bits/shared_ptr_base.h:1285:9
#6 0x7fc7bfa57c66 in kudu::tablet::TabletReplica::Stop() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_replica.cc:371:13
W20260811 18:57:28.221303 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21151ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
W20260811 18:57:28.224426 3824 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.155s (SENT)
I20260811 18:57:28.224609 3824 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:28.224807 3824 raft_consensus.cc:2670] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:28.225579 5080 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.225736 5080 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.226599 5080 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.227612 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:28.227885 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6df5545532e346f7abcdc4cd228b84da has no consensus state
I20260811 18:57:28.227943 3743 raft_consensus.cc:2454] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.228504 3824 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.228832 5080 raft_consensus.cc:2781] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.228940 5080 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.229017 5080 raft_consensus.cc:3037] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:28.229682 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260811 18:57:28.231031 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260811 18:57:28.232395 5080 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.234670 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.234932 3743 raft_consensus.cc:3037] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.238490 5080 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.238834 3743 raft_consensus.cc:2454] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.239254 5091 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.239557 3824 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.239449 5091 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.239756 4863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:28.239923 5080 raft_consensus.cc:2781] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.240487 5091 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:28.240696 3678 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.245s (SENT)
I20260811 18:57:28.241412 5080 raft_consensus.cc:686] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.241765 5080 consensus_queue.cc:237] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.242012 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.242735 3676 leader_election.cc:341] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:28.244287 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b57ed7814a1e440cba362613cbab32f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b57ed7814a1e440cba362613cbab32f8 already in progress: creating tablet
I20260811 18:57:28.245674 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7da0868b114052ba421801b76fb33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.247174 4163 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.247453 4163 ts_tablet_manager.cc:1397] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.321s user 0.012s sys 0.026s
I20260811 18:57:28.248908 4163 raft_consensus.cc:348] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.249332 4163 raft_consensus.cc:374] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.249464 4163 raft_consensus.cc:729] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.252921 4163 consensus_queue.cc:260] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.253376 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d93e11fb1464fcbb4999abaebc3d611" } new_table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" modify_external_catalogs: true
I20260811 18:57:28.253845 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.254441 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c70b4163634984bd02dd9b47ffb472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.254868 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255056 4163 ts_tablet_manager.cc:1428] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260811 18:57:28.255213 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255383 4163 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.255544 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255885 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.257071 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.257527 3824 consensus_peers.cc:487] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:28.263149 4169 ts_tablet_manager.cc:1918] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:28.263299 4169 log.cc:1196] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:28.263770 4169 ts_tablet_manager.cc:1939] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.267477 4162 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.267809 4162 ts_tablet_manager.cc:1397] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.130s user 0.005s sys 0.003s
I20260811 18:57:28.269423 4162 raft_consensus.cc:348] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.269847 4162 raft_consensus.cc:374] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.270962 4162 raft_consensus.cc:729] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.271306 4162 consensus_queue.cc:260] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.272346 4162 ts_tablet_manager.cc:1428] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:28.273041 4162 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.279546 4863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:57:28.280576 4163 tablet_bootstrap.cc:654] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.284029 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ccc63b46a4e346f79d1e03a82490794c consensus state has no leader
W20260811 18:57:28.284855 3678 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.153s (SENT)
I20260811 18:57:28.285089 4162 tablet_bootstrap.cc:654] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.290679 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9cadd28493e444185170bcd4c314208"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.298672 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9cadd28493e444185170bcd4c314208 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.304731 5091 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.304948 5091 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.305758 5091 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.321318 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:28.321504 4163 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.321797 4163 ts_tablet_manager.cc:1397] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.066s user 0.005s sys 0.001s
W20260811 18:57:28.322109 3676 leader_election.cc:341] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.321756 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 503) took 6058 ms. Trace:
I20260811 18:57:28.323089 4169 rpcz_store.cc:276] 0811 18:57:22.262859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.262943 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:27.840988 (+5578045us) ts_tablet_manager.cc:1145] Deleting tablet 0ace00158b744bd8a97dc1c47fc6049c
0811 18:57:27.841000 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.187746 (+346746us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.263030 (+ 75284us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.321740 (+ 58710us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5576785}
I20260811 18:57:28.324887 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_102e5367d375402c9800357142894fa0 [id=a067d27ecf9c402db2631f2df5f0fea4]) successfully deleted
I20260811 18:57:28.325839 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 509) took 5991 ms. Trace:
I20260811 18:57:28.326568 4167 ts_tablet_manager.cc:1918] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:28.326711 4167 log.cc:1196] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:28.327085 4167 ts_tablet_manager.cc:1939] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.327358 4169 rpcz_store.cc:276] 0811 18:57:22.334216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.334307 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:28.325693 (+5991386us) ts_tablet_manager.cc:1145] Deleting tablet 2605007cc2454599868f44c717deb2d2
0811 18:57:28.325705 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.325817 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5990971}
I20260811 18:57:28.327643 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 599) took 3484 ms. Trace:
I20260811 18:57:28.327754 4167 rpcz_store.cc:276] 0811 18:57:24.842943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.843022 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:28.147764 (+3304742us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:28.148223 (+ 459us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.221926 (+ 73703us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.326414 (+104488us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.327621 (+ 1207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3291176}
W20260811 18:57:28.328122 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2605007cc2454599868f44c717deb2d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2605007cc2454599868f44c717deb2d2 already in progress: creating tablet
I20260811 18:57:28.328536 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ef393c341bb54baf87176db5b5254ec7 (table default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]) successfully deleted
I20260811 18:57:28.329020 4169 tablet_replica.cc:330] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.329509 4162 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.329329 4169 raft_consensus.cc:2229] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.329710 4162 ts_tablet_manager.cc:1397] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.006s
I20260811 18:57:28.329810 4169 raft_consensus.cc:2258] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.330811 4162 raft_consensus.cc:348] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.331117 4162 raft_consensus.cc:374] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.331243 4162 raft_consensus.cc:729] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.331501 4162 consensus_queue.cc:260] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.332142 4162 ts_tablet_manager.cc:1428] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:28.333302 4162 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.326083 4163 raft_consensus.cc:348] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.334695 4163 raft_consensus.cc:374] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.334839 4163 raft_consensus.cc:729] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.335142 4163 consensus_queue.cc:260] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.336979 4169 ts_tablet_manager.cc:1905] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:28.343223 3678 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.114s (SENT)
I20260811 18:57:28.346649 3552 catalog_manager.cc:5495] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:28.349098 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:28.353323 4162 tablet_bootstrap.cc:654] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.354802 4167 tablet_replica.cc:330] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.355028 4167 raft_consensus.cc:2229] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.355273 4167 raft_consensus.cc:2258] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.357435 4167 ts_tablet_manager.cc:1905] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.359006 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7309030d82da426ab87ff64346efd25b" } modify_external_catalogs: true
I20260811 18:57:28.359385 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7309030d82da426ab87ff64346efd25b" } modify_external_catalogs: true
I20260811 18:57:28.362066 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_82ea06e136674461909baae7b6d4e8cd" } modify_external_catalogs: true
I20260811 18:57:28.362282 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_82ea06e136674461909baae7b6d4e8cd" } modify_external_catalogs: true
I20260811 18:57:28.363981 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_70493900fc9346389c1d4b71c611b962" } modify_external_catalogs: true
I20260811 18:57:28.364266 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_70493900fc9346389c1d4b71c611b962" } modify_external_catalogs: true
I20260811 18:57:28.367291 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 6df5545532e346f7abcdc4cd228b84da (DEFAULT_TABLE table=default.Table_ff02853da3f1406c90f961567adf92bf [id=79599e38b25046ae9ad11292ceeddcf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.367233 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4c88a7260fbc4472bbbd3244c0cc4a85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.367805 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df5545532e346f7abcdc4cd228b84da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.368176 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.372709 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_378dd90b6af541b8ab95d9f19681a03f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.373028 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 3317f9d8e2354df38c6342fc48d82f6f (DEFAULT_TABLE table=default.Table_1a2751cf08eb4ce0b354486cf5678c1c [id=10901c364ced4e9fbcbc9611a3ceb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.373458 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3317f9d8e2354df38c6342fc48d82f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.374862 3719 tablet_service.cc:1467] Processing CreateTablet for tablet e1ad3860d03f46baadd5ede854f1a909 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.375409 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ad3860d03f46baadd5ede854f1a909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.376343 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 547f6c33d6414bcfa3eeff20cb9a0302 (DEFAULT_TABLE table=default.Table_8ad9f329a9c449e4ade20157c2601ee8 [id=8f080597ed9e4f358b7539596a864252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352384 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 6df5545532e346f7abcdc4cd228b84da (DEFAULT_TABLE table=default.Table_ff02853da3f1406c90f961567adf92bf [id=79599e38b25046ae9ad11292ceeddcf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352488 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 547f6c33d6414bcfa3eeff20cb9a0302 (DEFAULT_TABLE table=default.Table_8ad9f329a9c449e4ade20157c2601ee8 [id=8f080597ed9e4f358b7539596a864252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352789 3854 tablet_service.cc:1467] Processing CreateTablet for tablet e1ad3860d03f46baadd5ede854f1a909 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.365613 5080 consensus_queue.cc:1035] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:28.371824 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3317f9d8e2354df38c6342fc48d82f6f (DEFAULT_TABLE table=default.Table_1a2751cf08eb4ce0b354486cf5678c1c [id=10901c364ced4e9fbcbc9611a3ceb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.378180 4163 ts_tablet_manager.cc:1428] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.056s user 0.001s sys 0.001s
I20260811 18:57:28.388569 4163 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.397537 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547f6c33d6414bcfa3eeff20cb9a0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.399201 5091 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.399375 5091 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.406760 5091 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.407796 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.408625 3676 leader_election.cc:341] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:28.423512 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_378dd90b6af541b8ab95d9f19681a03f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.434542 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_558f1f69ec7547c3b029b4672f04c6e6" } new_table_name: "default.Table_267cfa966911410c8d062f77ed7d2453" modify_external_catalogs: true
I20260811 18:57:28.442569 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d8146f47d9ae48889ac7efc954d4a6b7
I20260811 18:57:28.442904 3862 tablet_service.cc:1467] Processing CreateTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e (DEFAULT_TABLE table=default.Table_82ea06e136674461909baae7b6d4e8cd [id=a9b8af67e69f42229cc4340500fc95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.443511 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.439945 5207 tablet.cc:1722] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:28.444865 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:28.449043 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.384s (SENT) [suppressed 13 similar messages]
I20260811 18:57:28.446102 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_710d4f427cd6437a908b1109b4a140a1" } new_table_name: "default.Table_1c4b1f6ef3514273a9cb50060b3a72d3" modify_external_catalogs: true
I20260811 18:57:28.457540 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90" } modify_external_catalogs: true
I20260811 18:57:28.457957 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90" } modify_external_catalogs: true
I20260811 18:57:28.459290 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.459857 3714 tablet_service.cc:1467] Processing CreateTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e (DEFAULT_TABLE table=default.Table_82ea06e136674461909baae7b6d4e8cd [id=a9b8af67e69f42229cc4340500fc95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.460021 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.460031 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef4494689f0147118d362ed74d1bf78c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.461292 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38c172a9fef4a3ab8ee188335a7c97e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.462059 3678 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.302s (SENT)
W20260811 18:57:28.462443 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef4494689f0147118d362ed74d1bf78c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.471124 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a38c172a9fef4a3ab8ee188335a7c97e
I20260811 18:57:28.472219 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.474035 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.479075 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
I20260811 18:57:28.480299 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c3df98432314ff2b4cbd7004701b258"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.480710 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.481345 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c3df98432314ff2b4cbd7004701b258 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.485268 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.485966 5209 tablet.cc:1722] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:28.487170 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_832e411b94e24eb49b8798f66fa24774"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.491329 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_832e411b94e24eb49b8798f66fa24774 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.491611 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.493290 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.495833 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cf6c8d542ae2489ba466ffadd518db12
W20260811 18:57:28.499253 3678 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.153s (SENT)
I20260811 18:57:28.501099 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.504653 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdae8856d30a46259e91345b60181c2b" } new_table_name: "default.Table_f5f50e8f17e04760a61e10c54d1aa657" modify_external_catalogs: true
I20260811 18:57:28.508139 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.508503 4162 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.512262 4162 ts_tablet_manager.cc:1397] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.179s user 0.008s sys 0.011s
I20260811 18:57:28.518306 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_862e6dcdf4f748738d4f651170e5611f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.523718 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_862e6dcdf4f748738d4f651170e5611f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.519164 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6064366e4ea94146903cb48b739102e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.524611 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6064366e4ea94146903cb48b739102e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.527882 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38c172a9fef4a3ab8ee188335a7c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528254 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547f6c33d6414bcfa3eeff20cb9a0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528568 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3317f9d8e2354df38c6342fc48d82f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528985 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df5545532e346f7abcdc4cd228b84da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.530498 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ad3860d03f46baadd5ede854f1a909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.531510 4163 tablet_bootstrap.cc:654] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.538556 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:28.533421 4162 raft_consensus.cc:348] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.539525 4162 raft_consensus.cc:374] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.539601 4162 raft_consensus.cc:729] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.542259 3714 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0e443ab84424dac307b745b496986 (DEFAULT_TABLE table=default.Table_a35f3caa50e849abac2fc4d5711ea9fb [id=faa0f0b707b34742a67085c9c3f4fe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.543273 3858 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0e443ab84424dac307b745b496986 (DEFAULT_TABLE table=default.Table_a35f3caa50e849abac2fc4d5711ea9fb [id=faa0f0b707b34742a67085c9c3f4fe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.556143 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0e443ab84424dac307b745b496986. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.561476 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:28.563577 5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:28.570363 4863 server_base.cc:1034] running on GCE node
W20260811 18:57:28.572398 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e42f184431074846820dfd1939148025 consensus state has no leader
I20260811 18:57:28.572923 4863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:28.573153 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a79b27c0bd524af985c2e6167624551a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.573987 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a79b27c0bd524af985c2e6167624551a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.573985 4162 consensus_queue.cc:260] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:28.575515 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
I20260811 18:57:28.579861 4163 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.580400 4163 ts_tablet_manager.cc:1397] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.192s user 0.003s sys 0.006s
W20260811 18:57:28.586228 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 44045.4us
I20260811 18:57:28.589077 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 (DEFAULT_TABLE table=default.Table_068dc550d67d413c916d1384c3cb811c [id=377abe00a3d6469ab4285270de14cbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.589644 4863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:28.590655 4162 ts_tablet_manager.cc:1428] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.075s user 0.004s sys 0.000s
I20260811 18:57:28.591161 4162 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.600121 4163 raft_consensus.cc:348] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.600318 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 (DEFAULT_TABLE table=default.Table_068dc550d67d413c916d1384c3cb811c [id=377abe00a3d6469ab4285270de14cbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.600942 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d721b0e55384d8a8efd88a0e47b1504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.602548 4163 raft_consensus.cc:374] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.602712 4163 raft_consensus.cc:729] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.602999 4163 consensus_queue.cc:260] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.605851 4163 ts_tablet_manager.cc:1428] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260811 18:57:28.603519 4169 ts_tablet_manager.cc:1918] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.607009 4169 log.cc:1196] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e51a08e6dbf04e3fa58689ee0cd45db3
I20260811 18:57:28.608242 4169 ts_tablet_manager.cc:1939] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.610416 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f13a14bd64704798bf6de4fc05201a31
I20260811 18:57:28.621986 4863 hybrid_clock.cc:648] HybridClock initialized: now 1786474648621714 us; error 78 us; skew 500 ppm
I20260811 18:57:28.622665 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet f13a14bd64704798bf6de4fc05201a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.622824 4863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:28.626865 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d721b0e55384d8a8efd88a0e47b1504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.630270 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet f13a14bd64704798bf6de4fc05201a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.630837 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0e443ab84424dac307b745b496986. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.634922 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:28.635035 4163 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.635473 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.635658 4162 tablet_bootstrap.cc:654] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.636273 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.637912 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0a164081b1349d9bc89f20d1ea6425d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.639496 4167 ts_tablet_manager.cc:1918] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.639680 4167 log.cc:1196] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ff1be026ca404eada53f60f257ea367c
I20260811 18:57:28.640103 4167 ts_tablet_manager.cc:1939] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.640707 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 600) took 3797 ms. Trace:
I20260811 18:57:28.640810 4167 rpcz_store.cc:276] 0811 18:57:24.843433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.843499 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:28.354601 (+3511102us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:28.354756 (+ 155us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.555561 (+200805us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.639357 (+ 83796us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.640684 (+ 1327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3498185}
I20260811 18:57:28.641321 4163 tablet_bootstrap.cc:654] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.642303 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 511) took 6304 ms. Trace:
I20260811 18:57:28.650771 4169 rpcz_store.cc:276] 0811 18:57:22.337691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.337768 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:28.328896 (+5991128us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:28.328913 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.551583 (+222670us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.603364 (+ 51781us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.642279 (+ 38915us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5990699}
I20260811 18:57:28.651444 4169 tablet_replica.cc:330] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.651706 4169 raft_consensus.cc:2229] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.652014 4169 raft_consensus.cc:2258] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.652740 4169 ts_tablet_manager.cc:1905] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.654876 4162 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.655042 4162 ts_tablet_manager.cc:1397] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.002s
I20260811 18:57:28.657001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 601) took 3802 ms. Trace:
I20260811 18:57:28.657122 4167 rpcz_store.cc:276] 0811 18:57:24.854225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.854284 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:28.656857 (+3802573us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:28.656872 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.656975 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3800180}
I20260811 18:57:28.658166 4162 raft_consensus.cc:348] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.660053 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34c1bb6042d54870bb236a2204ce71ed" } new_table_name: "default.Table_56b89b2b2c4b45768b2866c74d9f1390" modify_external_catalogs: true
I20260811 18:57:28.660315 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 13788f40714847ff8ce6a810ac53e398 (DEFAULT_TABLE table=default.Table_0df46efea56e41b79fbb3e4dd034f84e [id=77dfe9e4685a4c02b9b440096e180d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.660845 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13788f40714847ff8ce6a810ac53e398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.661358 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 630) took 3311 ms. Trace:
I20260811 18:57:28.662556 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e51a08e6dbf04e3fa58689ee0cd45db3 (table default.Table_acb61e38f2a0442688865698892cc5c9 [id=3c7daa88956747719fb78e9860e7039f]) successfully deleted
I20260811 18:57:28.663112 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fcb655a52f4449839a4919d317bc6cfe" } new_table_name: "default.Table_e5769978bc1148a7a53c0df338266d1e" modify_external_catalogs: true
I20260811 18:57:28.663717 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 (DEFAULT_TABLE table=default.Table_c2053e3676584227a3ea13c4f912e315 [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.664088 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4451cc4d7c5940cdb41c021ae486a5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.646986 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ff1be026ca404eada53f60f257ea367c (table default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]) successfully deleted
I20260811 18:57:28.666389 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 482c2c39beb14880b40a600214c5efe4 (DEFAULT_TABLE table=default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.666783 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482c2c39beb14880b40a600214c5efe4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.667204 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because the tablet was not found. No further retry: Not found: Tablet not found: 0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:28.668124 3859 tablet_service.cc:1467] Processing CreateTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e (DEFAULT_TABLE table=default.Table_1bea4820fb5f4e65acb2a1a36332ac2d [id=55e75293fccb4dcfb18d24be0b92d4de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.668557 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb54beb34a654fbdb9c7632901bb0b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.673729 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1e36da0b2c374304a201a3972d4762f2 (DEFAULT_TABLE table=default.Table_3453632ca8274e459b096036508fbb8e [id=b9048b08f3a54c4787533497eb63ca95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.658412 4162 raft_consensus.cc:374] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.674633 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 99bb8e934e6c44ca88640715b11e6bef (DEFAULT_TABLE table=default.Table_fcb655a52f4449839a4919d317bc6cfe [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.675024 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99bb8e934e6c44ca88640715b11e6bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.667714 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1e36da0b2c374304a201a3972d4762f2 (DEFAULT_TABLE table=default.Table_3453632ca8274e459b096036508fbb8e [id=b9048b08f3a54c4787533497eb63ca95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.667013 3724 tablet_service.cc:1467] Processing CreateTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e (DEFAULT_TABLE table=default.Table_1bea4820fb5f4e65acb2a1a36332ac2d [id=55e75293fccb4dcfb18d24be0b92d4de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.676054 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e36da0b2c374304a201a3972d4762f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.676496 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb54beb34a654fbdb9c7632901bb0b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.676410 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" } modify_external_catalogs: true
I20260811 18:57:28.676993 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" } modify_external_catalogs: true
I20260811 18:57:28.667749 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 482c2c39beb14880b40a600214c5efe4 (DEFAULT_TABLE table=default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.678879 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482c2c39beb14880b40a600214c5efe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.681768 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e36da0b2c374304a201a3972d4762f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.682739 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4096c2e6bc426b97dde27e85fec544 (DEFAULT_TABLE table=default.Table_e1f25b2427bf4b23b84761e4fb0dba55 [id=892d56d389c44374afdb0d90eeb5ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.683095 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4096c2e6bc426b97dde27e85fec544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.668084 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 99bb8e934e6c44ca88640715b11e6bef (DEFAULT_TABLE table=default.Table_fcb655a52f4449839a4919d317bc6cfe [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.690050 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99bb8e934e6c44ca88640715b11e6bef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.693780 3678 leader_election.cc:334] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.172s (SENT)
I20260811 18:57:28.693895 3678 leader_election.cc:302] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:28.694067 3678 raft_consensus.cc:2670] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:28.668061 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 (DEFAULT_TABLE table=default.Table_c2053e3676584227a3ea13c4f912e315 [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.694869 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4451cc4d7c5940cdb41c021ae486a5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.672060 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 13788f40714847ff8ce6a810ac53e398 (DEFAULT_TABLE table=default.Table_0df46efea56e41b79fbb3e4dd034f84e [id=77dfe9e4685a4c02b9b440096e180d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.696524 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13788f40714847ff8ce6a810ac53e398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.698038 5161 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.698203 5161 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.674434 4162 raft_consensus.cc:729] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.699034 5161 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.704144 4163 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.704352 4163 ts_tablet_manager.cc:1397] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.006s
I20260811 18:57:28.705265 4163 raft_consensus.cc:348] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.705488 4163 raft_consensus.cc:374] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.705565 4163 raft_consensus.cc:729] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.667412 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4096c2e6bc426b97dde27e85fec544 (DEFAULT_TABLE table=default.Table_e1f25b2427bf4b23b84761e4fb0dba55 [id=892d56d389c44374afdb0d90eeb5ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.705837 4163 consensus_queue.cc:260] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.706427 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4096c2e6bc426b97dde27e85fec544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.707847 4163 ts_tablet_manager.cc:1428] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:28.708132 4163 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.709568 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 75941ddbffc24f9bb8ba6ed2e6682eae (DEFAULT_TABLE table=default.Table_1af253ec487e4e30b462dc9cb3b1b3cb [id=129d3800251c4ce78db2dc60da27a88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.710009 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75941ddbffc24f9bb8ba6ed2e6682eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.710235 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6dd21c795924a47a8a2435b5de1afb7" } modify_external_catalogs: true
I20260811 18:57:28.710649 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6dd21c795924a47a8a2435b5de1afb7" } modify_external_catalogs: true
I20260811 18:57:28.712396 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0" } modify_external_catalogs: true
I20260811 18:57:28.712584 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0" } modify_external_catalogs: true
I20260811 18:57:28.705183 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.714542 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
I20260811 18:57:28.714597 3743 raft_consensus.cc:2454] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.715004 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 deleted and replaced by tablet 8319453f68a54da9813747152ef08756
W20260811 18:57:28.719133 3678 leader_election.cc:334] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.520s (SENT)
I20260811 18:57:28.719770 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 with delete_type TABLET_DATA_DELETED (replaced by tablet 8319453f68a54da9813747152ef08756) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.720270 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" } modify_external_catalogs: true
I20260811 18:57:28.720803 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 75941ddbffc24f9bb8ba6ed2e6682eae (DEFAULT_TABLE table=default.Table_1af253ec487e4e30b462dc9cb3b1b3cb [id=129d3800251c4ce78db2dc60da27a88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.720880 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 with delete_type TABLET_DATA_DELETED (replaced by tablet 8319453f68a54da9813747152ef08756) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.721339 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75941ddbffc24f9bb8ba6ed2e6682eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.720674 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" } modify_external_catalogs: true
I20260811 18:57:28.725446 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.726120 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44c67a0eb7485fb8dcde09416cd331 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:28.726619 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b485cd94aa7c4642a0b3d0021cef63c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.727041 3824 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.727322 5161 raft_consensus.cc:2781] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.727401 5161 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.727460 5161 raft_consensus.cc:3037] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.727677 4863 webserver.cc:460] Webserver started at http://127.3.72.126:40953/ using document root <none> and password file <none>
I20260811 18:57:28.728945 4863 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:28.729277 4863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:28.729573 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2be35bff506498281ab88f9cbb55de8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.730836 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2be35bff506498281ab88f9cbb55de8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.730522 5161 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.731411 5161 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.737181 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.737457 3743 raft_consensus.cc:3037] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.740108 4163 tablet_bootstrap.cc:654] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.740437 3743 raft_consensus.cc:2454] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.741312 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44c67a0eb7485fb8dcde09416cd331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.743644 5161 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.744172 5161 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.744920 5161 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.745724 4169 ts_tablet_manager.cc:1918] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.745997 4169 log.cc:1196] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/125757b8c1a6448b841c63339f392339
I20260811 18:57:28.755154 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6ec419ed7747c3b24dc11992518e88" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.755666 3743 raft_consensus.cc:2454] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.662009 4167 rpcz_store.cc:276] 0811 18:57:25.349548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.349661 (+ 113us) service_pool.cc:227] Handling call
0811 18:57:28.661222 (+3311561us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:28.661235 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.661341 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3311268}
W20260811 18:57:28.756996 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 221d0d8d69f142a29ec7065a82b3f769 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 221d0d8d69f142a29ec7065a82b3f769 already in progress: creating tablet
I20260811 18:57:28.757097 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 631) took 3407 ms. Trace:
I20260811 18:57:28.757242 4167 rpcz_store.cc:276] 0811 18:57:25.349709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.350144 (+ 435us) service_pool.cc:227] Handling call
0811 18:57:28.756871 (+3406727us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:28.756898 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.757066 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3402604}
W20260811 18:57:28.757669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 68a7051bc539468aafba30fbaf0669d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68a7051bc539468aafba30fbaf0669d4 already in progress: creating tablet
I20260811 18:57:28.698938 4162 consensus_queue.cc:260] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.757957 4169 ts_tablet_manager.cc:1939] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.762791 4162 ts_tablet_manager.cc:1428] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.107s user 0.002s sys 0.000s
I20260811 18:57:28.761881 4163 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.763283 4163 ts_tablet_manager.cc:1397] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.003s
W20260811 18:57:28.763541 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1af33fd98c6486f975368ec90493db7 consensus state has no leader
I20260811 18:57:28.763967 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 629) took 3414 ms. Trace:
I20260811 18:57:28.764055 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 8e120a45539f49bb82ab0fb95eac36f9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.764205 4163 raft_consensus.cc:348] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.764511 4163 raft_consensus.cc:374] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.764578 4163 raft_consensus.cc:729] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.764050 4167 rpcz_store.cc:276] 0811 18:57:25.349275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.349815 (+ 540us) service_pool.cc:227] Handling call
0811 18:57:28.763842 (+3414027us) ts_tablet_manager.cc:1145] Deleting tablet 380c1f53b3ac4509a09b25a2800028af
0811 18:57:28.763853 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.763949 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3400270}
I20260811 18:57:28.764791 4163 consensus_queue.cc:260] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.765358 4163 ts_tablet_manager.cc:1428] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:28.762823 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 510) took 6425 ms. Trace:
I20260811 18:57:28.765466 4169 rpcz_store.cc:276] 0811 18:57:22.337499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.338669 (+ 1170us) service_pool.cc:227] Handling call
0811 18:57:28.651380 (+6312711us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:28.651395 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.726769 (+ 75374us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.745621 (+ 18852us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.762798 (+ 17177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6312276}
W20260811 18:57:28.766481 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 380c1f53b3ac4509a09b25a2800028af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 380c1f53b3ac4509a09b25a2800028af already in progress: creating tablet
I20260811 18:57:28.766597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 512) took 6428 ms. Trace:
I20260811 18:57:28.766678 4169 rpcz_store.cc:276] 0811 18:57:22.338611 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.339144 (+ 533us) service_pool.cc:227] Handling call
0811 18:57:28.766095 (+6426951us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:28.766375 (+ 280us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.766577 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6426592}
I20260811 18:57:28.767895 4163 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.756309 3824 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.768627 5161 raft_consensus.cc:2781] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.768889 5161 raft_consensus.cc:686] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.769186 5161 consensus_queue.cc:237] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.770511 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99bb8e934e6c44ca88640715b11e6bef consensus state has no leader
I20260811 18:57:28.770927 3824 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.771147 5182 raft_consensus.cc:2781] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.771209 5182 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.771261 5182 raft_consensus.cc:3037] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.774120 5182 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.775373 5182 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.767496 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 513) took 6427 ms. Trace:
I20260811 18:57:28.775663 4169 rpcz_store.cc:276] 0811 18:57:22.339811 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.339866 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:28.767282 (+6427416us) ts_tablet_manager.cc:1145] Deleting tablet e14f8717bcd94d6d88295fd8be130ca5
0811 18:57:28.767292 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.767478 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6427012}
I20260811 18:57:28.776190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 517) took 6381 ms. Trace:
I20260811 18:57:28.776353 4169 rpcz_store.cc:276] 0811 18:57:22.394591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.394654 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:28.776011 (+6381357us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:28.776024 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.776169 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6379644}
I20260811 18:57:28.776858 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 518) took 6351 ms. Trace:
I20260811 18:57:28.777002 4169 rpcz_store.cc:276] 0811 18:57:22.424934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.424985 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:28.776738 (+6351753us) ts_tablet_manager.cc:1145] Deleting tablet c6e9da8dfb4a4c6c9275c092fae7d910
0811 18:57:28.776751 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.776837 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6328926}
I20260811 18:57:28.777352 4169 tablet_replica.cc:330] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.777562 4162 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.777611 4169 raft_consensus.cc:2229] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.777933 4169 raft_consensus.cc:2258] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.778620 4169 ts_tablet_manager.cc:1905] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.779382 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 632) took 3416 ms. Trace:
I20260811 18:57:28.779639 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6ec419ed7747c3b24dc11992518e88" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.779930 3743 raft_consensus.cc:3037] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:28.781221 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3263da75f0d84528b7c8cc55dcaeaa46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3263da75f0d84528b7c8cc55dcaeaa46 already in progress: creating tablet
I20260811 18:57:28.783058 3743 raft_consensus.cc:2454] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.783555 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ce5b9945b46348149fb08dafb33f61b2" } new_table_name: "default.Table_5b05303fcd96430bb6ab896fbaa5097a" modify_external_catalogs: true
I20260811 18:57:28.786015 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0df46efea56e41b79fbb3e4dd034f84e" } new_table_name: "default.Table_62d056bd90b1458fb8d4452078546e14" modify_external_catalogs: true
W20260811 18:57:28.786667 3678 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.499s (SENT)
I20260811 18:57:28.788444 4863 fs_manager.cc:709] Time spent opening directory manager: real 0.054s user 0.002s sys 0.003s
W20260811 18:57:28.789991 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a847581412494e34b2c76e605ff11772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a847581412494e34b2c76e605ff11772 already in progress: creating tablet
W20260811 18:57:28.790575 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c6e9da8dfb4a4c6c9275c092fae7d910 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6e9da8dfb4a4c6c9275c092fae7d910 already in progress: creating tablet
#7 0x7fc7c23e62e6 in kudu::tserver::TSTabletManager::DeleteTablet(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, kudu::tserver::TabletServerErrorPB_Code*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1194:12
#8 0x7fc7c2418404 in kudu::tserver::DeleteTabletRunnable::Run() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1101:36
W20260811 18:57:28.792582 3678 leader_election.cc:334] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.300s (SENT)
I20260811 18:57:28.792719 3555 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c] alter complete (version 1)
I20260811 18:57:28.792989 4162 tablet_bootstrap.cc:654] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.779489 4167 rpcz_store.cc:276] 0811 18:57:25.363395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.367150 (+ 3755us) service_pool.cc:227] Handling call
0811 18:57:28.778401 (+3411251us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:28.778419 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.779362 (+ 943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3406203}
I20260811 18:57:28.791149 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.791711 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.791983 3824 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.795425 5080 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.796821 4163 tablet_bootstrap.cc:654] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.797015 5182 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.797600 5182 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.797516 5080 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.798142 5080 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.798131 5182 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.798457 5182 raft_consensus.cc:2781] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.798559 5182 raft_consensus.cc:686] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.798803 5182 consensus_queue.cc:237] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.801456 3678 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.387s (SENT)
I20260811 18:57:28.803088 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.803505 3743 raft_consensus.cc:2454] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.804580 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.804925 3743 raft_consensus.cc:2454] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:28.805343 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 45fde25bcb40493aafb083e161aa6932 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45fde25bcb40493aafb083e161aa6932 already in progress: creating tablet
I20260811 18:57:28.805449 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.805456 3824 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.805619 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 633) took 3167 ms. Trace:
I20260811 18:57:28.805747 4167 rpcz_store.cc:276] 0811 18:57:25.637953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.638210 (+ 257us) service_pool.cc:227] Handling call
0811 18:57:28.805515 (+3167305us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:28.805528 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.805601 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3167025}
I20260811 18:57:28.806327 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44c67a0eb7485fb8dcde09416cd331 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:28.807291 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because the tablet was not found. No further retry: Not found: Tablet not found: 33d995d72f8142afaeb57a2c29910fa3
I20260811 18:57:28.808219 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f89f09e89fbd4af7961d8db040ce3f49 (DEFAULT_TABLE table=default.Table_ea791cf775eb426ba289b61dfbaf8b0f [id=a8159550bbb94c04a3ebc52c05610daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.808866 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44c67a0eb7485fb8dcde09416cd331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.809684 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89f09e89fbd4af7961d8db040ce3f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.810861 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:28.811075 5182 raft_consensus.cc:2781] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.811254 5182 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.811385 5182 raft_consensus.cc:3037] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.814401 5182 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.815196 5182 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.815541 3824 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
#9 0x7fc7c23f9870 in kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&)::$_16::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1121:67
I20260811 18:57:28.817701 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.818015 3743 raft_consensus.cc:3037] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.809237 4167 tablet_replica.cc:330] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.818585 4167 raft_consensus.cc:2229] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:28.818908 4167 raft_consensus.cc:2258] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.819531 4167 ts_tablet_manager.cc:1905] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.821574 3743 raft_consensus.cc:2454] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.822129 5182 raft_consensus.cc:2781] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.822324 5182 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.822503 5182 raft_consensus.cc:3037] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.823189 5205 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.823320 5205 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.823907 5205 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.824807 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.828135 3676 leader_election.cc:341] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.828768 3854 tablet_service.cc:1467] Processing CreateTablet for tablet f89f09e89fbd4af7961d8db040ce3f49 (DEFAULT_TABLE table=default.Table_ea791cf775eb426ba289b61dfbaf8b0f [id=a8159550bbb94c04a3ebc52c05610daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.829241 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89f09e89fbd4af7961d8db040ce3f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.831030 3724 tablet_service.cc:1467] Processing CreateTablet for tablet ae75d6652c704e1995966c6dd4c06307 (DEFAULT_TABLE table=default.Table_706c784347b146089c032d699f6a9abe [id=2565a2c170c24cd592a1360a4491287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.831503 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae75d6652c704e1995966c6dd4c06307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.836259 5091 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.836433 5091 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.837069 5091 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:28.838523 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.840711 3676 leader_election.cc:341] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.842167 3854 tablet_service.cc:1467] Processing CreateTablet for tablet ae75d6652c704e1995966c6dd4c06307 (DEFAULT_TABLE table=default.Table_706c784347b146089c032d699f6a9abe [id=2565a2c170c24cd592a1360a4491287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.842778 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae75d6652c704e1995966c6dd4c06307. 1 dirs total, 0 dirs full, 0 dirs failed
#10 0x7fc7c23f95fd in std::_Function_handler<void (), kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&)::$_16>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:28.844864 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
#11 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:28.847630 4162 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.847816 4162 ts_tablet_manager.cc:1397] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.085s user 0.005s sys 0.000s
W20260811 18:57:28.850020 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21977ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:28.851157 3824 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.851430 5080 raft_consensus.cc:2781] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.851558 5080 raft_consensus.cc:686] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.851548 4162 raft_consensus.cc:348] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.851851 4162 raft_consensus.cc:374] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.851917 4162 raft_consensus.cc:729] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.851828 5080 consensus_queue.cc:237] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.855451 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (table default.Table_f5f50e8f17e04760a61e10c54d1aa657 [id=de9b6daef0ed416aa79b0dc0f4b02c2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:28.859315 5161 consensus_queue.cc:1035] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:28.860073 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_598311ac8416435286cfa4a6301380bd" } modify_external_catalogs: true
I20260811 18:57:28.860401 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_598311ac8416435286cfa4a6301380bd" } modify_external_catalogs: true
I20260811 18:57:28.871012 4163 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.871227 4163 ts_tablet_manager.cc:1397] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.103s user 0.001s sys 0.008s
I20260811 18:57:28.872603 4163 raft_consensus.cc:348] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.872888 4163 raft_consensus.cc:374] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.872953 4163 raft_consensus.cc:729] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.878298 4162 consensus_queue.cc:260] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.886754 4162 ts_tablet_manager.cc:1428] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.036s user 0.000s sys 0.002s
I20260811 18:57:28.887105 4162 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:28.888146 3678 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.260s (SENT)
I20260811 18:57:28.872977 5182 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.896965 5182 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.899442 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.899820 3743 raft_consensus.cc:3037] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.900085 4163 consensus_queue.cc:260] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.904639 4163 ts_tablet_manager.cc:1428] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.033s user 0.001s sys 0.002s
I20260811 18:57:28.904824 3743 raft_consensus.cc:2454] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.907464 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b203df4124364737839bf4c1ae072988
I20260811 18:57:28.908617 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.909372 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.910748 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:28.911253 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7df549360f014d629938e3e94659af8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.911815 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e2635923a5784be1839e861c2277e900"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.912487 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2635923a5784be1839e861c2277e900 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:28.912730 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7df549360f014d629938e3e94659af8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.914157 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b4dc2df4c6464d71bb13fe25e0046758
I20260811 18:57:28.914942 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_256aa74bb2884a0d83f0d17b159b9e49" } new_table_name: "default.Table_8518b7d355d14d14983f2c89aad83f5e" modify_external_catalogs: true
I20260811 18:57:28.915558 4163 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.915810 5230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:28.916896 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.917783 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:28.919355 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8d6cbc9a8ed4b798188572c6fc1ff52 has no consensus state
I20260811 18:57:28.922254 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2474ae972e9242b69f488b7a2fdbc481
I20260811 18:57:28.922989 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923457 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923862 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923887 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8e120a45539f49bb82ab0fb95eac36f9
I20260811 18:57:28.924427 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 8e120a45539f49bb82ab0fb95eac36f9 deleted and replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1
I20260811 18:57:28.924455 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.883148 4169 ts_tablet_manager.cc:1918] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.924871 4169 log.cc:1196] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:28.925436 4169 ts_tablet_manager.cc:1939] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.926457 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 521) took 6457 ms. Trace:
I20260811 18:57:28.926585 4169 rpcz_store.cc:276] 0811 18:57:22.469353 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.469444 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:28.777312 (+6307868us) ts_tablet_manager.cc:1145] Deleting tablet 494939a4eb5a45d6a123d174ad513bcb
0811 18:57:28.777321 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.809008 (+ 31687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.883043 (+ 74035us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.926437 (+ 43394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6277776}
I20260811 18:57:28.927062 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 534) took 6235 ms. Trace:
I20260811 18:57:28.927377 4169 rpcz_store.cc:276] 0811 18:57:22.746564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.746706 (+ 142us) service_pool.cc:227] Handling call
0811 18:57:28.926914 (+6180208us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:28.926927 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.927045 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6120374}
I20260811 18:57:28.927748 4169 tablet_replica.cc:330] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.928675 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 494939a4eb5a45d6a123d174ad513bcb (table default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]) successfully deleted
I20260811 18:57:28.928959 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 with delete_type TABLET_DATA_DELETED (replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:28.929080 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 452ee450778e4d33adfa7e2745b5d970 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 452ee450778e4d33adfa7e2745b5d970 already in progress: creating tablet
I20260811 18:57:28.930408 4163 tablet_bootstrap.cc:654] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.932960 4863 fs_manager.cc:725] Time spent opening block manager: real 0.068s user 0.002s sys 0.000s
I20260811 18:57:28.933257 4863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-zpfg"
I20260811 18:57:28.936452 4863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:28.937070 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet c77882ff8d974057a61f037f1943f58f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.912886 4162 tablet_bootstrap.cc:654] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.913304 4167 ts_tablet_manager.cc:1918] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.922574 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939458 4167 log.cc:1196] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:28.924157 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.924297 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.924727 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.925670 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 with delete_type TABLET_DATA_DELETED (replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939951 4167 ts_tablet_manager.cc:1939] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.929749 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939879 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3982bc659a664e949beb16b472873d9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.941110 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3982bc659a664e949beb16b472873d9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.941767 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 634) took 3303 ms. Trace:
I20260811 18:57:28.941874 4167 rpcz_store.cc:276] 0811 18:57:25.638128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.638541 (+ 413us) service_pool.cc:227] Handling call
0811 18:57:28.809182 (+3170641us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:28.809194 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.855148 (+ 45954us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.911722 (+ 56574us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.941741 (+ 30019us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170440}
I20260811 18:57:28.942179 4163 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.942584 5205 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.942606 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 460e03cc14444c83b6c08ca194ca5a8d (table default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]) successfully deleted
I20260811 18:57:28.942730 5205 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.943325 5205 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.943527 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f7a8c696f8df4bec92e19adef1eff6d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.944335 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7a8c696f8df4bec92e19adef1eff6d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.944514 3824 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.944515 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:28.945557 5229 raft_consensus.cc:2781] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.945770 5229 raft_consensus.cc:686] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
W20260811 18:57:28.945801 3676 leader_election.cc:341] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.946512 5229 consensus_queue.cc:237] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.947901 4162 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.948071 4162 ts_tablet_manager.cc:1397] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.003s
I20260811 18:57:28.948932 4162 raft_consensus.cc:348] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.949183 4162 raft_consensus.cc:374] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.949281 4162 raft_consensus.cc:729] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.949515 4162 consensus_queue.cc:260] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.950217 4169 raft_consensus.cc:2229] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.951763 4169 raft_consensus.cc:2258] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.952643 4169 ts_tablet_manager.cc:1905] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.942320 4163 ts_tablet_manager.cc:1397] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.004s
I20260811 18:57:28.955943 4163 raft_consensus.cc:348] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#12 0x7fc7b32c1344 in kudu::ThreadPool::DispatchThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:776:7
#13 0x7fc7b32c491b in kudu::ThreadPool::CreateThread()::$_2::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:849:48
#14 0x7fc7b32c4798 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
previously allocated by thread T135 (tablet-open [wo) here:
I20260811 18:57:28.970868 4162 ts_tablet_manager.cc:1428] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
#0 0x8c3cc8 in operator new(unsigned long) /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99:3
#1 0x7fc7bf9f51a5 in kudu::tablet::TabletBootstrap::OpenTablet(bool*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:693:29
#2 0x7fc7bf9f35ef in kudu::tablet::TabletBootstrap::RunBootstrap(std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:644:3
#3 0x7fc7bf9f1fd3 in kudu::tablet::TabletBootstrap::Bootstrap(std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:579:29
#4 0x7fc7bf9f18ac in kudu::tablet::BootstrapTablet(scoped_refptr<kudu::tablet::TabletMetadata>, kudu::consensus::RaftConfigPB, kudu::clock::Clock*, std::shared_ptr<kudu::MemTracker>, scoped_refptr<kudu::rpc::ResultTracker>, kudu::MetricRegistry*, kudu::FileCache*, scoped_refptr<kudu::tablet::TabletReplica>, scoped_refptr<kudu::log::LogAnchorRegistry>, std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:521:3
I20260811 18:57:28.977516 4162 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
#5 0x7fc7c23e19a6 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr<kudu::tablet::TabletReplica> const&, scoped_refptr<kudu::tserver::TransitionInProgressDeleter> const&, std::atomic<int>*, std::atomic<int>*, kudu::Timer*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1406:9
#6 0x7fc7c23f814a in kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:703:3
I20260811 18:57:28.981016 4162 tablet_bootstrap.cc:654] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#7 0x7fc7c23f7e8d in std::_Function_handler<void (), kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#8 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#9 0x7fc7b32c1344 in kudu::ThreadPool::DispatchThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:776:7
#10 0x7fc7b32c491b in kudu::ThreadPool::CreateThread()::$_2::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:849:48
#11 0x7fc7b32c4798 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#12 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#13 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#14 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Thread T54 (rpc worker-4005) created by T0 here:
W20260811 18:57:28.995182 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13788f40714847ff8ce6a810ac53e398 consensus state has no leader
I20260811 18:57:28.998127 4162 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.998306 4162 ts_tablet_manager.cc:1397] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20260811 18:57:28.999526 4162 raft_consensus.cc:348] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.999764 4162 raft_consensus.cc:374] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.999819 4162 raft_consensus.cc:729] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.978933 4169 ts_tablet_manager.cc:1918] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:28.991380 3824 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.047s (SENT)
I20260811 18:57:29.001914 3824 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:29.002250 3824 raft_consensus.cc:2670] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:29.002987 3824 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.612s (SENT)
I20260811 18:57:29.003110 3824 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:29.003259 3824 raft_consensus.cc:2670] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:29.005879 4169 log.cc:1196] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:29.006523 4169 ts_tablet_manager.cc:1939] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.007225 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 535) took 6202 ms. Trace:
I20260811 18:57:29.007426 4169 rpcz_store.cc:276] 0811 18:57:22.804771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.804843 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:28.927704 (+6122861us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:28.927716 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.960512 (+ 32796us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.978811 (+ 18299us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.007203 (+ 28392us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6048702}
I20260811 18:57:29.007872 4169 tablet_replica.cc:330] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.008383 4169 raft_consensus.cc:2229] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.008345 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:29.008805 4169 raft_consensus.cc:2258] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.009579 4169 ts_tablet_manager.cc:1905] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.010089 4162 consensus_queue.cc:260] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.009789 5229 consensus_queue.cc:1035] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:29.013443 4162 ts_tablet_manager.cc:1428] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260811 18:57:29.013652 4163 raft_consensus.cc:374] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.013829 4163 raft_consensus.cc:729] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.018232 4162 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.027741 4162 tablet_bootstrap.cc:654] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.014163 4163 consensus_queue.cc:260] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.033759 4163 ts_tablet_manager.cc:1428] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.079s user 0.000s sys 0.002s
I20260811 18:57:29.048545 4163 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.049328 4162 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.049485 4162 ts_tablet_manager.cc:1397] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260811 18:57:29.050237 4162 raft_consensus.cc:348] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.050488 4162 raft_consensus.cc:374] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.050545 4162 raft_consensus.cc:729] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.050781 4162 consensus_queue.cc:260] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.052045 4162 ts_tablet_manager.cc:1428] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:29.052783 3824 consensus_peers.cc:487] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff1be026ca404eada53f60f257ea367c. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:29.054263 4162 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.055924 4162 tablet_bootstrap.cc:654] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.059952 4163 tablet_bootstrap.cc:654] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:29.062971 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114d22f22d3d4dc09563aa1108e1f509 consensus state has no leader
I20260811 18:57:29.064560 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c77882ff8d974057a61f037f1943f58f
I20260811 18:57:29.066404 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet c77882ff8d974057a61f037f1943f58f with delete_type TABLET_DATA_DELETED (replaced by tablet c0f68d33e1b14144b34c17e3c7201408) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.067569 3554 catalog_manager.cc:6565] ReplaceTablet: tablet c77882ff8d974057a61f037f1943f58f deleted and replaced by tablet c0f68d33e1b14144b34c17e3c7201408
I20260811 18:57:29.067538 4169 ts_tablet_manager.cc:1918] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.067675 4169 log.cc:1196] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:29.068087 4169 ts_tablet_manager.cc:1939] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.068794 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet b57ed7814a1e440cba362613cbab32f8 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:29.068696 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 102259us
I20260811 18:57:29.069821 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b012268c924347d8a87ea52350010dc3" } new_table_name: "default.Table_a055754403774bc0a94fda2f957c70d5" modify_external_catalogs: true
I20260811 18:57:29.072072 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48" } new_table_name: "default.Table_0db19ab70f764e00b17ee2e0f4742b8b" modify_external_catalogs: true
I20260811 18:57:29.072484 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet c77882ff8d974057a61f037f1943f58f with delete_type TABLET_DATA_DELETED (replaced by tablet c0f68d33e1b14144b34c17e3c7201408) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.073624 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 536) took 6214 ms. Trace:
I20260811 18:57:29.073750 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b8dd0414c4784bbea975b8d3f1491cd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.074934 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8dd0414c4784bbea975b8d3f1491cd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.076874 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.077689 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f14f93014a8d464ba0da0bab0ceb9ddc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.077867 5080 consensus_queue.cc:1035] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:29.080885 4162 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.081032 4162 ts_tablet_manager.cc:1397] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:29.083009 4162 raft_consensus.cc:348] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.083235 4162 raft_consensus.cc:374] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.083285 4162 raft_consensus.cc:729] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.083496 4162 consensus_queue.cc:260] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.084473 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 635) took 3331 ms. Trace:
I20260811 18:57:29.084569 4167 rpcz_store.cc:276] 0811 18:57:25.753465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.753547 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:29.074165 (+3320618us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:29.074180 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.084450 (+ 10270us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3320213}
I20260811 18:57:29.085142 4162 ts_tablet_manager.cc:1428] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260811 18:57:29.085278 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8304ac7709ca4392baec4cfa78ef5b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8304ac7709ca4392baec4cfa78ef5b41 already in progress: creating tablet
I20260811 18:57:29.085723 4162 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.087690 4162 tablet_bootstrap.cc:654] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.090449 4167 tablet_replica.cc:330] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.090919 4167 raft_consensus.cc:2229] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.091229 4167 raft_consensus.cc:2258] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.092183 4167 ts_tablet_manager.cc:1905] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.109354 4162 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.109520 4162 ts_tablet_manager.cc:1397] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260811 18:57:29.110265 4162 raft_consensus.cc:348] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.110529 4162 raft_consensus.cc:374] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.110590 4162 raft_consensus.cc:729] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.110785 4162 consensus_queue.cc:260] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.112239 4162 ts_tablet_manager.cc:1428] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:29.117802 4162 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.121546 4162 tablet_bootstrap.cc:654] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.123848 4169 rpcz_store.cc:276] 0811 18:57:22.859061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.859942 (+ 881us) service_pool.cc:227] Handling call
0811 18:57:29.007817 (+6147875us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:29.007830 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.018191 (+ 10361us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.067367 (+ 49176us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.073601 (+ 6234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6114238}
I20260811 18:57:29.125335 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 94943cfbe41a47b3a36c81957a3adaeb (table default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]) successfully deleted
I20260811 18:57:29.125391 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 538) took 6235 ms. Trace:
I20260811 18:57:29.125494 4169 rpcz_store.cc:276] 0811 18:57:22.906010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.906277 (+ 267us) service_pool.cc:227] Handling call
0811 18:57:29.125175 (+6218898us) ts_tablet_manager.cc:1145] Deleting tablet b20872d80c9545e5822bfc82d496ce97
0811 18:57:29.125191 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.125366 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6181656}
W20260811 18:57:29.129118 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b20872d80c9545e5822bfc82d496ce97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b20872d80c9545e5822bfc82d496ce97 already in progress: creating tablet
I20260811 18:57:29.129277 4167 ts_tablet_manager.cc:1918] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:29.129276 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 541) took 6009 ms. Trace:
I20260811 18:57:29.129390 4167 log.cc:1196] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a443fe209556489abb47f4faa5d696e5
I20260811 18:57:29.129374 4169 rpcz_store.cc:276] 0811 18:57:23.119734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.119799 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.129092 (+6009293us) ts_tablet_manager.cc:1145] Deleting tablet 21cc1822d4b94081ad86a6177acb3afe
0811 18:57:29.129118 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.129251 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6008748}
I20260811 18:57:29.129779 4167 ts_tablet_manager.cc:1939] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:29.132292 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 21cc1822d4b94081ad86a6177acb3afe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21cc1822d4b94081ad86a6177acb3afe already in progress: creating tablet
I20260811 18:57:29.132417 4169 tablet_replica.cc:330] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.137142 4169 raft_consensus.cc:2229] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.138283 4169 pending_rounds.cc:62] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:29.139191 4169 pending_rounds.cc:68] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400161482723328 op_type: NO_OP noop_request { }
I20260811 18:57:29.140074 4169 raft_consensus.cc:2866] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:29.140559 4169 raft_consensus.cc:2258] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.142393 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddc87769613d45caa57f2df0032dc979"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.143412 4169 ts_tablet_manager.cc:1905] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.143558 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddc87769613d45caa57f2df0032dc979 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.147686 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" } modify_external_catalogs: true
I20260811 18:57:29.148083 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" } modify_external_catalogs: true
I20260811 18:57:29.149155 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_068dc550d67d413c916d1384c3cb811c" } modify_external_catalogs: true
I20260811 18:57:29.149509 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_068dc550d67d413c916d1384c3cb811c" } modify_external_catalogs: true
I20260811 18:57:29.150250 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 636) took 3381 ms. Trace:
I20260811 18:57:29.150370 4167 rpcz_store.cc:276] 0811 18:57:25.769033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.769221 (+ 188us) service_pool.cc:227] Handling call
0811 18:57:29.085947 (+3316726us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:29.085958 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.113159 (+ 27201us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.129158 (+ 15999us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.150223 (+ 21065us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3316432}
W20260811 18:57:29.150534 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c142d8ef40947e29f3c94e1e8e4c517 consensus state has no leader
I20260811 18:57:29.150740 4167 tablet_replica.cc:330] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.150885 5205 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.150939 4167 raft_consensus.cc:2229] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.151013 5205 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.151216 4167 raft_consensus.cc:2258] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.151233 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_647ae82e63464976abb7f26c9526a349" } modify_external_catalogs: true
I20260811 18:57:29.151607 5205 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.151546 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_647ae82e63464976abb7f26c9526a349" } modify_external_catalogs: true
I20260811 18:57:29.151747 4167 ts_tablet_manager.cc:1905] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.152302 4162 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.152446 4162 ts_tablet_manager.cc:1397] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.001s
I20260811 18:57:29.152426 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.152523 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a443fe209556489abb47f4faa5d696e5 (table default.Table_bf28eaf7393d4d508363fbc8dd91686b [id=1a0803466858425cadc33f51f0933a95]) successfully deleted
W20260811 18:57:29.153091 3676 leader_election.cc:341] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.153841 4162 raft_consensus.cc:348] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.154115 4162 raft_consensus.cc:374] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.154191 4162 raft_consensus.cc:729] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.154506 4162 consensus_queue.cc:260] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.157343 4162 ts_tablet_manager.cc:1428] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:29.162781 4162 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.169283 4162 tablet_bootstrap.cc:654] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.176014 4169 ts_tablet_manager.cc:1918] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.187009 4863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:29.189262 4863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:29.189965 4863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:29.190677 4167 ts_tablet_manager.cc:1918] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.190829 4167 log.cc:1196] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:29.191069 4169 log.cc:1196] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:29.191303 4167 ts_tablet_manager.cc:1939] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.192239 4162 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.192473 4162 ts_tablet_manager.cc:1397] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.005s
I20260811 18:57:29.193801 4162 raft_consensus.cc:348] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.194177 4162 raft_consensus.cc:374] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.194319 4162 raft_consensus.cc:729] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.194698 4162 consensus_queue.cc:260] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.195641 4162 ts_tablet_manager.cc:1428] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:29.196269 4162 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.198815 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 637) took 3429 ms. Trace:
I20260811 18:57:29.198932 4167 rpcz_store.cc:276] 0811 18:57:25.769165 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.769582 (+ 417us) service_pool.cc:227] Handling call
0811 18:57:29.150682 (+3381100us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:29.150696 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.170337 (+ 19641us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.190547 (+ 20210us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.198792 (+ 8245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3380923}
I20260811 18:57:29.199338 4167 tablet_replica.cc:330] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:29.199450 3678 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.326s (SENT)
I20260811 18:57:29.199571 4167 raft_consensus.cc:2229] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.199749 4167 raft_consensus.cc:2258] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.199786 5205 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.199887 5205 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.200342 4167 ts_tablet_manager.cc:1905] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.200498 5205 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.200703 4162 tablet_bootstrap.cc:654] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.201236 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.201424 4169 ts_tablet_manager.cc:1939] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.202028 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 542) took 6079 ms. Trace:
W20260811 18:57:29.202116 3676 leader_election.cc:341] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.202132 4169 rpcz_store.cc:276] 0811 18:57:23.122710 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.122770 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:29.132355 (+6009585us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:29.132370 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.171893 (+ 39523us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.175872 (+ 3979us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.202003 (+ 26131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6009135}
I20260811 18:57:29.202782 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f61d3822fc2d4b569f223238a6464e0a (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:29.219609 4169 tablet_replica.cc:330] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:29.219951 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3dd4ec30acde4585a1c3451927217cb3 consensus state has no leader
I20260811 18:57:29.220009 4169 raft_consensus.cc:2229] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.220305 4169 raft_consensus.cc:2258] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.220783 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.221144 4169 ts_tablet_manager.cc:1905] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.221642 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.223269 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e425950244124732b380187898a2b68f" } new_table_name: "default.Table_1e5db5512ab043148049e8476163a5a8" modify_external_catalogs: true
I20260811 18:57:29.227041 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c" } modify_external_catalogs: true
I20260811 18:57:29.227342 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c" } modify_external_catalogs: true
I20260811 18:57:29.228309 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b3ee7e916a0c4d2180027e84bc055f54" } new_table_name: "default.Table_5627fd661e4441d8a677d35bc0f16cf0" modify_external_catalogs: true
I20260811 18:57:29.229394 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 4b6ec419ed7747c3b24dc11992518e88 (table default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]) successfully deleted
I20260811 18:57:29.237694 4167 ts_tablet_manager.cc:1918] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.237923 4167 log.cc:1196] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:29.243268 4163 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:29.243554 4163 ts_tablet_manager.cc:1397] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.195s user 0.005s sys 0.017s
I20260811 18:57:29.245180 4163 raft_consensus.cc:348] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.245536 4163 raft_consensus.cc:374] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.245652 4163 raft_consensus.cc:729] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.246059 4163 consensus_queue.cc:260] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.247314 4167 ts_tablet_manager.cc:1939] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.248404 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 638) took 3448 ms. Trace:
I20260811 18:57:29.248570 4167 rpcz_store.cc:276] 0811 18:57:25.799648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.799719 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:29.199284 (+3399565us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:29.199297 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.221998 (+ 22701us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.237522 (+ 15524us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.248375 (+ 10853us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3399170}
I20260811 18:57:29.249244 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:29.249939 4163 ts_tablet_manager.cc:1428] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:29.254923 4167 tablet_replica.cc:330] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.255221 4167 raft_consensus.cc:2229] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.255597 4167 raft_consensus.cc:2258] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.256278 4167 ts_tablet_manager.cc:1905] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.267633 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50af4e9b20a74ed983b26281c3c6eda1 consensus state has no leader
W20260811 18:57:29.268165 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:29.269268 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 6880ad27a3eb4599b407edeb2969a08f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:29.273221 4163 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.284490 5205 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.284687 5205 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.285526 5205 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.286316 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.287106 3676 leader_election.cc:341] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.302026 4167 ts_tablet_manager.cc:1918] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:29.304296 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_16d7946e87be4700a338a7744c866d5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.305267 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16d7946e87be4700a338a7744c866d5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.307447 4167 log.cc:1196] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:29.307982 4167 ts_tablet_manager.cc:1939] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.308498 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a677dcb5a43648a49944a18f38fb74fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.309195 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a677dcb5a43648a49944a18f38fb74fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.310608 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 639) took 3494 ms. Trace:
I20260811 18:57:29.310714 4167 rpcz_store.cc:276] 0811 18:57:25.816103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.816243 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:29.254831 (+3438588us) ts_tablet_manager.cc:1145] Deleting tablet 01d564ad80064912a8ce8e31c1b6c4d2
0811 18:57:29.254847 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.278671 (+ 23824us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.301867 (+ 23196us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.310583 (+ 8716us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3438328}
I20260811 18:57:29.311528 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]) successfully deleted
I20260811 18:57:29.313643 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 640) took 3381 ms. Trace:
I20260811 18:57:29.313755 4167 rpcz_store.cc:276] 0811 18:57:25.932436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.932489 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:29.313456 (+3380967us) ts_tablet_manager.cc:1145] Deleting tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
0811 18:57:29.313469 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.313620 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3380761}
W20260811 18:57:29.314352 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa already in progress: creating tablet
I20260811 18:57:29.317706 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d79d187445ee43d6864f03f905400405" } new_table_name: "default.Table_c499c7d38b65414692ba6f4d7307a7bf" modify_external_catalogs: true
I20260811 18:57:29.318878 4167 tablet_replica.cc:330] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.319128 4167 raft_consensus.cc:2229] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.321127 4167 raft_consensus.cc:2258] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.323468 5205 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.323655 5205 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.324635 5205 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.325335 4167 ts_tablet_manager.cc:1905] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.327394 4162 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.327538 4162 ts_tablet_manager.cc:1397] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.007s
I20260811 18:57:29.330045 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:29.331624 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_32f181a8b481409187abd0dc0a1765ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.332587 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32f181a8b481409187abd0dc0a1765ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.335740 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_944ef43f4d664e43b89a55f61dffcfb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.336534 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_944ef43f4d664e43b89a55f61dffcfb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.337774 3824 consensus_peers.cc:487] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67e63ee9b9fd4b12ae791b9d5bc2e648. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:29.343546 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.346602 3676 leader_election.cc:341] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.330745 4162 raft_consensus.cc:348] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.348469 4162 raft_consensus.cc:374] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.349081 4162 raft_consensus.cc:729] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.349844 4163 tablet_bootstrap.cc:654] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.349654 4162 consensus_queue.cc:260] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.358845 4162 ts_tablet_manager.cc:1428] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.031s user 0.003s sys 0.000s
I20260811 18:57:29.365170 3854 tablet_service.cc:1467] Processing CreateTablet for tablet a3466b2364d64b0cad93b68732e0dae2 (DEFAULT_TABLE table=default.Table_a79b27c0bd524af985c2e6167624551a [id=b46bfa34023d4fdf8d4864dec08ce39b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.365712 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3466b2364d64b0cad93b68732e0dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.372587 4162 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.374068 4169 ts_tablet_manager.cc:1918] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.5
I20260811 18:57:29.375492 3708 tablet_service.cc:1467] Processing CreateTablet for tablet a3466b2364d64b0cad93b68732e0dae2 (DEFAULT_TABLE table=default.Table_a79b27c0bd524af985c2e6167624551a [id=b46bfa34023d4fdf8d4864dec08ce39b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.375959 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3466b2364d64b0cad93b68732e0dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.383203 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1ddb10e5ee5b441c875dd0b345201949 (DEFAULT_TABLE table=default.Table_6064366e4ea94146903cb48b739102e9 [id=94bd67a82648467ab2ffb80d4e1b9ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.383852 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ddb10e5ee5b441c875dd0b345201949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.386766 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5638cefdec734ed58f3de696740ba891 (DEFAULT_TABLE table=default.Table_832e411b94e24eb49b8798f66fa24774 [id=8b596fd1e4af496680f105839144bf79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.387250 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 08f4d8bbe23b463680c153ae4ff4ebcf (DEFAULT_TABLE table=default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a [id=7de6fc7b04824669845b02a164b3a819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.397184 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5638cefdec734ed58f3de696740ba891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.403235 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f4d8bbe23b463680c153ae4ff4ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.404748 5205 raft_consensus.cc:482] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.404934 5205 raft_consensus.cc:504] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.406004 5205 leader_election.cc:288] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:29.407555 4167 ts_tablet_manager.cc:1918] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.407722 4167 log.cc:1196] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:29.408268 4167 ts_tablet_manager.cc:1939] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.408767 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 641) took 3471 ms. Trace:
I20260811 18:57:29.408902 4167 rpcz_store.cc:276] 0811 18:57:25.936853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.937016 (+ 163us) service_pool.cc:227] Handling call
0811 18:57:29.318823 (+3381807us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:29.318836 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.329231 (+ 10395us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.404557 (+ 75326us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.408746 (+ 4189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3381559}
I20260811 18:57:29.409778 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b6f36ab6bb774257b5c2692c64eff63d (table default.Table_e0c0cbe2c2f846c2b85cda84670f448c [id=22529e4acc82410a93bca07dfda7d1b9]) successfully deleted
I20260811 18:57:29.415068 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 642) took 3460 ms. Trace:
I20260811 18:57:29.415233 4167 rpcz_store.cc:276] 0811 18:57:25.954790 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.954855 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.414915 (+3460060us) ts_tablet_manager.cc:1145] Deleting tablet 84e152c74f33463e83843b92c0783803
0811 18:57:29.414929 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.415046 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3459857}
W20260811 18:57:29.415788 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 84e152c74f33463e83843b92c0783803 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e152c74f33463e83843b92c0783803 already in progress: creating tablet
I20260811 18:57:29.417181 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 643) took 3461 ms. Trace:
I20260811 18:57:29.417279 4167 rpcz_store.cc:276] 0811 18:57:25.956113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.956171 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:29.417030 (+3460859us) ts_tablet_manager.cc:1145] Deleting tablet 649cb98f553a418b911e307fdaf49917
0811 18:57:29.417052 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.417150 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1792,"tablet-delete.queue_time_us":3460531}
W20260811 18:57:29.417788 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 649cb98f553a418b911e307fdaf49917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 649cb98f553a418b911e307fdaf49917 already in progress: creating tablet
I20260811 18:57:29.420476 4169 log.cc:1196] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:29.420522 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515d5b5afbd74ec5ab90a503aef2f4fb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.420850 3887 raft_consensus.cc:2454] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:29.421278 4169 ts_tablet_manager.cc:1939] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.421417 3676 leader_election.cc:302] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:29.421881 5205 raft_consensus.cc:2781] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:29.421979 5205 raft_consensus.cc:482] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:29.422065 5205 raft_consensus.cc:3037] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:29.425648 5205 raft_consensus.cc:504] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.427031 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515d5b5afbd74ec5ab90a503aef2f4fb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:29.427371 3887 raft_consensus.cc:3037] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:29.428320 5205 leader_election.cc:288] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:29.430250 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 4149626f48354ba0923cad28b95be048 (DEFAULT_TABLE table=default.Table_5296710afead477eb67f356c9afbc4a4 [id=54cc236881e246a18f2af7ae6df6b983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.430730 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 23c85b2315254e4c8d9d69fec4cc9aa2 (DEFAULT_TABLE table=default.Table_e79b6e054e914fd39ca0a6282eb0ce17 [id=8d92dfcc02424160a65fca84e2417a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.431049 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 72f11748cf984942ba99bda4ad6605ab (DEFAULT_TABLE table=default.Table_c6284ad4471b4b7095b11fe7a8f304e2 [id=53f3832432d242058789ef564beeae06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.429807 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 468c71415a984ea7ab04c21fa9612070 (DEFAULT_TABLE table=default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 [id=4a0a8c18d3114ce4a1c4e69e733d372b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.433403 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 543) took 6179 ms. Trace:
I20260811 18:57:29.433533 4169 rpcz_store.cc:276] 0811 18:57:23.253508 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.255514 (+ 2006us) service_pool.cc:227] Handling call
0811 18:57:29.219545 (+5964031us) ts_tablet_manager.cc:1145] Deleting tablet 9de9acdb33a2438eb4378a5eac844c35
0811 18:57:29.219559 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.335174 (+115615us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.373885 (+ 38711us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.433374 (+ 59489us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5924771}
I20260811 18:57:29.434015 4169 tablet_replica.cc:330] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.434311 4169 raft_consensus.cc:2229] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.434549 4169 raft_consensus.cc:2258] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.435293 4169 ts_tablet_manager.cc:1905] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.438956 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 644) took 3474 ms. Trace:
I20260811 18:57:29.439127 4167 rpcz_store.cc:276] 0811 18:57:25.964049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.964374 (+ 325us) service_pool.cc:227] Handling call
0811 18:57:29.419726 (+3455352us) ts_tablet_manager.cc:1145] Deleting tablet 86d690ec09524248bc147ad7408e2ee6
0811 18:57:29.419751 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.438914 (+ 19163us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3455131}
I20260811 18:57:29.440249 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c85b2315254e4c8d9d69fec4cc9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.440640 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468c71415a984ea7ab04c21fa9612070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.446676 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f11748cf984942ba99bda4ad6605ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.450572 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4149626f48354ba0923cad28b95be048. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.457033 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06c8776bb47c4cb891ef13a2a1e36781 consensus state has no leader
I20260811 18:57:29.459256 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 47a782b8470b4d14ac89b3b7d463a125
I20260811 18:57:29.459568 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 645) took 3495 ms. Trace:
I20260811 18:57:29.459734 4167 rpcz_store.cc:276] 0811 18:57:25.964321 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.964617 (+ 296us) service_pool.cc:227] Handling call
0811 18:57:29.459312 (+3494695us) ts_tablet_manager.cc:1145] Deleting tablet 6d28c486e408451a990ae3905ef3c2ff
0811 18:57:29.459335 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.459500 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3494225}
I20260811 18:57:29.460476 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6880ad27a3eb4599b407edeb2969a08f
I20260811 18:57:29.460817 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]). Expected version 3 got 0
I20260811 18:57:29.461102 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:29.461414 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.462782 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1c74abe9dcf642e9b4d40df6ff3639ef
I20260811 18:57:29.463042 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8fb6e34f004c466381fe6ab5b780568f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.464366 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 646) took 3498 ms. Trace:
I20260811 18:57:29.465549 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1d721b0e55384d8a8efd88a0e47b1504
I20260811 18:57:29.466154 3551 catalog_manager.cc:5495] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:29.466889 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:29.467068 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:29.464485 4167 rpcz_store.cc:276] 0811 18:57:25.965826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.965881 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.464178 (+3498297us) ts_tablet_manager.cc:1145] Deleting tablet b7b48a35acf1486bbb63680ca5da31dd
0811 18:57:29.464195 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.464336 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3498113}
I20260811 18:57:29.469450 4167 tablet_replica.cc:330] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.469542 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5781967ec2ee41a5ab982d9d7394c274" } new_table_name: "default.Table_e03268eb26a144ba8b53cbd960628036" modify_external_catalogs: true
I20260811 18:57:29.469754 4167 raft_consensus.cc:2229] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.469828 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 6880ad27a3eb4599b407edeb2969a08f deleted and replaced by tablet 101994cc708a4fadb46fc8c5915a5587
I20260811 18:57:29.469997 4167 raft_consensus.cc:2258] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.482177 4167 ts_tablet_manager.cc:1905] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.482357 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_862e6dcdf4f748738d4f651170e5611f" } new_table_name: "default.Table_2940279efa3f41ec980785fb086dd12d" modify_external_catalogs: true
I20260811 18:57:29.487917 3887 raft_consensus.cc:2454] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:29.489830 3676 leader_election.cc:302] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:29.490226 5091 raft_consensus.cc:2781] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:29.490387 5091 raft_consensus.cc:686] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:29.490615 5091 consensus_queue.cc:237] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.492188 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_13786cfeafbc4312bf84f93b204add4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.493139 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13786cfeafbc4312bf84f93b204add4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.494406 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fb6e34f004c466381fe6ab5b780568f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.494874 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbf1868109474dcea3712c10bb4d4f3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.499748 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6d28c486e408451a990ae3905ef3c2ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d28c486e408451a990ae3905ef3c2ff already in progress: creating tablet
I20260811 18:57:29.500090 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cef66a1c435645d6822188ec4776526b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.503145 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cef66a1c435645d6822188ec4776526b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.500452 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b38a0019b5541c495088e96174c8de4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.506925 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b38a0019b5541c495088e96174c8de4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.508513 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880ad27a3eb4599b407edeb2969a08f with delete_type TABLET_DATA_DELETED (replaced by tablet 101994cc708a4fadb46fc8c5915a5587) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.508543 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.509114 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ce3acbad48bb431fad5b6904dec8e5ae (DEFAULT_TABLE table=default.Table_9743ba51a07b4fca8aaf1a188a692aac [id=4e7d8b7d88074d04b97ad14a02626671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.509681 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.510070 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.510396 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.516793 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 73e15954a1604189af634a727cb48690 (DEFAULT_TABLE table=default.Table_3de36d9c6d4542be9ee2c873f0bfcfca [id=d446e82dcf024d6e972911e406ca588a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.519390 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b752cb750e9447b092ecac296b5f836e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.520460 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3acbad48bb431fad5b6904dec8e5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.521515 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e15954a1604189af634a727cb48690. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.524968 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b752cb750e9447b092ecac296b5f836e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.526919 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b7b48a35acf1486bbb63680ca5da31dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b48a35acf1486bbb63680ca5da31dd already in progress: creating tablet
I20260811 18:57:29.529348 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.529106 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0495c5b770b34d54afb1379a6e36204c (DEFAULT_TABLE table=default.Table_cafc7915ae7e4280a4d71e3bce4ca57c [id=bcffeabc4e924226a6c508ea5540bb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.529678 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.530052 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.530107 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.531629 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb3e997b4454598bdce102e92296887 (DEFAULT_TABLE table=default.Table_f9cadd28493e444185170bcd4c314208 [id=d0d78361eb3d4f0e8a6fa132a05a34e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.532089 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 2ee5acbaa9f54d4ba523b603641e4a97 (DEFAULT_TABLE table=default.Table_9914a933914f4377ba70e0b57c70edff [id=6d6cbea588c54de4a0966727cc2efaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.532588 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 8319453f68a54da9813747152ef08756 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.533335 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb3e997b4454598bdce102e92296887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.534607 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8319453f68a54da9813747152ef08756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.538118 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ee5acbaa9f54d4ba523b603641e4a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.544219 5080 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.544389 5080 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.545250 5080 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:29.551940 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 385fef74b0c1421ea86c23bfee38c681 (DEFAULT_TABLE table=default.Table_ef4494689f0147118d362ed74d1bf78c [id=bd0e71434abf458795a3e2f8b6251847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.555764 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385fef74b0c1421ea86c23bfee38c681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.557940 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0495c5b770b34d54afb1379a6e36204c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.564144 4169 ts_tablet_manager.cc:1918] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.565019 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 90fde17971ea49aaad212a5836661b08 (DEFAULT_TABLE table=default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 [id=e354251945a14c9fb3846267e17bcb74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.570399 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fde17971ea49aaad212a5836661b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.572531 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f3670f8908d54ebb82f698cdd207e48f
I20260811 18:57:29.580194 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9dd303f12b584056a04e385cc80faffe" } new_table_name: "default.Table_9bb28cb6f9204966a43b9a2304b164ac" modify_external_catalogs: true
I20260811 18:57:29.578935 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 985dac332bff463781ae93c36f45fdda (DEFAULT_TABLE table=default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a [id=cc40efcc988e48c7b37a10f5f1217956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.583912 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet f3670f8908d54ebb82f698cdd207e48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.584261 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.585078 3706 tablet_service.cc:1467] Processing CreateTablet for tablet c8d6cbc9a8ed4b798188572c6fc1ff52 (DEFAULT_TABLE table=default.Table_5b05303fcd96430bb6ab896fbaa5097a [id=e81a772f5829416f974f5f6a19cc09b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.592680 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d6cbc9a8ed4b798188572c6fc1ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.585606 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 (DEFAULT_TABLE table=default.Table_0c3df98432314ff2b4cbd7004701b258 [id=6e4e47c552c943aca96b05e0959d681d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.594803 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9b279c6c7344f1931b81e10248b1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.586066 3720 tablet_service.cc:1467] Processing CreateTablet for tablet d32212c817c94fad8379462b027e7d3d (DEFAULT_TABLE table=default.Table_f0a164081b1349d9bc89f20d1ea6425d [id=e76087cfeb3c4a338de2a131620bc473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.598771 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32212c817c94fad8379462b027e7d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.586637 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a (DEFAULT_TABLE table=default.Table_378dd90b6af541b8ab95d9f19681a03f [id=d43a5bc491c048718e2504b3d22d2890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.602721 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.605551 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 476da7e2681540a89824b58fb7237df9 consensus state has no leader
I20260811 18:57:29.606225 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c86ddee4faa543c79b02e248ac801fb3" } new_table_name: "default.Table_c71b8760db044bad851b4bcd819bf75f" modify_external_catalogs: true
I20260811 18:57:29.606315 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5aa83c98b49c466cb01cdf92154ad384"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.607218 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5aa83c98b49c466cb01cdf92154ad384 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.615837 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e32f59bdd0a54fcb9c76d7d9132d966c consensus state has no leader
I20260811 18:57:29.626786 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 86a57b040ab44812a2001705493f2bcd (DEFAULT_TABLE table=default.Table_8b908c7d655a467bbda1f9d2fdc1b99a [id=f795c514f7144933b455d56dbd038cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.627409 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a57b040ab44812a2001705493f2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.628238 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 33569b8849444137bb5469995bee24c4 (DEFAULT_TABLE table=default.Table_9fc9077de3204866a090b8d53ea5bd1d [id=a147807e34814e75a700c83b4ebeab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.628683 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33569b8849444137bb5469995bee24c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.631644 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 (DEFAULT_TABLE table=default.Table_bd92de7863b94ff38b2c402c27ddf26c [id=2a5e3486b8ce482ca74dc96e4dc13862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.632112 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a782b8470b4d14ac89b3b7d463a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.633491 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985dac332bff463781ae93c36f45fdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.638779 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 476da7e2681540a89824b58fb7237df9 (DEFAULT_TABLE table=default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.639318 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476da7e2681540a89824b58fb7237df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.639287 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5b0e7b307dc84d88aff7e744fe4ed97b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.640709 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b0e7b307dc84d88aff7e744fe4ed97b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.644479 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 consensus state has no leader
I20260811 18:57:29.648159 4162 tablet_bootstrap.cc:700] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent opening tablet: real 0.204s user 0.001s sys 0.000s
I20260811 18:57:29.648626 4162 tablet_bootstrap.cc:654] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:29.650923 3678 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.208s (SENT)
W20260811 18:57:29.647910 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 273375us
I20260811 18:57:29.652019 4169 log.cc:1196] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:29.652478 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]) successfully deleted
I20260811 18:57:29.653558 4169 ts_tablet_manager.cc:1939] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.654115 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 544) took 6303 ms. Trace:
I20260811 18:57:29.648422 4163 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:29.654536 4163 ts_tablet_manager.cc:1397] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.381s user 0.003s sys 0.011s
I20260811 18:57:29.654553 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 1ddb10e5ee5b441c875dd0b345201949 (DEFAULT_TABLE table=default.Table_6064366e4ea94146903cb48b739102e9 [id=94bd67a82648467ab2ffb80d4e1b9ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.655700 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5638cefdec734ed58f3de696740ba891 (DEFAULT_TABLE table=default.Table_832e411b94e24eb49b8798f66fa24774 [id=8b596fd1e4af496680f105839144bf79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.656060 4163 raft_consensus.cc:348] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.656436 4163 raft_consensus.cc:374] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.656549 4163 raft_consensus.cc:729] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.656912 4163 consensus_queue.cc:260] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.657253 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 72f11748cf984942ba99bda4ad6605ab (DEFAULT_TABLE table=default.Table_c6284ad4471b4b7095b11fe7a8f304e2 [id=53f3832432d242058789ef564beeae06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.656951 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 08f4d8bbe23b463680c153ae4ff4ebcf (DEFAULT_TABLE table=default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a [id=7de6fc7b04824669845b02a164b3a819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.657156 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 468c71415a984ea7ab04c21fa9612070 (DEFAULT_TABLE table=default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 [id=4a0a8c18d3114ce4a1c4e69e733d372b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.659138 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 23c85b2315254e4c8d9d69fec4cc9aa2 (DEFAULT_TABLE table=default.Table_e79b6e054e914fd39ca0a6282eb0ce17 [id=8d92dfcc02424160a65fca84e2417a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.659816 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 4149626f48354ba0923cad28b95be048 (DEFAULT_TABLE table=default.Table_5296710afead477eb67f356c9afbc4a4 [id=54cc236881e246a18f2af7ae6df6b983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.660360 3866 tablet_service.cc:1467] Processing CreateTablet for tablet ce3acbad48bb431fad5b6904dec8e5ae (DEFAULT_TABLE table=default.Table_9743ba51a07b4fca8aaf1a188a692aac [id=4e7d8b7d88074d04b97ad14a02626671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.660696 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.660923 5091 raft_consensus.cc:482] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.661541 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.662652 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.670969 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ddb10e5ee5b441c875dd0b345201949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.671283 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4149626f48354ba0923cad28b95be048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.681013 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.661082 5091 raft_consensus.cc:504] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.687891 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880ad27a3eb4599b407edeb2969a08f with delete_type TABLET_DATA_DELETED (replaced by tablet 101994cc708a4fadb46fc8c5915a5587) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.699586 5091 leader_election.cc:288] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.700282 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43ad6993b63e4bbd98c8cc52f1a53451" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.700955 3676 leader_election.cc:341] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:29.696720 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 45808.9us
W20260811 18:57:29.701537 3678 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.139s (SENT)
I20260811 18:57:29.706465 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:29.706760 3743 raft_consensus.cc:2454] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:29.711547 4163 ts_tablet_manager.cc:1428] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.057s user 0.002s sys 0.001s
W20260811 18:57:29.711714 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 20007ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:29.714231 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 73e15954a1604189af634a727cb48690 (DEFAULT_TABLE table=default.Table_3de36d9c6d4542be9ee2c873f0bfcfca [id=d446e82dcf024d6e972911e406ca588a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.714675 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3652d29236a4b12b7abd44659c70311" } new_table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" modify_external_catalogs: true
I20260811 18:57:29.718755 4169 rpcz_store.cc:276] 0811 18:57:23.350625 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.360208 (+ 9583us) service_pool.cc:227] Handling call
0811 18:57:29.433962 (+6073754us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:29.433976 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.471748 (+ 37772us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.564008 (+ 92260us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.654093 (+ 90085us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6054256}
I20260811 18:57:29.720160 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 554) took 5977 ms. Trace:
I20260811 18:57:29.720482 4169 rpcz_store.cc:276] 0811 18:57:23.742800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.742855 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.719773 (+5976918us) ts_tablet_manager.cc:1145] Deleting tablet d593aea288e14bb1a4c54360b260cffa
0811 18:57:29.719823 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.720098 (+ 275us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5966264}
I20260811 18:57:29.729497 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a72e17b47b774a0fae13895039a39086"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.730486 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a72e17b47b774a0fae13895039a39086 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.725404 4163 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.727046 5080 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.735731 5080 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.736683 5080 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:29.745965 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b" } modify_external_catalogs: true
I20260811 18:57:29.746459 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b" } modify_external_catalogs: true
I20260811 18:57:29.747372 5080 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.747555 5080 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.748396 5080 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:29.751276 3678 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.001s (SENT)
I20260811 18:57:29.755898 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 555) took 6009 ms. Trace:
I20260811 18:57:29.761965 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3fbd6111333f4635a274f38a20902738"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.762864 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fbd6111333f4635a274f38a20902738 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.762734 5091 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.764148 5091 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.765005 5091 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.766825 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 0495c5b770b34d54afb1379a6e36204c (DEFAULT_TABLE table=default.Table_cafc7915ae7e4280a4d71e3bce4ca57c [id=bcffeabc4e924226a6c508ea5540bb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.765877 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.771065 3676 leader_election.cc:341] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.758487 4167 ts_tablet_manager.cc:1918] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.782910 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be655558b83041fcace57520a599b8a0" } modify_external_catalogs: true
I20260811 18:57:29.783416 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be655558b83041fcace57520a599b8a0" } modify_external_catalogs: true
I20260811 18:57:29.761108 4169 rpcz_store.cc:276] 0811 18:57:23.745355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.747573 (+ 2218us) service_pool.cc:227] Handling call
0811 18:57:29.741921 (+5994348us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:29.742475 (+ 554us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.754332 (+ 11857us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5986722}
I20260811 18:57:29.796736 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61b2dc398e804cdabeda17315814c849" } modify_external_catalogs: true
I20260811 18:57:29.797062 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61b2dc398e804cdabeda17315814c849" } modify_external_catalogs: true
I20260811 18:57:29.793112 4167 log.cc:1196] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:29.814182 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" } modify_external_catalogs: true
I20260811 18:57:29.814507 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" } modify_external_catalogs: true
I20260811 18:57:29.817816 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 90fde17971ea49aaad212a5836661b08 (DEFAULT_TABLE table=default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 [id=e354251945a14c9fb3846267e17bcb74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.812882 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 556) took 6061 ms. Trace:
I20260811 18:57:29.822553 4169 rpcz_store.cc:276] 0811 18:57:23.748658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.754405 (+ 5747us) service_pool.cc:227] Handling call
0811 18:57:29.795216 (+6040811us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:29.801942 (+ 6726us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.809967 (+ 8025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6031578}
I20260811 18:57:29.832041 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" } modify_external_catalogs: true
I20260811 18:57:29.832350 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" } modify_external_catalogs: true
I20260811 18:57:29.835212 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 553) took 6093 ms. Trace:
I20260811 18:57:29.840025 5091 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.840075 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c395f6c4c364b7eb463f78e16675847" } new_table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" modify_external_catalogs: true
I20260811 18:57:29.840236 5091 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.841109 5091 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.842191 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.855809 3676 leader_election.cc:341] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.855434 4169 rpcz_store.cc:276] 0811 18:57:23.742061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.742183 (+ 122us) service_pool.cc:227] Handling call
0811 18:57:29.835006 (+6092823us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:29.835028 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.835187 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6071613}
I20260811 18:57:29.860316 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 557) took 6095 ms. Trace:
W20260811 18:57:29.860913 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ea579c33f8b14e9fa7fa389ce7f45931 (table default.Table_498524390fc04da5b952cdc86fe5decd [id=77b12e379e234e6f8285ebe4c3c46aba], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:29.867239 4162 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.867563 4162 ts_tablet_manager.cc:1397] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.495s user 0.002s sys 0.007s
I20260811 18:57:29.869386 4162 raft_consensus.cc:348] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.870061 4162 raft_consensus.cc:374] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.870267 4162 raft_consensus.cc:729] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:29.878826 3678 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.147s (SENT)
I20260811 18:57:29.878965 3678 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:29.879189 3678 raft_consensus.cc:2670] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:29.860494 4169 rpcz_store.cc:276] 0811 18:57:23.764614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.765352 (+ 738us) service_pool.cc:227] Handling call
0811 18:57:29.860046 (+6094694us) ts_tablet_manager.cc:1145] Deleting tablet 1f60a286ac024c45bcb845227c5445c9
0811 18:57:29.860068 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.860283 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6094231}
I20260811 18:57:29.870756 4162 consensus_queue.cc:260] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.891386 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb3e997b4454598bdce102e92296887 (DEFAULT_TABLE table=default.Table_f9cadd28493e444185170bcd4c314208 [id=d0d78361eb3d4f0e8a6fa132a05a34e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.889430 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 558) took 6124 ms. Trace:
I20260811 18:57:29.892469 4169 rpcz_store.cc:276] 0811 18:57:23.764904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.779759 (+ 14855us) service_pool.cc:227] Handling call
0811 18:57:29.885381 (+6105622us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:29.885831 (+ 450us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.889350 (+ 3519us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6087016}
I20260811 18:57:29.928757 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2ee5acbaa9f54d4ba523b603641e4a97 (DEFAULT_TABLE table=default.Table_9914a933914f4377ba70e0b57c70edff [id=6d6cbea588c54de4a0966727cc2efaf6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:29.932799 3678 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.874s (SENT)
I20260811 18:57:29.935135 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3c807c78cf1249abbf878e819573553d" } new_table_name: "default.Table_c5b1c8014bf048a09ea36fcf6713e7ba" modify_external_catalogs: true
I20260811 18:57:29.964570 5091 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.964888 5091 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.965847 5091 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:29.966332 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 49488us
I20260811 18:57:29.919539 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 570) took 5960 ms. Trace:
I20260811 18:57:29.967588 4167 ts_tablet_manager.cc:1939] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.988152 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 653) took 3937 ms. Trace:
I20260811 18:57:29.988610 4167 rpcz_store.cc:276] 0811 18:57:26.050908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.050973 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.469372 (+3418399us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:29.469389 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.526678 (+ 57289us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.715691 (+189013us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.988081 (+272390us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3418094}
I20260811 18:57:29.989939 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 799ba9a044e145ec89d0025c03f391ea (table default.Table_56b89b2b2c4b45768b2866c74d9f1390 [id=7736472828a3499c8ad8026e164b9759]) successfully deleted
I20260811 18:57:29.991480 4162 ts_tablet_manager.cc:1428] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.124s user 0.003s sys 0.004s
I20260811 18:57:29.997716 4162 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:30.008242 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 985dac332bff463781ae93c36f45fdda (DEFAULT_TABLE table=default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a [id=cc40efcc988e48c7b37a10f5f1217956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.010785 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c8d6cbc9a8ed4b798188572c6fc1ff52 (DEFAULT_TABLE table=default.Table_5b05303fcd96430bb6ab896fbaa5097a [id=e81a772f5829416f974f5f6a19cc09b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.012347 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 (DEFAULT_TABLE table=default.Table_0c3df98432314ff2b4cbd7004701b258 [id=6e4e47c552c943aca96b05e0959d681d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.012535 3861 tablet_service.cc:1467] Processing CreateTablet for tablet d32212c817c94fad8379462b027e7d3d (DEFAULT_TABLE table=default.Table_f0a164081b1349d9bc89f20d1ea6425d [id=e76087cfeb3c4a338de2a131620bc473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.016387 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 385fef74b0c1421ea86c23bfee38c681 (DEFAULT_TABLE table=default.Table_ef4494689f0147118d362ed74d1bf78c [id=bd0e71434abf458795a3e2f8b6251847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.017846 5091 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.018061 5091 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:30.019086 5091 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:30.025679 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.026410 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet f3670f8908d54ebb82f698cdd207e48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.027010 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a (DEFAULT_TABLE table=default.Table_378dd90b6af541b8ab95d9f19681a03f [id=d43a5bc491c048718e2504b3d22d2890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.027531 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.026270 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8319453f68a54da9813747152ef08756 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.029109 4169 rpcz_store.cc:276] 0811 18:57:23.946951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.949599 (+ 2648us) service_pool.cc:227] Handling call
0811 18:57:29.907215 (+5957616us) ts_tablet_manager.cc:1145] Deleting tablet 56d24aff9fd34f3eb05fdf942cc7efc7
0811 18:57:29.907236 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.907390 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5931437}
I20260811 18:57:30.033056 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 575) took 5457 ms. Trace:
I20260811 18:57:30.034410 4169 rpcz_store.cc:276] 0811 18:57:24.575897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.575958 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:30.031888 (+5455930us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:30.032037 (+ 149us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.032906 (+ 869us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5447097}
W20260811 18:57:30.039167 3676 leader_election.cc:341] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:30.040539 3676 leader_election.cc:341] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:30.070916 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.103582 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 576) took 5519 ms. Trace:
I20260811 18:57:30.112121 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 658) took 4023 ms. Trace:
W20260811 18:57:30.116474 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 133579us
W20260811 18:57:30.121603 3678 leader_election.cc:334] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.192s (SENT)
W20260811 18:57:30.125445 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cdb92902c2584751af77d3117f52cb4e consensus state has no leader
I20260811 18:57:30.104871 4169 rpcz_store.cc:276] 0811 18:57:24.583861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.584016 (+ 155us) service_pool.cc:227] Handling call
0811 18:57:30.037192 (+5453176us) ts_tablet_manager.cc:1145] Deleting tablet 14e34732bda44670ae9da8facc405221
0811 18:57:30.037356 (+ 164us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.103091 (+ 65735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5447722}
I20260811 18:57:30.134223 4167 rpcz_store.cc:276] 0811 18:57:26.088903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.088958 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.989726 (+3900768us) ts_tablet_manager.cc:1145] Deleting tablet ea4ad482bde44d6bab5ec809054aaebd
0811 18:57:29.989746 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.111975 (+122229us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3900507}
I20260811 18:57:30.134999 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 671) took 3946 ms. Trace:
W20260811 18:57:30.135236 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ea4ad482bde44d6bab5ec809054aaebd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea4ad482bde44d6bab5ec809054aaebd already in progress: creating tablet
I20260811 18:57:30.135286 4167 rpcz_store.cc:276] 0811 18:57:26.188789 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.188845 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:30.134791 (+3945946us) ts_tablet_manager.cc:1145] Deleting tablet b16cb4c1796147fbb10109f52071505e
0811 18:57:30.134812 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.134970 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3945680}
I20260811 18:57:30.135972 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 670) took 3947 ms. Trace:
W20260811 18:57:30.136068 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b16cb4c1796147fbb10109f52071505e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16cb4c1796147fbb10109f52071505e already in progress: creating tablet
I20260811 18:57:30.136116 4167 rpcz_store.cc:276] 0811 18:57:26.188663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.188715 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:30.135803 (+3947088us) ts_tablet_manager.cc:1145] Deleting tablet 54fa32f8fa1144c4bd9dcd00a550001a
0811 18:57:30.135819 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.135946 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3946303}
I20260811 18:57:30.136723 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 669) took 3948 ms. Trace:
W20260811 18:57:30.136868 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 54fa32f8fa1144c4bd9dcd00a550001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54fa32f8fa1144c4bd9dcd00a550001a already in progress: creating tablet
I20260811 18:57:30.136830 4167 rpcz_store.cc:276] 0811 18:57:26.188460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.217735 (+ 29275us) service_pool.cc:227] Handling call
0811 18:57:30.136592 (+3918857us) ts_tablet_manager.cc:1145] Deleting tablet 1d81e93739f44d9e99b6ebc647e1d0e4
0811 18:57:30.136604 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.136702 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3918472}
I20260811 18:57:30.137244 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 672) took 3875 ms. Trace:
I20260811 18:57:30.137343 4167 rpcz_store.cc:276] 0811 18:57:26.261412 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.261485 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:30.137089 (+3875604us) ts_tablet_manager.cc:1145] Deleting tablet 1811e5f62a714f8388460dfcb1717e81
0811 18:57:30.137102 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.137222 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3875132}
W20260811 18:57:30.137439 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d81e93739f44d9e99b6ebc647e1d0e4 already in progress: creating tablet
I20260811 18:57:30.137651 4167 tablet_replica.cc:330] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:30.138051 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1811e5f62a714f8388460dfcb1717e81 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1811e5f62a714f8388460dfcb1717e81 already in progress: creating tablet
W20260811 18:57:30.138945 3678 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.060s (SENT)
I20260811 18:57:30.155288 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 98ad733726594d2a8087f857f155dbe2
I20260811 18:57:30.157716 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 85140a3800f34a23b716703fd4e73d17
W20260811 18:57:30.161248 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38a4783168554053aa953dba5e893b6b consensus state has no leader
I20260811 18:57:30.161727 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ad733726594d2a8087f857f155dbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.164031 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 85140a3800f34a23b716703fd4e73d17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.165536 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_827736b8218e48e086bf05354914d450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.166308 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_827736b8218e48e086bf05354914d450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.167706 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de8a2d61a3cc411a97ad8e8df11c7d89" } new_table_name: "default.Table_972a064a812a441bb4068503de32341e" modify_external_catalogs: true
I20260811 18:57:30.168324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1cdea1db192a43f4b210579b0813e33a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.168776 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 577) took 5583 ms. Trace:
I20260811 18:57:30.168900 4169 rpcz_store.cc:276] 0811 18:57:24.585283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.585377 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:30.168505 (+5583128us) ts_tablet_manager.cc:1145] Deleting tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
0811 18:57:30.168545 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.168710 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5538336}
I20260811 18:57:30.169421 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 578) took 5572 ms. Trace:
I20260811 18:57:30.169507 4169 rpcz_store.cc:276] 0811 18:57:24.596730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.597567 (+ 837us) service_pool.cc:227] Handling call
0811 18:57:30.169288 (+5571721us) ts_tablet_manager.cc:1145] Deleting tablet 64b922f2a71544e8a18533aa0a1ccb52
0811 18:57:30.169303 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.169404 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5564294}
I20260811 18:57:30.169785 4169 tablet_replica.cc:330] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:30.170014 4169 raft_consensus.cc:2229] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.170243 4169 raft_consensus.cc:2258] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.171022 4169 ts_tablet_manager.cc:1905] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:30.171741 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cdea1db192a43f4b210579b0813e33a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.174623 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93b2871d4c454120a6b609ae91e096e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.175379 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2a92f77f0074806a1b56a31c303dd69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.178017 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2076434270fa4c9c99b4d12f1aa924c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.179960 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_80a7a76a26ab4fc9919bfef0a312f217"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.182968 5286 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.183401 5286 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.184445 5286 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:30.186088 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2a92f77f0074806a1b56a31c303dd69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.190790 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93b2871d4c454120a6b609ae91e096e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.193056 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2076434270fa4c9c99b4d12f1aa924c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.194139 3678 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT)
I20260811 18:57:30.194507 3678 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:30.194983 5091 raft_consensus.cc:2726] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:30.195044 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80a7a76a26ab4fc9919bfef0a312f217 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.196615 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 460e03cc14444c83b6c08ca194ca5a8d (table default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]) successfully deleted
I20260811 18:57:30.211531 4162 tablet_bootstrap.cc:654] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.215657 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d593aea288e14bb1a4c54360b260cffa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d593aea288e14bb1a4c54360b260cffa already in progress: creating tablet
W20260811 18:57:30.216209 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe203387bc9c45ba82afe6b058aa5e5f already in progress: creating tablet
W20260811 18:57:30.216761 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5d495354c0994fb4a74c376078e41a26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d495354c0994fb4a74c376078e41a26 already in progress: creating tablet
W20260811 18:57:30.217067 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b3f0cdcee681460084662e01152ad9dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3f0cdcee681460084662e01152ad9dd already in progress: creating tablet
W20260811 18:57:30.217363 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1f60a286ac024c45bcb845227c5445c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f60a286ac024c45bcb845227c5445c9 already in progress: creating tablet
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
W20260811 18:57:30.238575 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 122b7898fea44ef78f269b1505d8ddc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 122b7898fea44ef78f269b1505d8ddc4 already in progress: creating tablet
W20260811 18:57:30.254560 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7057b8a7bae249caaed304b5c66e6871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7057b8a7bae249caaed304b5c66e6871 already in progress: creating tablet
W20260811 18:57:30.254866 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 14e34732bda44670ae9da8facc405221 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e34732bda44670ae9da8facc405221 already in progress: creating tablet
I20260811 18:57:30.256049 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3acbad48bb431fad5b6904dec8e5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.256312 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.258119 4163 tablet_bootstrap.cc:654] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:30.258389 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d6cbc9a8ed4b798188572c6fc1ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.259349 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c85b2315254e4c8d9d69fec4cc9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.262478 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9b279c6c7344f1931b81e10248b1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.262744 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8319453f68a54da9813747152ef08756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.263476 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0495c5b770b34d54afb1379a6e36204c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.265573 3850 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 143) took 1038 ms. Trace:
I20260811 18:57:30.265712 3850 rpcz_store.cc:276] 0811 18:57:29.227368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.227435 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:29.348708 (+121273us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x61700010570c. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d15aded6e 00007f4d158e815e 00007f4d181e2a88 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378
0811 18:57:30.265506 (+916798us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":124492544}
I20260811 18:57:30.265862 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385fef74b0c1421ea86c23bfee38c681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.267484 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32212c817c94fad8379462b027e7d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.272993 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985dac332bff463781ae93c36f45fdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.273303 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.273507 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.277541 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb3e997b4454598bdce102e92296887. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.282092 3678 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.005s (SENT)
I20260811 18:57:30.283160 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.283494 3745 raft_consensus.cc:2454] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.286158 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.286599 3743 raft_consensus.cc:2454] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.292387 4167 raft_consensus.cc:2229] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.279855 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 86a57b040ab44812a2001705493f2bcd (DEFAULT_TABLE table=default.Table_8b908c7d655a467bbda1f9d2fdc1b99a [id=f795c514f7144933b455d56dbd038cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.280558 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 (DEFAULT_TABLE table=default.Table_bd92de7863b94ff38b2c402c27ddf26c [id=2a5e3486b8ce482ca74dc96e4dc13862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.280962 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 476da7e2681540a89824b58fb7237df9 (DEFAULT_TABLE table=default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.294387 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a57b040ab44812a2001705493f2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.294649 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ee5acbaa9f54d4ba523b603641e4a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.287230 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 33569b8849444137bb5469995bee24c4 (DEFAULT_TABLE table=default.Table_9fc9077de3204866a090b8d53ea5bd1d [id=a147807e34814e75a700c83b4ebeab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.295094 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e15954a1604189af634a727cb48690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.288834 4163 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.298549 4163 ts_tablet_manager.cc:1397] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.573s user 0.010s sys 0.019s
I20260811 18:57:30.300107 4167 raft_consensus.cc:2258] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.300426 4163 raft_consensus.cc:348] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.300762 4163 raft_consensus.cc:374] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.300884 4163 raft_consensus.cc:729] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.301173 4163 consensus_queue.cc:260] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.301864 4163 ts_tablet_manager.cc:1428] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:30.303845 4163 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:30.305035 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468c71415a984ea7ab04c21fa9612070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.306203 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f4d8bbe23b463680c153ae4ff4ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.310520 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a782b8470b4d14ac89b3b7d463a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.313548 4163 tablet_bootstrap.cc:654] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.314297 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:30.315526 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 7d747f03ad234805a4ea4ef60840d426 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:30.316820 4167 ts_tablet_manager.cc:1905] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.333595 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ebca8dd587b144d7909761c052084855"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.334878 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebca8dd587b144d7909761c052084855 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.337563 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.338543 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f13efa6fe524f5bbd7154aa2f991d38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
I20260811 18:57:30.340458 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e9433d18ac7b46958942fc1f4b458354" } new_table_name: "default.Table_772fdfa6598e4b2ca2aa3a4b28746b24" modify_external_catalogs: true
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
I20260811 18:57:30.359477 4162 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:30.359825 4162 ts_tablet_manager.cc:1397] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.362s user 0.005s sys 0.003s
I20260811 18:57:30.361155 4162 raft_consensus.cc:348] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
#3 0x7fc7b9db98c6 in kudu::rpc::ServicePool::Init(int) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
I20260811 18:57:30.313784 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f11748cf984942ba99bda4ad6605ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.369020 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476da7e2681540a89824b58fb7237df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.369230 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5638cefdec734ed58f3de696740ba891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.370510 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33569b8849444137bb5469995bee24c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.371112 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:30.375202 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.376227 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd5ee099cbc94fd285f1b71eea0cde80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.378481 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fde17971ea49aaad212a5836661b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.380700 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_41c44c966ece466eb232aeab834c5fe6" } new_table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" modify_external_catalogs: true
I20260811 18:57:30.384260 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aaf507832a664e6ebcdafffc433bbac7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.385291 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aaf507832a664e6ebcdafffc433bbac7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.391484 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8" } modify_external_catalogs: true
I20260811 18:57:30.391758 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8" } modify_external_catalogs: true
I20260811 18:57:30.392405 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9de9363efd764bd4ac323f86fd70658d" } new_table_name: "default.Table_4a429d101b1e46efaafa42a86e2f64f7" modify_external_catalogs: true
I20260811 18:57:30.394177 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d5ebf213ce401984be01bac1813640" } modify_external_catalogs: true
I20260811 18:57:30.394472 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d5ebf213ce401984be01bac1813640" } modify_external_catalogs: true
W20260811 18:57:30.323444 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68766.1us
I20260811 18:57:30.331493 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:30.398145 3824 consensus_peers.cc:487] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6f36ab6bb774257b5c2692c64eff63d. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:30.398491 3824 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.398882 5287 raft_consensus.cc:2781] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.398996 5287 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.399081 5287 raft_consensus.cc:3037] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.400111 5286 consensus_queue.cc:1035] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.400969 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59f9d96102f349498c7f854e48e3d7c5" } new_table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" modify_external_catalogs: true
I20260811 18:57:30.412134 3824 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.412569 5286 raft_consensus.cc:2781] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.412741 5286 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.412901 5286 raft_consensus.cc:3037] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.423269 5080 consensus_queue.cc:1035] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.423274 3824 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.427526 5080 raft_consensus.cc:2781] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.427644 5080 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.427726 5080 raft_consensus.cc:3037] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.361436 4162 raft_consensus.cc:374] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.427966 4162 raft_consensus.cc:729] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:30.428658 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 already in progress: creating tablet
W20260811 18:57:30.429105 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 64b922f2a71544e8a18533aa0a1ccb52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64b922f2a71544e8a18533aa0a1ccb52 already in progress: creating tablet
I20260811 18:57:30.430090 4162 consensus_queue.cc:260] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:30.439858 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ca801c79fc499ab50cfe76cdb03b00 consensus state has no leader
W20260811 18:57:30.440755 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 101994cc708a4fadb46fc8c5915a5587 has no consensus state
W20260811 18:57:30.461721 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 05cba6b392b642e59321570aac3541d0 consensus state has no leader
I20260811 18:57:30.447237 5313 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:30.449030 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.462291 3743 raft_consensus.cc:2454] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.451498 3740 raft_consensus.cc:1261] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:30.468328 5287 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.469534 5287 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.473742 5286 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.475044 5286 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.480683 4162 ts_tablet_manager.cc:1428] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.121s user 0.000s sys 0.002s
I20260811 18:57:30.487015 4163 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.493283 4162 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:30.494767 4163 ts_tablet_manager.cc:1397] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.001s
I20260811 18:57:30.501034 4163 raft_consensus.cc:348] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.513754 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_949ccbca1e8b44b79578377f67f2422c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.515077 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_949ccbca1e8b44b79578377f67f2422c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.516530 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7a8c696f8df4bec92e19adef1eff6d2" } new_table_name: "default.Table_495f858f5eec496680f982bccd44415d" modify_external_catalogs: true
I20260811 18:57:30.518797 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6204a77bcccf427db16e32981a5621fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.519569 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6204a77bcccf427db16e32981a5621fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.520659 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d584e1cc7d8b45be912d8610e391f4f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.521369 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d584e1cc7d8b45be912d8610e391f4f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.530563 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.530802 3740 raft_consensus.cc:3037] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.531605 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.531777 3743 raft_consensus.cc:3037] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:30.532172 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 531ba05901734d6b93bc2be506e8ec5b has no consensus state
I20260811 18:57:30.534096 4863 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:34821
I20260811 18:57:30.534230 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aadfbb276f80467d884601ea6c92a401"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.534749 4162 tablet_bootstrap.cc:654] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.535024 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aadfbb276f80467d884601ea6c92a401 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.537289 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_634c196a72864956aedd6ceb1757173a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.538069 4863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
W20260811 18:57:30.538472 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_634c196a72864956aedd6ceb1757173a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.538527 5326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:34821 every 8 connection(s)
I20260811 18:57:30.541786 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 4863
I20260811 18:57:30.545457 3743 raft_consensus.cc:2454] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.547147 3740 raft_consensus.cc:2454] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.504824 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 85140a3800f34a23b716703fd4e73d17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.505030 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.552420 4163 raft_consensus.cc:374] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.552731 4163 raft_consensus.cc:729] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.552628 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_94caf53b154f4e39b9ea7bf02dcb11c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.553092 4163 consensus_queue.cc:260] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:30.553634 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.505541 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ad733726594d2a8087f857f155dbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.520499 5323 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:30.568702 5080 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.569618 5080 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:30.571234 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 83952.7us
I20260811 18:57:30.528115 4169 ts_tablet_manager.cc:1918] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:30.591851 4169 log.cc:1196] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:30.592345 4169 ts_tablet_manager.cc:1939] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:30.593183 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 579) took 5788 ms. Trace:
I20260811 18:57:30.593290 4169 rpcz_store.cc:276] 0811 18:57:24.804985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.805042 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:30.169734 (+5364692us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:30.169746 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.409096 (+239350us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.527967 (+118871us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.593164 (+ 65197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5364239}
I20260811 18:57:30.594077 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 584) took 5762 ms. Trace:
I20260811 18:57:30.594321 4169 rpcz_store.cc:276] 0811 18:57:24.831132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.831186 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:30.593911 (+5762725us) ts_tablet_manager.cc:1145] Deleting tablet 0b40da8cb7324da9804e6fdafd8e8266
0811 18:57:30.593926 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.594056 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5762369}
I20260811 18:57:30.594938 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 587) took 5762 ms. Trace:
I20260811 18:57:30.595118 4169 rpcz_store.cc:276] 0811 18:57:24.832436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.832487 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:30.594792 (+5762305us) ts_tablet_manager.cc:1145] Deleting tablet 3b40e44a2f4940af891eddf6c3b6293c
0811 18:57:30.594807 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.594909 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5761883}
W20260811 18:57:30.529500 3824 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.333s (SENT)
I20260811 18:57:30.595451 3824 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:30.595676 3824 raft_consensus.cc:2670] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:30.596496 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 67025.5us
W20260811 18:57:30.596796 3824 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
W20260811 18:57:30.597201 3824 consensus_peers.cc:487] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 799ba9a044e145ec89d0025c03f391ea. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:30.597849 3824 consensus_peers.cc:487] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:30.598071 3824 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.598565 5080 raft_consensus.cc:2781] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.598812 5080 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.598901 5080 raft_consensus.cc:3037] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.601619 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24c4140a00244c8d8446d91705a8d4a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.602475 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24c4140a00244c8d8446d91705a8d4a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.602860 5080 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.604123 5080 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.606833 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3a0329aed6694593bc21571dd8f59670"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.607847 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a0329aed6694593bc21571dd8f59670 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.610673 4163 ts_tablet_manager.cc:1428] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.112s user 0.003s sys 0.000s
I20260811 18:57:30.636139 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c5b1c8014bf048a09ea36fcf6713e7ba [id=1f86b4e3748c4cb7b34b2a24524dd1ed] alter complete (version 2)
I20260811 18:57:30.636861 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7ed0b81c09744e7af02ac80fb40508c" } new_table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" modify_external_catalogs: true
W20260811 18:57:30.642915 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 44857.7us
I20260811 18:57:30.643867 3824 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.644215 3824 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.644948 5286 raft_consensus.cc:2781] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.644891 5080 consensus_queue.cc:1035] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.645459 5283 raft_consensus.cc:2781] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.646100 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4133a1bbf6474492bbf3683ca9608a52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.647014 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4133a1bbf6474492bbf3683ca9608a52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.648999 5091 consensus_queue.cc:1035] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.654376 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 585) took 5822 ms. Trace:
I20260811 18:57:30.655505 4167 ts_tablet_manager.cc:1918] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:30.655664 4167 log.cc:1196] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:30.656047 4167 ts_tablet_manager.cc:1939] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:30.654503 4169 rpcz_store.cc:276] 0811 18:57:24.831808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.831874 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:30.654202 (+5822328us) ts_tablet_manager.cc:1145] Deleting tablet 19d7716739f848d78b9cbbd3975f3d44
0811 18:57:30.654215 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.654328 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5813280}
I20260811 18:57:30.663329 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 586) took 5831 ms. Trace:
I20260811 18:57:30.663465 4169 rpcz_store.cc:276] 0811 18:57:24.832203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.832261 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:30.663174 (+5830913us) ts_tablet_manager.cc:1145] Deleting tablet 2d0b59cafb154416952aecaa418e861f
0811 18:57:30.663190 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.663306 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5821627}
I20260811 18:57:30.665560 5286 raft_consensus.cc:686] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.665927 5286 consensus_queue.cc:237] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.674015 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f61d3822fc2d4b569f223238a6464e0a (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
W20260811 18:57:30.674415 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0b40da8cb7324da9804e6fdafd8e8266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b40da8cb7324da9804e6fdafd8e8266 already in progress: creating tablet
I20260811 18:57:30.674428 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.674824 3745 raft_consensus.cc:3037] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.676694 5283 raft_consensus.cc:686] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.677111 5283 consensus_queue.cc:237] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.678210 3745 raft_consensus.cc:2454] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.681149 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 588) took 5848 ms. Trace:
I20260811 18:57:30.681283 4169 rpcz_store.cc:276] 0811 18:57:24.833160 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833213 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:30.680987 (+5847774us) ts_tablet_manager.cc:1145] Deleting tablet 92e49846ab714dd294acdd09d49a83bd
0811 18:57:30.681002 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.681122 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5838817}
I20260811 18:57:30.681802 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 589) took 5848 ms. Trace:
I20260811 18:57:30.681888 4169 rpcz_store.cc:276] 0811 18:57:24.833609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833678 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:30.681678 (+5848000us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:30.681689 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.681784 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5839090}
I20260811 18:57:30.682305 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 619) took 5255 ms. Trace:
I20260811 18:57:30.682411 4169 rpcz_store.cc:276] 0811 18:57:25.426729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.427036 (+ 307us) service_pool.cc:227] Handling call
0811 18:57:30.682205 (+5255169us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:30.682215 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.682288 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5254928}
I20260811 18:57:30.682845 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 620) took 5256 ms. Trace:
I20260811 18:57:30.683250 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 673) took 4383 ms. Trace:
I20260811 18:57:30.683352 4167 rpcz_store.cc:276] 0811 18:57:26.299963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.300031 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:30.137606 (+3837575us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:30.137615 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.501865 (+364250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.551693 (+ 49828us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.683222 (+131529us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31396,"tablet-delete.queue_time_us":3812313}
I20260811 18:57:30.683902 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 674) took 4279 ms. Trace:
I20260811 18:57:30.684048 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ad424c8843ff434ba0449a86849bb5a0 (table default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]) successfully deleted
I20260811 18:57:30.684000 4167 rpcz_store.cc:276] 0811 18:57:26.404865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.404948 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:30.683772 (+4278824us) ts_tablet_manager.cc:1145] Deleting tablet 335fc15ea75d483990e6ae5a5872682f
0811 18:57:30.683787 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.683882 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4278527}
I20260811 18:57:30.684496 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 675) took 4228 ms. Trace:
I20260811 18:57:30.684577 4167 rpcz_store.cc:276] 0811 18:57:26.456387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.456459 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:30.684383 (+4227924us) ts_tablet_manager.cc:1145] Deleting tablet 705605cd929940f8a252b6e95abbeb95
0811 18:57:30.684395 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.684478 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4227656}
W20260811 18:57:30.684686 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 335fc15ea75d483990e6ae5a5872682f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 335fc15ea75d483990e6ae5a5872682f already in progress: creating tablet
I20260811 18:57:30.685057 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 676) took 4210 ms. Trace:
I20260811 18:57:30.685140 4167 rpcz_store.cc:276] 0811 18:57:26.474187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.474255 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:30.684798 (+4210543us) ts_tablet_manager.cc:1145] Deleting tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
0811 18:57:30.684807 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.685040 (+ 233us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4210261}
W20260811 18:57:30.685272 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 705605cd929940f8a252b6e95abbeb95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 705605cd929940f8a252b6e95abbeb95 already in progress: creating tablet
I20260811 18:57:30.685421 4167 tablet_replica.cc:330] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:30.685874 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a already in progress: creating tablet
I20260811 18:57:30.686985 4162 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:30.691761 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f7c3f328d7e49db85f19e13aa462cd6" } new_table_name: "default.Table_efc46b70936f4bd292f775b723c1fbff" modify_external_catalogs: true
I20260811 18:57:30.691606 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.695197 4162 ts_tablet_manager.cc:1397] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.202s user 0.005s sys 0.002s
I20260811 18:57:30.692539 5091 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.696205 5091 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.696962 5327 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:30.697240 5091 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:30.693215 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.697809 3740 raft_consensus.cc:3037] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.701054 3740 raft_consensus.cc:2454] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.701922 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.702780 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.696171 4162 raft_consensus.cc:348] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.708405 4162 raft_consensus.cc:374] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.708542 4162 raft_consensus.cc:729] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.708822 4162 consensus_queue.cc:260] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.709364 4162 ts_tablet_manager.cc:1428] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20260811 18:57:30.709710 4162 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:30.719036 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3b40e44a2f4940af891eddf6c3b6293c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b40e44a2f4940af891eddf6c3b6293c already in progress: creating tablet
W20260811 18:57:30.719448 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 19d7716739f848d78b9cbbd3975f3d44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d7716739f848d78b9cbbd3975f3d44 already in progress: creating tablet
I20260811 18:57:30.721562 3713 tablet_service.cc:1467] Processing CreateTablet for tablet d21973bc8e514a089c2beb24a686060f (DEFAULT_TABLE table=default.Table_5aa83c98b49c466cb01cdf92154ad384 [id=01ddedd33be14c9ab35e1578361dd733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.722076 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21973bc8e514a089c2beb24a686060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.722831 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b (DEFAULT_TABLE table=default.Table_8fb6e34f004c466381fe6ab5b780568f [id=80d3089bffc748fc93707657e47ba89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.723628 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.725689 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf54e295be64486a90918a8d2eb8b33 (DEFAULT_TABLE table=default.Table_4b38a0019b5541c495088e96174c8de4 [id=05dc3f3a80494c168dcc8e02a62e2bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.726184 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf54e295be64486a90918a8d2eb8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.726639 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 75ceba7bfc8240268b16c789f07813d1 (DEFAULT_TABLE table=default.Table_13786cfeafbc4312bf84f93b204add4f [id=a11b457b0ad74279949798ccab096915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.727077 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ceba7bfc8240268b16c789f07813d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.682928 4169 rpcz_store.cc:276] 0811 18:57:25.426813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.427324 (+ 511us) service_pool.cc:227] Handling call
0811 18:57:30.682748 (+5255424us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:30.682758 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.682829 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5255218}
I20260811 18:57:30.732316 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 618) took 5305 ms. Trace:
I20260811 18:57:30.732424 4169 rpcz_store.cc:276] 0811 18:57:25.426635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.431643 (+ 5008us) service_pool.cc:227] Handling call
0811 18:57:30.732133 (+5300490us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:30.732147 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.732293 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5300275}
I20260811 18:57:30.732748 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 617) took 5320 ms. Trace:
I20260811 18:57:30.732854 4169 rpcz_store.cc:276] 0811 18:57:25.412202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.421317 (+ 9115us) service_pool.cc:227] Handling call
0811 18:57:30.732644 (+5311327us) ts_tablet_manager.cc:1145] Deleting tablet 380c1f53b3ac4509a09b25a2800028af
0811 18:57:30.732653 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.732731 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5275503}
I20260811 18:57:30.701756 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.733219 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 621) took 4980 ms. Trace:
I20260811 18:57:30.733285 4169 rpcz_store.cc:276] 0811 18:57:25.753038 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.753148 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:30.733122 (+4979974us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:30.733132 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.733205 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4979672}
I20260811 18:57:30.733546 4169 tablet_replica.cc:330] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:30.733703 3676 leader_election.cc:341] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:30.688597 4163 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:30.720563 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.737651 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:30.737932 4169 raft_consensus.cc:2229] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.738210 4169 raft_consensus.cc:2258] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.738484 4167 raft_consensus.cc:2229] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.739100 3848 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 146) took 1402 ms. Trace:
W20260811 18:57:30.740372 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2d0b59cafb154416952aecaa418e861f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d0b59cafb154416952aecaa418e861f already in progress: creating tablet
W20260811 18:57:30.740852 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 92e49846ab714dd294acdd09d49a83bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e49846ab714dd294acdd09d49a83bd already in progress: creating tablet
W20260811 18:57:30.741642 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b57ed7814a1e440cba362613cbab32f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b57ed7814a1e440cba362613cbab32f8 already in progress: creating tablet
W20260811 18:57:30.741987 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 68a7051bc539468aafba30fbaf0669d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68a7051bc539468aafba30fbaf0669d4 already in progress: creating tablet
I20260811 18:57:30.742534 4169 ts_tablet_manager.cc:1905] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.739923 3848 rpcz_store.cc:276] 0811 18:57:29.336872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.336966 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:30.475499 (+1138533us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x6150000bb424. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.739023 (+263524us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":150257408}
I20260811 18:57:30.740422 4167 raft_consensus.cc:2258] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.754613 4163 tablet_bootstrap.cc:654] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:30.752346 3824 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.759037 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap starting.
I20260811 18:57:30.759720 3824 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.760021 5286 raft_consensus.cc:2781] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.761066 5286 raft_consensus.cc:686] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.761358 5286 consensus_queue.cc:237] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.762157 5323 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:30.761543 5283 raft_consensus.cc:2781] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.762620 5283 raft_consensus.cc:686] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.762882 5283 consensus_queue.cc:237] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:30.767268 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 45fde25bcb40493aafb083e161aa6932 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45fde25bcb40493aafb083e161aa6932 already in progress: creating tablet
W20260811 18:57:30.767740 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 221d0d8d69f142a29ec7065a82b3f769 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 221d0d8d69f142a29ec7065a82b3f769 already in progress: creating tablet
W20260811 18:57:30.768015 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 380c1f53b3ac4509a09b25a2800028af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 380c1f53b3ac4509a09b25a2800028af already in progress: creating tablet
W20260811 18:57:30.768285 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8304ac7709ca4392baec4cfa78ef5b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8304ac7709ca4392baec4cfa78ef5b41 already in progress: creating tablet
I20260811 18:57:30.771534 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 14799a6f61a946008acee8a09d050c1e (DEFAULT_TABLE table=default.Table_b752cb750e9447b092ecac296b5f836e [id=839eec1f02c342f3925e487ab8277f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.774039 4167 ts_tablet_manager.cc:1905] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.774657 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d21973bc8e514a089c2beb24a686060f (DEFAULT_TABLE table=default.Table_5aa83c98b49c466cb01cdf92154ad384 [id=01ddedd33be14c9ab35e1578361dd733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.775139 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21973bc8e514a089c2beb24a686060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.778621 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec5b2214e0c439ea122d11aa6e6f396 (DEFAULT_TABLE table=default.Table_a677dcb5a43648a49944a18f38fb74fa [id=f3978719f37e4b43847eda64b65f6cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.779191 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec5b2214e0c439ea122d11aa6e6f396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.784471 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14799a6f61a946008acee8a09d050c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.793242 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 03ad1334d0b14ca8adb813e85a429f9e (DEFAULT_TABLE table=default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 [id=76b97a0511a84aa5b2106c833a908799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.798580 3719 tablet_service.cc:1467] Processing CreateTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 (DEFAULT_TABLE table=default.Table_f14f93014a8d464ba0da0bab0ceb9ddc [id=9ea613e25082465f89a34c9da09d3855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.799021 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb1c9787d3f4fd181f86deca28ad160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.802371 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ad1334d0b14ca8adb813e85a429f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.799872 5313 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
#4 0x7fc7bc1faa67 in kudu::RpcServer::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:238:3
I20260811 18:57:30.813551 4169 ts_tablet_manager.cc:1918] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:30.813715 4169 log.cc:1196] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:30.818184 4169 ts_tablet_manager.cc:1939] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:30.819489 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:30.831231 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7300b005e34e4c95de73e8d62faa64 (DEFAULT_TABLE table=default.Table_3982bc659a664e949beb16b472873d9a [id=da2587935be0402cad1db22e34c471b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.832329 3714 tablet_service.cc:1467] Processing CreateTablet for tablet e5f47ccc193d48efa15d60a06b148e00 (DEFAULT_TABLE table=default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a [id=3c2bb15d58c34694b03ffc72080c28f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.833302 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fa5a3d1ada554d778d507382c8836133 (DEFAULT_TABLE table=default.Table_e2635923a5784be1839e861c2277e900 [id=f6576859698f477bae1ad63c86d99eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.833601 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc (DEFAULT_TABLE table=default.Table_b485cd94aa7c4642a0b3d0021cef63c4 [id=256649498ffa421b9ef8837c0c25137e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.834403 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 6f804703e20c4214b8b2a31efeaef59e (DEFAULT_TABLE table=default.Table_dbf1868109474dcea3712c10bb4d4f3f [id=f5435385b235444594731609da68bd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.835493 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 73c519528f75423195b6bdf6683fd91b (DEFAULT_TABLE table=default.Table_7df549360f014d629938e3e94659af8a [id=0915677ac44e4a1dad598b19c54633b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.835436 3720 tablet_service.cc:1467] Processing CreateTablet for tablet ab04517e943e410cb65ae11d4abf5286 (DEFAULT_TABLE table=default.Table_32f181a8b481409187abd0dc0a1765ba [id=f9ceba7031af4bd8a4c270368e018851]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.836794 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1800e9e2b064a6fb546b9d33cefa68b consensus state has no leader
I20260811 18:57:30.839717 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 22abc6f8e9ef4c32ab69af31bdaad872
I20260811 18:57:30.840495 3553 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1022) took 1100 ms. Trace:
I20260811 18:57:30.840548 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b (DEFAULT_TABLE table=default.Table_8fb6e34f004c466381fe6ab5b780568f [id=80d3089bffc748fc93707657e47ba89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.840757 3553 rpcz_store.cc:276] 0811 18:57:29.744151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.745486 (+ 1335us) service_pool.cc:227] Handling call
0811 18:57:29.746741 (+ 1255us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.746987 (+ 246us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.747120 (+ 133us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.695640 (+948520us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.839401 (+143761us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.839483 (+ 82us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.839524 (+ 41us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.840350 (+ 826us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.704173 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.704324 (+ 151us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.704332 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:30.704334 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.704455 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.704470 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.704477 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.704615 (+ 138us) tablet.cc:699] Row locks acquired
0811 18:57:30.704619 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:30.704660 (+ 41us) write_op.cc:270] Start()
0811 18:57:30.704702 (+ 42us) write_op.cc:276] Timestamp: P: 1786474650704609 usec, L: 0
0811 18:57:30.704706 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.704907 (+ 201us) log.cc:842] Serialized 725 byte log entry
0811 18:57:30.832468 (+127561us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.834427 (+ 1959us) write_op.cc:301] APPLY: starting
0811 18:57:30.834443 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.834485 (+ 42us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.834592 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.834703 (+ 111us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.834741 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.834751 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:30.834820 (+ 69us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.834866 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.834884 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:30.834891 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.834952 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.834958 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1913,"apply.run_cpu_time_us":586,"apply.run_wall_time_us":584,"num_ops":2,"prepare.queue_time_us":7667,"prepare.run_cpu_time_us":833,"prepare.run_wall_time_us":1317,"replication_time_us":127743,"threads_started":1}]]}
I20260811 18:57:30.842008 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 75ceba7bfc8240268b16c789f07813d1 (DEFAULT_TABLE table=default.Table_13786cfeafbc4312bf84f93b204add4f [id=a11b457b0ad74279949798ccab096915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.843935 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 452ee450778e4d33adfa7e2745b5d970 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:30.848248 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3030c31bc2b24ee296f29b5d5aa402ff
I20260811 18:57:30.848860 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1026) took 1040 ms. Trace:
I20260811 18:57:30.849078 3546 rpcz_store.cc:276] 0811 18:57:29.813255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.813936 (+ 681us) service_pool.cc:227] Handling call
0811 18:57:29.814659 (+ 723us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.814862 (+ 203us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.814953 (+ 91us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.696924 (+881971us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.847930 (+151006us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.848063 (+ 133us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.848120 (+ 57us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.848682 (+ 562us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.705537 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.705626 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.705632 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:30.705634 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.705736 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.705748 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.705753 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.705884 (+ 131us) tablet.cc:699] Row locks acquired
0811 18:57:30.705886 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:30.705923 (+ 37us) write_op.cc:270] Start()
0811 18:57:30.705967 (+ 44us) write_op.cc:276] Timestamp: P: 1786474650705871 usec, L: 0
0811 18:57:30.705972 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.706180 (+ 208us) log.cc:842] Serialized 756 byte log entry
0811 18:57:30.845203 (+139023us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.845570 (+ 367us) write_op.cc:301] APPLY: starting
0811 18:57:30.845585 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.845628 (+ 43us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.845667 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.845735 (+ 68us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.845768 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.845775 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:30.845849 (+ 74us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.845906 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.845926 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:30.845934 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.845983 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.845987 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":464,"num_ops":2,"prepare.queue_time_us":7893,"prepare.run_cpu_time_us":719,"prepare.run_wall_time_us":1121,"replication_time_us":139202}]]}
I20260811 18:57:30.850142 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a" } new_table_name: "default.Table_8708191242b744d0996310157990b329" modify_external_catalogs: true
I20260811 18:57:30.852799 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf54e295be64486a90918a8d2eb8b33 (DEFAULT_TABLE table=default.Table_4b38a0019b5541c495088e96174c8de4 [id=05dc3f3a80494c168dcc8e02a62e2bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.853914 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 14799a6f61a946008acee8a09d050c1e (DEFAULT_TABLE table=default.Table_b752cb750e9447b092ecac296b5f836e [id=839eec1f02c342f3925e487ab8277f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.855845 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6009c9f6bbb0493ebe125120a1a47f3f
I20260811 18:57:30.856660 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1027) took 1033 ms. Trace:
I20260811 18:57:30.856992 3550 rpcz_store.cc:276] 0811 18:57:29.827539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.831654 (+ 4115us) service_pool.cc:227] Handling call
0811 18:57:29.832490 (+ 836us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.832685 (+ 195us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.832764 (+ 79us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.698384 (+865620us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.850426 (+152042us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.855617 (+ 5191us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.855659 (+ 42us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.856523 (+ 864us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.706694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.706783 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.706788 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:30.706790 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.706920 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.706932 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.706938 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.707047 (+ 109us) tablet.cc:699] Row locks acquired
0811 18:57:30.707052 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:30.707104 (+ 52us) write_op.cc:270] Start()
0811 18:57:30.707141 (+ 37us) write_op.cc:276] Timestamp: P: 1786474650707039 usec, L: 0
0811 18:57:30.707147 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.707347 (+ 200us) log.cc:842] Serialized 756 byte log entry
0811 18:57:30.845319 (+137972us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.846063 (+ 744us) write_op.cc:301] APPLY: starting
0811 18:57:30.846072 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.846109 (+ 37us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.846129 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.846170 (+ 41us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.846200 (+ 30us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.846207 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:30.846252 (+ 45us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.846286 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.846299 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:30.846304 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.846370 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.846374 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":715,"apply.run_cpu_time_us":350,"apply.run_wall_time_us":350,"num_ops":2,"prepare.queue_time_us":7417,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":1114,"replication_time_us":138155}]]}
I20260811 18:57:30.857928 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec5b2214e0c439ea122d11aa6e6f396 (DEFAULT_TABLE table=default.Table_a677dcb5a43648a49944a18f38fb74fa [id=f3978719f37e4b43847eda64b65f6cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.858716 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 03ad1334d0b14ca8adb813e85a429f9e (DEFAULT_TABLE table=default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 [id=76b97a0511a84aa5b2106c833a908799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.858649 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_03b64ac0b80f46288fcf2b34397efcaa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.859426 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03b64ac0b80f46288fcf2b34397efcaa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.859367 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 (DEFAULT_TABLE table=default.Table_f14f93014a8d464ba0da0bab0ceb9ddc [id=9ea613e25082465f89a34c9da09d3855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.859977 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 531ba05901734d6b93bc2be506e8ec5b (DEFAULT_TABLE table=default.Table_495f858f5eec496680f982bccd44415d [id=40aeb51344cf4308ab20fa39bffbfecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.860613 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7300b005e34e4c95de73e8d62faa64 (DEFAULT_TABLE table=default.Table_3982bc659a664e949beb16b472873d9a [id=da2587935be0402cad1db22e34c471b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.862227 3857 tablet_service.cc:1467] Processing CreateTablet for tablet e5f47ccc193d48efa15d60a06b148e00 (DEFAULT_TABLE table=default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a [id=3c2bb15d58c34694b03ffc72080c28f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.862710 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 90fde17971ea49aaad212a5836661b08
I20260811 18:57:30.863659 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 73c519528f75423195b6bdf6683fd91b (DEFAULT_TABLE table=default.Table_7df549360f014d629938e3e94659af8a [id=0915677ac44e4a1dad598b19c54633b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.863811 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aadee0d920fc4be5aa7f965bc9db93e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.864621 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aadee0d920fc4be5aa7f965bc9db93e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.866170 4163 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.866614 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7d747f03ad234805a4ea4ef60840d426
I20260811 18:57:30.866309 4163 ts_tablet_manager.cc:1397] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.178s user 0.007s sys 0.004s
I20260811 18:57:30.867231 3549 catalog_manager.cc:6565] ReplaceTablet: tablet 7d747f03ad234805a4ea4ef60840d426 deleted and replaced by tablet 97d5c52e361c4f08904475acf62dae2b
I20260811 18:57:30.869204 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b79839172cff41c3a8235675d44c21fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.870071 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b79839172cff41c3a8235675d44c21fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.870931 3870 tablet_service.cc:1467] Processing CreateTablet for tablet fa5a3d1ada554d778d507382c8836133 (DEFAULT_TABLE table=default.Table_e2635923a5784be1839e861c2277e900 [id=f6576859698f477bae1ad63c86d99eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.872613 4163 raft_consensus.cc:348] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.872942 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddf22cbb415c485ebc1407d9104b715d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.873750 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddf22cbb415c485ebc1407d9104b715d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.875420 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (table default.Table_f5f50e8f17e04760a61e10c54d1aa657 [id=de9b6daef0ed416aa79b0dc0f4b02c2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:30.877728 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d4fb4116244d9e8c08ce623bea3a2e" } new_table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" modify_external_catalogs: true
I20260811 18:57:30.873858 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc (DEFAULT_TABLE table=default.Table_b485cd94aa7c4642a0b3d0021cef63c4 [id=256649498ffa421b9ef8837c0c25137e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.875149 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6f804703e20c4214b8b2a31efeaef59e (DEFAULT_TABLE table=default.Table_dbf1868109474dcea3712c10bb4d4f3f [id=f5435385b235444594731609da68bd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.875748 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ab04517e943e410cb65ae11d4abf5286 (DEFAULT_TABLE table=default.Table_32f181a8b481409187abd0dc0a1765ba [id=f9ceba7031af4bd8a4c270368e018851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.876636 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 1474691d8bec4698981ff71ef23ff857 (DEFAULT_TABLE table=default.Table_b2be35bff506498281ab88f9cbb55de8 [id=32ffcc8e6240470683988a9c96d21573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.877334 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b702aef88e124443888002d8ccd47b2f (DEFAULT_TABLE table=default.Table_cef66a1c435645d6822188ec4776526b [id=73c0679bc54b446a927b0f8a96cc4afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.877889 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 37d6d857025e4541bd9f480d0a3f4d45 (DEFAULT_TABLE table=default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd [id=e3136d15218f463ab30849b81869fc5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.878039 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 30d44d69b76545b8879bdfc0e6b7008e (DEFAULT_TABLE table=default.Table_ddc87769613d45caa57f2df0032dc979 [id=c2eaa36cde1c45c096e0d584a4448d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.878861 4163 raft_consensus.cc:374] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.880714 3547 catalog_manager.cc:5495] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:30.881280 3862 tablet_service.cc:1467] Processing CreateTablet for tablet afa57ed239ac4afebe517cc48d879645 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.881734 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:30.880438 4163 raft_consensus.cc:729] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.882920 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8ad9f329a9c449e4ade20157c2601ee8" } new_table_name: "default.Table_605185b9f67e456ba194522b24112e39" modify_external_catalogs: true
I20260811 18:57:30.885205 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_e03268eb26a144ba8b53cbd960628036 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 3 got 0
I20260811 18:57:30.884346 4163 consensus_queue.cc:260] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.886318 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b3822f191c44112a69c6300fa97a3ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.888021 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b3822f191c44112a69c6300fa97a3ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.889729 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 22272ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:30.892302 4163 ts_tablet_manager.cc:1428] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
W20260811 18:57:30.883718 3824 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.576s (SENT)
I20260811 18:57:30.894470 3824 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:30.895412 3824 raft_consensus.cc:2670] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:30.896371 3849 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 147) took 1190 ms. Trace:
I20260811 18:57:30.899133 3849 rpcz_store.cc:276] 0811 18:57:29.705499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.705566 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:30.573916 (+868350us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x6150000977a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.715444 (+141528us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x6150000bb424. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.896249 (+180805us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":381380224}
I20260811 18:57:30.902524 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7300b005e34e4c95de73e8d62faa64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.904371 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f47ccc193d48efa15d60a06b148e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.904814 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 1474691d8bec4698981ff71ef23ff857 (DEFAULT_TABLE table=default.Table_b2be35bff506498281ab88f9cbb55de8 [id=32ffcc8e6240470683988a9c96d21573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.905419 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1474691d8bec4698981ff71ef23ff857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.906782 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ceba7bfc8240268b16c789f07813d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.907207 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb1c9787d3f4fd181f86deca28ad160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.910697 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531ba05901734d6b93bc2be506e8ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.911584 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.912918 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d44d69b76545b8879bdfc0e6b7008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913270 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f804703e20c4214b8b2a31efeaef59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913599 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1474691d8bec4698981ff71ef23ff857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913921 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec5b2214e0c439ea122d11aa6e6f396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.914186 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab04517e943e410cb65ae11d4abf5286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.914499 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f47ccc193d48efa15d60a06b148e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.918078 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c519528f75423195b6bdf6683fd91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.918444 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d6d857025e4541bd9f480d0a3f4d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.922493 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b06fa572045f3a5fc5a0b123d77cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.953424 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_04e3e316fa374868a4be1497035d91e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.961050 3678 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.673s (SENT)
I20260811 18:57:30.961321 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 37d6d857025e4541bd9f480d0a3f4d45 (DEFAULT_TABLE table=default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd [id=e3136d15218f463ab30849b81869fc5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.961308 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 531ba05901734d6b93bc2be506e8ec5b (DEFAULT_TABLE table=default.Table_495f858f5eec496680f982bccd44415d [id=40aeb51344cf4308ab20fa39bffbfecb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.959318 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04e3e316fa374868a4be1497035d91e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.979904 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa5a3d1ada554d778d507382c8836133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.981485 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 30d44d69b76545b8879bdfc0e6b7008e (DEFAULT_TABLE table=default.Table_ddc87769613d45caa57f2df0032dc979 [id=c2eaa36cde1c45c096e0d584a4448d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.983906 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d6d857025e4541bd9f480d0a3f4d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.983841 3709 tablet_service.cc:1467] Processing CreateTablet for tablet b702aef88e124443888002d8ccd47b2f (DEFAULT_TABLE table=default.Table_cef66a1c435645d6822188ec4776526b [id=73c0679bc54b446a927b0f8a96cc4afa]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.984068 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 80020.1us
I20260811 18:57:30.994205 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bf4fa780a3ba4824b3317ae7e9a379fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.995007 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa5a3d1ada554d778d507382c8836133. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.995400 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf4fa780a3ba4824b3317ae7e9a379fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.995584 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 547f6c33d6414bcfa3eeff20cb9a0302 consensus state has no leader
I20260811 18:57:30.996277 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_25e47ee54a2742a7839d23fc2adce361"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.996888 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25e47ee54a2742a7839d23fc2adce361 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.999069 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c0f68d33e1b14144b34c17e3c7201408
W20260811 18:57:30.994227 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7b38d3b9570467a9128bd7fe73acf2f consensus state has no leader
W20260811 18:57:31.001310 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99b788d7e78c4945bb82fe86597aa634 consensus state has no leader
I20260811 18:57:31.003870 3717 tablet_service.cc:1467] Processing CreateTablet for tablet afa57ed239ac4afebe517cc48d879645 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.004701 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf8eed41af347f6af7435854850a617 (DEFAULT_TABLE table=default.Table_b8dd0414c4784bbea975b8d3f1491cd6 [id=f2733be482bf4c9a97103d57fd5804f3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:31.005144 3678 leader_election.cc:334] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.134s (SENT)
I20260811 18:57:31.005451 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d44d69b76545b8879bdfc0e6b7008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.012465 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf8eed41af347f6af7435854850a617 (DEFAULT_TABLE table=default.Table_b8dd0414c4784bbea975b8d3f1491cd6 [id=f2733be482bf4c9a97103d57fd5804f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.014642 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ad1334d0b14ca8adb813e85a429f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.020716 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.023564 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa57ed239ac4afebe517cc48d879645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.024734 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.025267 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b702aef88e124443888002d8ccd47b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.026419 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab04517e943e410cb65ae11d4abf5286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.026516 3678 leader_election.cc:302] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:31.027262 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf54e295be64486a90918a8d2eb8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.028707 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7300b005e34e4c95de73e8d62faa64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.029947 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fde17971ea49aaad212a5836661b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.030877 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d747f03ad234805a4ea4ef60840d426 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d5c52e361c4f08904475acf62dae2b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.032315 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14799a6f61a946008acee8a09d050c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.027493 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 101994cc708a4fadb46fc8c5915a5587 (DEFAULT_TABLE table=default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.032982 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101994cc708a4fadb46fc8c5915a5587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.025142 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.035370 3678 raft_consensus.cc:2670] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:31.039260 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c519528f75423195b6bdf6683fd91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.042121 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 622) took 5226 ms. Trace:
I20260811 18:57:31.044178 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0322ec9fd9f24338bb9196b099f56643" } modify_external_catalogs: true
I20260811 18:57:31.044572 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0322ec9fd9f24338bb9196b099f56643" } modify_external_catalogs: true
I20260811 18:57:31.045691 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c284a387f502480c8faa36ea0e40a8f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.046674 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c284a387f502480c8faa36ea0e40a8f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.047417 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_313ce0e9dd1042e2827c0eebceaf3729" } new_table_name: "default.Table_febb766a2cc1415b8977a6539a9e58eb" modify_external_catalogs: true
I20260811 18:57:31.052551 4169 rpcz_store.cc:276] 0811 18:57:25.815755 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.815884 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:30.733504 (+4917620us) ts_tablet_manager.cc:1145] Deleting tablet 01d564ad80064912a8ce8e31c1b6c4d2
0811 18:57:30.733513 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.765992 (+ 32479us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.813437 (+ 47445us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.042089 (+228652us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":138102,"tablet-delete.queue_time_us":4917279}
I20260811 18:57:31.053054 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf8eed41af347f6af7435854850a617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.055383 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b3822f191c44112a69c6300fa97a3ac" } modify_external_catalogs: true
I20260811 18:57:31.055748 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b3822f191c44112a69c6300fa97a3ac" } modify_external_catalogs: true
I20260811 18:57:31.058035 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1cdea1db192a43f4b210579b0813e33a" } new_table_name: "default.Table_bf526916d0504506a3e70cc658d22416" modify_external_catalogs: true
I20260811 18:57:31.063668 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:31.068710 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 101994cc708a4fadb46fc8c5915a5587 (DEFAULT_TABLE table=default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.069972 4162 tablet_bootstrap.cc:654] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.071115 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e1af5a92fcee4bc4898572e55da333b5" } modify_external_catalogs: true
I20260811 18:57:31.071558 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e1af5a92fcee4bc4898572e55da333b5" } modify_external_catalogs: true
W20260811 18:57:31.072041 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 52461.4us
I20260811 18:57:31.073907 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531ba05901734d6b93bc2be506e8ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.074185 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b06fa572045f3a5fc5a0b123d77cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.074867 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f804703e20c4214b8b2a31efeaef59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.083019 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.084055 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.084411 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:31.087220 3678 leader_election.cc:334] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.345s (SENT)
I20260811 18:57:31.088115 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_160e44b6b513432c89082037a071fbb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.088714 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101994cc708a4fadb46fc8c5915a5587. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.089251 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_160e44b6b513432c89082037a071fbb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.091264 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" } modify_external_catalogs: true
I20260811 18:57:31.091693 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" } modify_external_catalogs: true
I20260811 18:57:31.095220 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d747f03ad234805a4ea4ef60840d426 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d5c52e361c4f08904475acf62dae2b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.111922 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b702aef88e124443888002d8ccd47b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.122439 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf8eed41af347f6af7435854850a617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.123870 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa57ed239ac4afebe517cc48d879645. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.132928 3678 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.941s (SENT)
W20260811 18:57:31.093063 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 141587us
I20260811 18:57:31.135876 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]) successfully deleted
I20260811 18:57:31.137151 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 297988c33d4e42e886fb6f1bd37005a5 (DEFAULT_TABLE table=default.Table_16d7946e87be4700a338a7744c866d5e [id=db85e36a578f47638611ba09c924fa25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.137259 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fde17971ea49aaad212a5836661b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.137929 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297988c33d4e42e886fb6f1bd37005a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.138895 3710 tablet_service.cc:1467] Processing CreateTablet for tablet e7b6f913d82c4ea9830e6e680de3dda1 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.147728 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b6f913d82c4ea9830e6e680de3dda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.147599 3705 tablet_service.cc:1467] Processing CreateTablet for tablet c0f68d33e1b14144b34c17e3c7201408 (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.151103 5334 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.151556 5334 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.152798 5334 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.136160 5300 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:31.157831 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 810b9bdc330e40619f45c70e57724da0 consensus state has no leader
#5 0x7fc7bc21b0b7 in kudu::server::ServerBase::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:1171:23
I20260811 18:57:31.163825 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f68d33e1b14144b34c17e3c7201408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.167799 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f68d33e1b14144b34c17e3c7201408 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.182698 3546 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 06b472ad782c4f9fa9a99a5ae8582ec5
W20260811 18:57:31.189913 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60091bd891a94cc4aac6ef47673a4ee5 has no consensus state
I20260811 18:57:31.196255 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.197171 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.198855 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 99e9474ef8814514832f0e48c2b3495f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.201282 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_016d0800749f4083ad87c2a640e058c8" } new_table_name: "default.Table_19b497f543db41bd9abc6e46da338cd5" modify_external_catalogs: true
I20260811 18:57:31.168520 5300 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.206836 4169 tablet_replica.cc:330] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.206770 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0036c28f59cb4f229da243389602bbd9" } modify_external_catalogs: true
I20260811 18:57:31.207098 4169 raft_consensus.cc:2229] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.207059 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0036c28f59cb4f229da243389602bbd9" } modify_external_catalogs: true
I20260811 18:57:31.207471 4169 raft_consensus.cc:2258] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.207470 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3de36d9c6d4542be9ee2c873f0bfcfca" } new_table_name: "default.Table_0a6176c962034100a92814898fb6e4d0" modify_external_catalogs: true
I20260811 18:57:31.208168 4169 ts_tablet_manager.cc:1905] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.210942 4163 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.217824 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:31.218521 3676 leader_election.cc:341] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.229998 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 01a5fdab68694761b7493b42a0bf7c5b
I20260811 18:57:31.231104 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.231828 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.216516 5300 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:31.239212 4162 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.239418 4162 ts_tablet_manager.cc:1397] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.530s user 0.008s sys 0.007s
I20260811 18:57:31.242743 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c944d5cfd9b3405b8151abfc9f2fd998
I20260811 18:57:31.243688 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f0090725e77446588856c07015fda936"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.244457 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0090725e77446588856c07015fda936 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.259936 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.263082 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.268101 4162 raft_consensus.cc:348] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.268952 4162 raft_consensus.cc:374] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.269208 4162 raft_consensus.cc:729] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.269476 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4e0747ef88574c68ae074732cb7095b9
I20260811 18:57:31.269611 4162 consensus_queue.cc:260] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.271613 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.272387 4162 ts_tablet_manager.cc:1428] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.033s user 0.000s sys 0.003s
I20260811 18:57:31.274871 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_98f9637c9f7a47559887a4c73c52daea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.275672 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98f9637c9f7a47559887a4c73c52daea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.279943 3852 tablet_service.cc:1467] Processing CreateTablet for tablet e7b6f913d82c4ea9830e6e680de3dda1 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.280301 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 297988c33d4e42e886fb6f1bd37005a5 (DEFAULT_TABLE table=default.Table_16d7946e87be4700a338a7744c866d5e [id=db85e36a578f47638611ba09c924fa25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.285143 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7e8ba858b0c44d588499e1463673957" } new_table_name: "default.Table_1314176a6f3346d7a808360af2c1cfd2" modify_external_catalogs: true
I20260811 18:57:31.285964 4162 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.288376 4162 tablet_bootstrap.cc:654] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.289148 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c0f68d33e1b14144b34c17e3c7201408 (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.289150 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b6f913d82c4ea9830e6e680de3dda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.290122 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.290323 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f68d33e1b14144b34c17e3c7201408 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.290994 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297988c33d4e42e886fb6f1bd37005a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.291594 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f68d33e1b14144b34c17e3c7201408. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.289182 3824 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.918s (SENT)
W20260811 18:57:31.293162 3824 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.395s (SENT)
I20260811 18:57:31.294610 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0747ef88574c68ae074732cb7095b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.319373 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0747ef88574c68ae074732cb7095b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.319419 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.319649 3743 raft_consensus.cc:2454] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.321945 4167 ts_tablet_manager.cc:1918] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.322088 4167 log.cc:1196] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c08436f33db043b1bc6395cd92102e59
I20260811 18:57:31.324996 3824 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.326136 4167 ts_tablet_manager.cc:1939] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.326534 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 677) took 4806 ms. Trace:
I20260811 18:57:31.326622 4167 rpcz_store.cc:276] 0811 18:57:26.520485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.520662 (+ 177us) service_pool.cc:227] Handling call
0811 18:57:30.685374 (+4164712us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:30.685385 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.218876 (+533491us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.321835 (+102959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.326516 (+ 4681us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164510}
I20260811 18:57:31.327029 4167 tablet_replica.cc:330] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.327205 4167 raft_consensus.cc:2229] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.327804 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c08436f33db043b1bc6395cd92102e59 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
I20260811 18:57:31.328984 4163 tablet_bootstrap.cc:654] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.330184 4167 raft_consensus.cc:2258] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.330778 4167 ts_tablet_manager.cc:1905] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.332149 5334 raft_consensus.cc:2781] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.332278 5334 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.332376 5334 raft_consensus.cc:3037] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.339773 5334 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.354269 5334 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.355221 4169 ts_tablet_manager.cc:1918] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.355383 4169 log.cc:1196] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:31.355859 4169 ts_tablet_manager.cc:1939] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.356282 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 623) took 5436 ms. Trace:
I20260811 18:57:31.356390 4169 rpcz_store.cc:276] 0811 18:57:25.920076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.920285 (+ 209us) service_pool.cc:227] Handling call
0811 18:57:31.052919 (+5132634us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:31.204263 (+151344us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.323522 (+119259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.330554 (+ 7032us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.356258 (+ 25704us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5107381}
I20260811 18:57:31.356997 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:31.359787 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_e03268eb26a144ba8b53cbd960628036 [id=64009634513f46eeae6442cfadb7b0f5] alter complete (version 3)
I20260811 18:57:31.360834 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 624) took 5414 ms. Trace:
I20260811 18:57:31.360932 4169 rpcz_store.cc:276] 0811 18:57:25.946900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.946959 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:31.360693 (+5413734us) ts_tablet_manager.cc:1145] Deleting tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
0811 18:57:31.360705 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.360812 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5412354}
I20260811 18:57:31.361440 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 625) took 5401 ms. Trace:
I20260811 18:57:31.361522 4169 rpcz_store.cc:276] 0811 18:57:25.959568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.959646 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:31.361326 (+5401680us) ts_tablet_manager.cc:1145] Deleting tablet 84e152c74f33463e83843b92c0783803
0811 18:57:31.361337 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.361423 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401444}
I20260811 18:57:31.362035 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 627) took 5401 ms. Trace:
I20260811 18:57:31.362128 4169 rpcz_store.cc:276] 0811 18:57:25.960158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960237 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:31.361925 (+5401688us) ts_tablet_manager.cc:1145] Deleting tablet 86d690ec09524248bc147ad7408e2ee6
0811 18:57:31.361938 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.362017 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401495}
W20260811 18:57:31.363068 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa already in progress: creating tablet
W20260811 18:57:31.363430 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 84e152c74f33463e83843b92c0783803 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e152c74f33463e83843b92c0783803 already in progress: creating tablet
I20260811 18:57:31.363420 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.364074 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.364285 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 626) took 5402 ms. Trace:
I20260811 18:57:31.364384 4169 rpcz_store.cc:276] 0811 18:57:25.960055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960356 (+ 301us) service_pool.cc:227] Handling call
0811 18:57:31.362521 (+5402165us) ts_tablet_manager.cc:1145] Deleting tablet 649cb98f553a418b911e307fdaf49917
0811 18:57:31.362532 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.362616 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401984}
I20260811 18:57:31.364940 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 628) took 5404 ms. Trace:
I20260811 18:57:31.365121 4169 rpcz_store.cc:276] 0811 18:57:25.960296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960472 (+ 176us) service_pool.cc:227] Handling call
0811 18:57:31.364800 (+5404328us) ts_tablet_manager.cc:1145] Deleting tablet 6d28c486e408451a990ae3905ef3c2ff
0811 18:57:31.364817 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.364919 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5404045}
I20260811 18:57:31.365725 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 629) took 5396 ms. Trace:
I20260811 18:57:31.365859 4169 rpcz_store.cc:276] 0811 18:57:25.969588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.973583 (+ 3995us) service_pool.cc:227] Handling call
0811 18:57:31.365616 (+5392033us) ts_tablet_manager.cc:1145] Deleting tablet b7b48a35acf1486bbb63680ca5da31dd
0811 18:57:31.365628 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.365707 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5390916}
W20260811 18:57:31.366539 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 649cb98f553a418b911e307fdaf49917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 649cb98f553a418b911e307fdaf49917 already in progress: creating tablet
W20260811 18:57:31.366881 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6d28c486e408451a990ae3905ef3c2ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d28c486e408451a990ae3905ef3c2ff already in progress: creating tablet
W20260811 18:57:31.367142 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b7b48a35acf1486bbb63680ca5da31dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b48a35acf1486bbb63680ca5da31dd already in progress: creating tablet
I20260811 18:57:31.369416 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5b76a052465c4749a059cbf9171ab6be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.369917 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.370158 3743 raft_consensus.cc:3037] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:31.370671 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b76a052465c4749a059cbf9171ab6be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.372800 3743 raft_consensus.cc:2454] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.373742 3824 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.374250 5334 raft_consensus.cc:2781] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.374428 5334 raft_consensus.cc:686] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.374673 5334 consensus_queue.cc:237] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.375128 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 761902f7366d43ffbde226a0ade2e188
I20260811 18:57:31.375174 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 99e9474ef8814514832f0e48c2b3495f
I20260811 18:57:31.375707 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 99e9474ef8814514832f0e48c2b3495f deleted and replaced by tablet bcfad3da366e4554b06bfa51a2535419
I20260811 18:57:31.376058 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 3dd4ec30acde4585a1c3451927217cb3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.376539 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e9474ef8814514832f0e48c2b3495f with delete_type TABLET_DATA_DELETED (replaced by tablet bcfad3da366e4554b06bfa51a2535419) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.377559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 641) took 5267 ms. Trace:
W20260811 18:57:31.377609 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e4f4cc4dace944be894fe12690b18ccb consensus state has no leader
I20260811 18:57:31.377664 4169 rpcz_store.cc:276] 0811 18:57:26.110680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.110738 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:31.377405 (+5266667us) ts_tablet_manager.cc:1145] Deleting tablet ea4ad482bde44d6bab5ec809054aaebd
0811 18:57:31.377418 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.377531 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5262932}
I20260811 18:57:31.378424 4169 tablet_replica.cc:330] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.378285 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_96bf47f7003740cba4c66f780597a121"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.379182 4169 raft_consensus.cc:2229] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:31.379405 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96bf47f7003740cba4c66f780597a121 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.379521 4169 raft_consensus.cc:2258] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.382655 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 761902f7366d43ffbde226a0ade2e188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.383337 4169 ts_tablet_manager.cc:1905] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.383510 5283 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.383636 5283 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.384248 5283 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.386502 5313 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.387168 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e9474ef8814514832f0e48c2b3495f with delete_type TABLET_DATA_DELETED (replaced by tablet bcfad3da366e4554b06bfa51a2535419) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.387522 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 761902f7366d43ffbde226a0ade2e188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.388558 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ea4ad482bde44d6bab5ec809054aaebd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea4ad482bde44d6bab5ec809054aaebd already in progress: creating tablet
I20260811 18:57:31.388818 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_619d1e7c0e384107962b41e86e45c45b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.389784 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_619d1e7c0e384107962b41e86e45c45b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.392694 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.392987 3744 raft_consensus.cc:2454] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.393877 3824 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.394220 5283 raft_consensus.cc:2781] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.394327 5283 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.394438 5283 raft_consensus.cc:3037] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.397310 5283 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.398873 5283 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.399890 4163 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:31.400954 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 60743.4us
I20260811 18:57:31.401727 4163 ts_tablet_manager.cc:1397] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.197s user 0.003s sys 0.007s
I20260811 18:57:31.403328 4163 raft_consensus.cc:348] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.406936 4163 raft_consensus.cc:374] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.407591 4163 raft_consensus.cc:729] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.407626 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.408340 4163 consensus_queue.cc:260] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.409806 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.410566 3743 raft_consensus.cc:3037] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.409301 5361 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:31.411610 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73e15954a1604189af634a727cb48690 consensus state has no leader
I20260811 18:57:31.415506 4163 ts_tablet_manager.cc:1428] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:31.430784 4163 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.425846 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:31.441107 3678 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.285s (SENT)
I20260811 18:57:31.441912 3678 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:31.442186 3678 raft_consensus.cc:2670] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:31.444455 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9183f700c0544cd5942abebaaa8a121f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.445653 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9183f700c0544cd5942abebaaa8a121f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.451563 5313 tablet.cc:1722] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.458211 4163 tablet_bootstrap.cc:654] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.448707 3743 raft_consensus.cc:2454] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.465565 3824 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.465880 5334 raft_consensus.cc:2781] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.466024 5334 raft_consensus.cc:686] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.466472 5334 consensus_queue.cc:237] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.470428 5361 tablet.cc:1722] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.471313 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2698f49dd4b64fce9a411b1c3307abf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.472277 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2698f49dd4b64fce9a411b1c3307abf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.477109 3678 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.631s (SENT)
I20260811 18:57:31.477592 4167 ts_tablet_manager.cc:1918] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.480825 4163 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.480999 4163 ts_tablet_manager.cc:1397] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.002s
I20260811 18:57:31.481752 4167 log.cc:1196] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:31.481980 4163 raft_consensus.cc:348] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.482223 4163 raft_consensus.cc:374] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.482293 4163 raft_consensus.cc:729] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.482609 4163 consensus_queue.cc:260] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.482756 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8214722ffbdc49bf98d5aa539aebed52" } new_table_name: "default.Table_6936ad30157d4323baa32612df13a29d" modify_external_catalogs: true
I20260811 18:57:31.483357 4163 ts_tablet_manager.cc:1428] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:31.483656 4163 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.484130 4167 ts_tablet_manager.cc:1939] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.488638 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 678) took 4781 ms. Trace:
I20260811 18:57:31.461525 4169 ts_tablet_manager.cc:1918] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.492537 4169 log.cc:1196] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ff1be026ca404eada53f60f257ea367c
I20260811 18:57:31.493014 4169 ts_tablet_manager.cc:1939] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.493400 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 649) took 5339 ms. Trace:
I20260811 18:57:31.493501 4169 rpcz_store.cc:276] 0811 18:57:26.153766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.153893 (+ 127us) service_pool.cc:227] Handling call
0811 18:57:31.378372 (+5224479us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:31.378384 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.453563 (+ 75179us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.461411 (+ 7848us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.493382 (+ 31971us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5222934}
I20260811 18:57:31.493922 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 648) took 5340 ms. Trace:
I20260811 18:57:31.494012 4169 rpcz_store.cc:276] 0811 18:57:26.153613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.153853 (+ 240us) service_pool.cc:227] Handling call
0811 18:57:31.493808 (+5339955us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:31.493821 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.493903 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5338164}
I20260811 18:57:31.494410 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 654) took 5288 ms. Trace:
I20260811 18:57:31.497661 4163 tablet_bootstrap.cc:654] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:31.498924 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:31.499506 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_533afa7d5e834d5cb281ddc9619575f8" } modify_external_catalogs: true
I20260811 18:57:31.499871 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_533afa7d5e834d5cb281ddc9619575f8" } modify_external_catalogs: true
I20260811 18:57:31.494494 4169 rpcz_store.cc:276] 0811 18:57:26.206183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.206255 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:31.494281 (+5288026us) ts_tablet_manager.cc:1145] Deleting tablet 54fa32f8fa1144c4bd9dcd00a550001a
0811 18:57:31.494291 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.494392 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5283592}
I20260811 18:57:31.501606 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_378dd90b6af541b8ab95d9f19681a03f" } new_table_name: "default.Table_6090c727ddfa494594649314a3fdf57e" modify_external_catalogs: true
I20260811 18:57:31.502136 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 655) took 5295 ms. Trace:
I20260811 18:57:31.502246 4169 rpcz_store.cc:276] 0811 18:57:26.206304 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.210707 (+ 4403us) service_pool.cc:227] Handling call
0811 18:57:31.501986 (+5291279us) ts_tablet_manager.cc:1145] Deleting tablet b16cb4c1796147fbb10109f52071505e
0811 18:57:31.502002 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.502117 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5268892}
I20260811 18:57:31.504639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 653) took 5298 ms. Trace:
I20260811 18:57:31.505928 4169 rpcz_store.cc:276] 0811 18:57:26.206056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.206157 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:31.504498 (+5298341us) ts_tablet_manager.cc:1145] Deleting tablet 1d81e93739f44d9e99b6ebc647e1d0e4
0811 18:57:31.504513 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.504617 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5263880}
I20260811 18:57:31.507205 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 656) took 5236 ms. Trace:
I20260811 18:57:31.508425 4169 rpcz_store.cc:276] 0811 18:57:26.270719 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.270781 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:31.507077 (+5236296us) ts_tablet_manager.cc:1145] Deleting tablet 1811e5f62a714f8388460dfcb1717e81
0811 18:57:31.507089 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.507187 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5236031}
I20260811 18:57:31.509730 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 657) took 4910 ms. Trace:
I20260811 18:57:31.510908 4169 rpcz_store.cc:276] 0811 18:57:26.599056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.599617 (+ 561us) service_pool.cc:227] Handling call
0811 18:57:31.509600 (+4909983us) ts_tablet_manager.cc:1145] Deleting tablet 335fc15ea75d483990e6ae5a5872682f
0811 18:57:31.509614 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.509711 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4907122}
I20260811 18:57:31.494073 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ff1be026ca404eada53f60f257ea367c (table default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]) successfully deleted
W20260811 18:57:31.514632 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c2247194db704f9abd54a8a0df1ca566 because the tablet was not found. No further retry: Not found: Tablet not found: c2247194db704f9abd54a8a0df1ca566
W20260811 18:57:31.515009 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 54fa32f8fa1144c4bd9dcd00a550001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54fa32f8fa1144c4bd9dcd00a550001a already in progress: creating tablet
W20260811 18:57:31.515278 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b16cb4c1796147fbb10109f52071505e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16cb4c1796147fbb10109f52071505e already in progress: creating tablet
W20260811 18:57:31.515518 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d81e93739f44d9e99b6ebc647e1d0e4 already in progress: creating tablet
W20260811 18:57:31.515800 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1811e5f62a714f8388460dfcb1717e81 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1811e5f62a714f8388460dfcb1717e81 already in progress: creating tablet
W20260811 18:57:31.516121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 335fc15ea75d483990e6ae5a5872682f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 335fc15ea75d483990e6ae5a5872682f already in progress: creating tablet
I20260811 18:57:31.495963 5091 raft_consensus.cc:482] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.522182 5091 raft_consensus.cc:504] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.522953 5091 leader_election.cc:288] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.523553 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f0cdcee681460084662e01152ad9dd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:31.523625 4167 rpcz_store.cc:276] 0811 18:57:26.706835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.707031 (+ 196us) service_pool.cc:227] Handling call
0811 18:57:31.326981 (+4619950us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:31.326992 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.350825 (+ 23833us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.474084 (+123259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.488612 (+ 14528us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4619721}
W20260811 18:57:31.524071 3676 leader_election.cc:341] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.524659 4167 tablet_replica.cc:330] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.524844 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b0b199f4c9e14b468cb26cbd61fc8586 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:31.525044 4167 raft_consensus.cc:2229] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.525282 4167 raft_consensus.cc:2258] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.526007 4167 ts_tablet_manager.cc:1905] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.531213 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 659) took 4812 ms. Trace:
I20260811 18:57:31.532133 4169 rpcz_store.cc:276] 0811 18:57:26.719126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.719208 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:31.531082 (+4811874us) ts_tablet_manager.cc:1145] Deleting tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
0811 18:57:31.531094 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.531190 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4811553}
I20260811 18:57:31.533308 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 658) took 4814 ms. Trace:
I20260811 18:57:31.533696 4162 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.533939 4162 ts_tablet_manager.cc:1397] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.248s user 0.008s sys 0.018s
I20260811 18:57:31.534060 4169 rpcz_store.cc:276] 0811 18:57:26.718855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.718941 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:31.533173 (+4814232us) ts_tablet_manager.cc:1145] Deleting tablet 705605cd929940f8a252b6e95abbeb95
0811 18:57:31.533186 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.533289 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4804744}
I20260811 18:57:31.534893 4162 raft_consensus.cc:348] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.535118 4162 raft_consensus.cc:374] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.535178 4162 raft_consensus.cc:729] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.536173 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 680) took 4462 ms. Trace:
I20260811 18:57:31.536895 4169 rpcz_store.cc:276] 0811 18:57:27.073302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.073385 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:31.536047 (+4462662us) ts_tablet_manager.cc:1145] Deleting tablet 37be4f7579fa4ab99ddbacfa80282b42
0811 18:57:31.536058 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.536158 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4462381}
I20260811 18:57:31.537889 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 681) took 4464 ms. Trace:
I20260811 18:57:31.538491 4169 rpcz_store.cc:276] 0811 18:57:27.073538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.073594 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:31.537789 (+4464195us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:31.537800 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.537874 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4464019}
I20260811 18:57:31.539608 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 682) took 4465 ms. Trace:
I20260811 18:57:31.540270 4169 rpcz_store.cc:276] 0811 18:57:27.073959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.074008 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:31.539502 (+4465494us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:31.539514 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.539592 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4465336}
I20260811 18:57:31.541846 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 683) took 4441 ms. Trace:
I20260811 18:57:31.542372 4169 rpcz_store.cc:276] 0811 18:57:27.100057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.100124 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:31.541327 (+4441203us) ts_tablet_manager.cc:1145] Deleting tablet 28af2d2c61254b0ca399dddb05e9ee70
0811 18:57:31.541342 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.541822 (+ 480us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4440756}
I20260811 18:57:31.543363 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 684) took 4442 ms. Trace:
I20260811 18:57:31.543991 4169 rpcz_store.cc:276] 0811 18:57:27.100734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.100810 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:31.543251 (+4442441us) ts_tablet_manager.cc:1145] Deleting tablet 9722572533de436cb9149e6230570382
0811 18:57:31.543274 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.543348 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4442231}
I20260811 18:57:31.544909 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 691) took 4317 ms. Trace:
I20260811 18:57:31.545466 4169 rpcz_store.cc:276] 0811 18:57:27.227809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.232631 (+ 4822us) service_pool.cc:227] Handling call
0811 18:57:31.544816 (+4312185us) ts_tablet_manager.cc:1145] Deleting tablet 90576c068ce4468a9c572307dcb79b10
0811 18:57:31.544826 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.544896 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4311839}
I20260811 18:57:31.546324 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 696) took 3996 ms. Trace:
I20260811 18:57:31.546527 4162 consensus_queue.cc:260] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.547154 4169 rpcz_store.cc:276] 0811 18:57:27.549972 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.550526 (+ 554us) service_pool.cc:227] Handling call
0811 18:57:31.546207 (+3995681us) ts_tablet_manager.cc:1145] Deleting tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
0811 18:57:31.546218 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.546308 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3995447}
I20260811 18:57:31.548094 5334 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.548295 5334 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.549378 5334 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.549791 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 697) took 3999 ms. Trace:
I20260811 18:57:31.550745 4169 rpcz_store.cc:276] 0811 18:57:27.550173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.550805 (+ 632us) service_pool.cc:227] Handling call
0811 18:57:31.549680 (+3998875us) ts_tablet_manager.cc:1145] Deleting tablet a49b8b344177487fb5bd096ee9cdea2f
0811 18:57:31.549692 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.549775 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3998631}
I20260811 18:57:31.550727 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.551245 3740 raft_consensus.cc:2454] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.551661 3824 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.552031 5334 raft_consensus.cc:2781] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260811 18:57:31.533169 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a already in progress: creating tablet
I20260811 18:57:31.552211 5334 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.552340 5334 raft_consensus.cc:3037] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:31.552634 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 705605cd929940f8a252b6e95abbeb95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 705605cd929940f8a252b6e95abbeb95 already in progress: creating tablet
W20260811 18:57:31.553051 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 37be4f7579fa4ab99ddbacfa80282b42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37be4f7579fa4ab99ddbacfa80282b42 already in progress: creating tablet
W20260811 18:57:31.553409 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0aee6beb2e70421083841520733087f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aee6beb2e70421083841520733087f8 already in progress: creating tablet
W20260811 18:57:31.553798 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7251204325dd424192a6f58e18e71c34 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7251204325dd424192a6f58e18e71c34 already in progress: creating tablet
W20260811 18:57:31.554085 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 28af2d2c61254b0ca399dddb05e9ee70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28af2d2c61254b0ca399dddb05e9ee70 already in progress: creating tablet
W20260811 18:57:31.554425 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9722572533de436cb9149e6230570382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9722572533de436cb9149e6230570382 already in progress: creating tablet
W20260811 18:57:31.554733 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 90576c068ce4468a9c572307dcb79b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90576c068ce4468a9c572307dcb79b10 already in progress: creating tablet
W20260811 18:57:31.555205 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 already in progress: creating tablet
W20260811 18:57:31.555534 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a49b8b344177487fb5bd096ee9cdea2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49b8b344177487fb5bd096ee9cdea2f already in progress: creating tablet
I20260811 18:57:31.555637 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 698) took 4005 ms. Trace:
I20260811 18:57:31.556449 4169 rpcz_store.cc:276] 0811 18:57:27.550316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.551082 (+ 766us) service_pool.cc:227] Handling call
0811 18:57:31.555507 (+4004425us) ts_tablet_manager.cc:1145] Deleting tablet c21eb02db3de4c97878fa8badccaa6e3
0811 18:57:31.555521 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.555617 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4004188}
I20260811 18:57:31.558825 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 699) took 4008 ms. Trace:
W20260811 18:57:31.558784 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c21eb02db3de4c97878fa8badccaa6e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c21eb02db3de4c97878fa8badccaa6e3 already in progress: creating tablet
I20260811 18:57:31.559975 4169 rpcz_store.cc:276] 0811 18:57:27.550461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.551385 (+ 924us) service_pool.cc:227] Handling call
0811 18:57:31.558697 (+4007312us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:31.558711 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.558806 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4007118}
I20260811 18:57:31.559908 4163 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.560732 4163 ts_tablet_manager.cc:1397] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.006s sys 0.005s
W20260811 18:57:31.560760 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0cccebfc54484a9b9abffcce4e459fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cccebfc54484a9b9abffcce4e459fb5 already in progress: creating tablet
I20260811 18:57:31.561676 4163 raft_consensus.cc:348] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.562288 4163 raft_consensus.cc:374] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.562446 4163 raft_consensus.cc:729] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.562752 4163 consensus_queue.cc:260] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.558306 5334 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.562060 4169 tablet_replica.cc:330] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.564009 4169 raft_consensus.cc:2229] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.564635 5334 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.565074 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.565366 3743 raft_consensus.cc:3037] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.569280 3743 raft_consensus.cc:2454] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.570613 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet c08436f33db043b1bc6395cd92102e59 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.572365 4162 ts_tablet_manager.cc:1428] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.038s user 0.002s sys 0.000s
I20260811 18:57:31.569718 3824 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.575850 5283 raft_consensus.cc:2781] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.575973 5283 raft_consensus.cc:686] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.576218 5283 consensus_queue.cc:237] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.581532 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" } modify_external_catalogs: true
I20260811 18:57:31.581836 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" } modify_external_catalogs: true
I20260811 18:57:31.582317 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c05dd720298449e9aa85faeadb50b96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.583009 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c05dd720298449e9aa85faeadb50b96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.587055 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" } modify_external_catalogs: true
I20260811 18:57:31.575172 4169 pending_rounds.cc:62] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:31.587371 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" } modify_external_catalogs: true
I20260811 18:57:31.587529 4169 pending_rounds.cc:68] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400155312156672 op_type: NO_OP noop_request { }
I20260811 18:57:31.588119 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_680160aaac0a4c8da66634dc41d6abb9" } modify_external_catalogs: true
I20260811 18:57:31.588344 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_680160aaac0a4c8da66634dc41d6abb9" } modify_external_catalogs: true
I20260811 18:57:31.587702 4169 raft_consensus.cc:2866] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:31.588537 4169 raft_consensus.cc:2258] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.595225 4162 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.597196 4162 tablet_bootstrap.cc:654] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.599848 4169 ts_tablet_manager.cc:1905] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.575558 4163 ts_tablet_manager.cc:1428] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
W20260811 18:57:31.605400 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a consensus state has no leader
I20260811 18:57:31.605792 4163 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.607410 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cef66a1c435645d6822188ec4776526b" } modify_external_catalogs: true
I20260811 18:57:31.608039 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0090725e77446588856c07015fda936" } new_table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" modify_external_catalogs: true
I20260811 18:57:31.609104 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cef66a1c435645d6822188ec4776526b" } modify_external_catalogs: true
W20260811 18:57:31.617077 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9948c33ed58b4160932be82ef9615ba5 consensus state has no leader
I20260811 18:57:31.617654 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b08e57f933b24cd0aee53441350e6775" } new_table_name: "default.Table_7065b527f8104608b1efc6787f0b2b77" modify_external_catalogs: true
I20260811 18:57:31.619529 4163 tablet_bootstrap.cc:654] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.631253 4163 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.632062 4163 ts_tablet_manager.cc:1397] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20260811 18:57:31.634958 4163 raft_consensus.cc:348] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.640156 4163 raft_consensus.cc:374] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.640460 4163 raft_consensus.cc:729] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.644471 5091 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.644629 5091 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.642205 4163 consensus_queue.cc:260] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.645277 5091 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.646011 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:31.646659 3676 leader_election.cc:341] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.648420 4167 ts_tablet_manager.cc:1918] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.649539 4167 log.cc:1196] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4687b95728fc485fa0e842bc447eb83f
#6 0x7fc7c23171a3 in kudu::tserver::TabletServer::Start() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server.cc:157:3
W20260811 18:57:31.651232 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb has no consensus state
I20260811 18:57:31.652364 4169 ts_tablet_manager.cc:1918] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.652544 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e0b184f30d7464fbb8120d29bd0a80b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.653347 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e0b184f30d7464fbb8120d29bd0a80b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.661899 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
#7 0x7fc7c2326380 in kudu::tserver::RunTabletServer() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server_runner.cc:86:3
W20260811 18:57:31.663946 3714 consensus_meta.cc:287] T e5f47ccc193d48efa15d60a06b148e00 P bd8c4bdbd811417ab011a7131f17d283: Time spent flushing consensus metadata: real 0.622s user 0.028s sys 0.001s
I20260811 18:57:31.666370 4169 log.cc:1196] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:31.666129 4167 ts_tablet_manager.cc:1939] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.659942 5334 consensus_queue.cc:1035] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:31.670655 5334 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.670854 5334 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.672070 5334 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.669962 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 679) took 4962 ms. Trace:
I20260811 18:57:31.672427 4167 rpcz_store.cc:276] 0811 18:57:26.706983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.707313 (+ 330us) service_pool.cc:227] Handling call
0811 18:57:31.524597 (+4817284us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:31.524611 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.579886 (+ 55275us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.648231 (+ 68345us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.669434 (+ 21203us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":128,"tablet-delete.queue_time_us":4817065}
I20260811 18:57:31.673997 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4687b95728fc485fa0e842bc447eb83f (table default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]) successfully deleted
I20260811 18:57:31.653394 4163 ts_tablet_manager.cc:1428] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260811 18:57:31.678589 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.678854 3740 raft_consensus.cc:2454] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.679036 4163 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:31.679368 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53039.4us
I20260811 18:57:31.681538 4169 ts_tablet_manager.cc:1939] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.684340 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb (DEFAULT_TABLE table=default.Table_4133a1bbf6474492bbf3683ca9608a52 [id=5d4c64c60c974852bceb5a5a71174e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.687107 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdab1c3c431c4dceb08b00eabcfa1ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.687299 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 711) took 4040 ms. Trace:
I20260811 18:57:31.688060 3824 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.687402 4169 rpcz_store.cc:276] 0811 18:57:27.646886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.646977 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:31.562010 (+3915033us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:31.562022 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.628599 (+ 66577us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.652232 (+ 23633us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.687275 (+ 35043us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10753,"tablet-delete.queue_time_us":3914753}
I20260811 18:57:31.688912 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ef393c341bb54baf87176db5b5254ec7 (table default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]) successfully deleted
I20260811 18:57:31.689222 5334 raft_consensus.cc:2781] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.689318 5334 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.689419 5283 consensus_queue.cc:1035] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:31.689422 5334 raft_consensus.cc:3037] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.690842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 712) took 4043 ms. Trace:
I20260811 18:57:31.691290 4169 rpcz_store.cc:276] 0811 18:57:27.647027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.647087 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:31.690700 (+4043613us) ts_tablet_manager.cc:1145] Deleting tablet f287040b879642448367abe2b1c73cd9
0811 18:57:31.690714 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.690823 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4043266}
W20260811 18:57:31.692106 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f287040b879642448367abe2b1c73cd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f287040b879642448367abe2b1c73cd9 already in progress: creating tablet
I20260811 18:57:31.692559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 713) took 4044 ms. Trace:
I20260811 18:57:31.692873 4169 rpcz_store.cc:276] 0811 18:57:27.647726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.647800 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:31.692098 (+4044298us) ts_tablet_manager.cc:1145] Deleting tablet 2be6762e326448d7b677a5641e2b3103
0811 18:57:31.692115 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.692536 (+ 421us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4044020}
W20260811 18:57:31.693732 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2be6762e326448d7b677a5641e2b3103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be6762e326448d7b677a5641e2b3103 already in progress: creating tablet
I20260811 18:57:31.693801 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 721) took 3978 ms. Trace:
I20260811 18:57:31.694553 4169 rpcz_store.cc:276] 0811 18:57:27.715744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.715805 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:31.693670 (+3977865us) ts_tablet_manager.cc:1145] Deleting tablet 4059e5907b1e488ab9e1d04e563f96cd
0811 18:57:31.693682 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.693781 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3977538}
I20260811 18:57:31.694626 5334 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.696170 5334 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.696705 4169 tablet_replica.cc:330] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:31.696691 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4059e5907b1e488ab9e1d04e563f96cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4059e5907b1e488ab9e1d04e563f96cd already in progress: creating tablet
I20260811 18:57:31.696667 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.696913 3745 raft_consensus.cc:3037] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.697273 4169 raft_consensus.cc:2229] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.698467 4169 raft_consensus.cc:2258] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.700875 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7f50a732b44c5a8b6f2eb392455409 (DEFAULT_TABLE table=default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 [id=e0a0d66062a04bd7843d699a54683785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.701403 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7f50a732b44c5a8b6f2eb392455409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.701331 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 825b6743798144deb24330304d6b5d2b (DEFAULT_TABLE table=default.Table_3a0329aed6694593bc21571dd8f59670 [id=37bcf611810c4cd8b8a42926c1064705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.701725 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825b6743798144deb24330304d6b5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.707494 3853 tablet_service.cc:1467] Processing CreateTablet for tablet fae33c49dab3444988738f31180de35a (DEFAULT_TABLE table=default.Table_d584e1cc7d8b45be912d8610e391f4f7 [id=b0d36b86f3114722acc3d45b30e041bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.707830 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7f50a732b44c5a8b6f2eb392455409 (DEFAULT_TABLE table=default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 [id=e0a0d66062a04bd7843d699a54683785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.708098 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 75af8f21dcc7443fabd13e06061d19a6 (DEFAULT_TABLE table=default.Table_949ccbca1e8b44b79578377f67f2422c [id=2764171b8d1a4494bd0d30ecfd267a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.708233 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7f50a732b44c5a8b6f2eb392455409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.700500 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 825b6743798144deb24330304d6b5d2b (DEFAULT_TABLE table=default.Table_3a0329aed6694593bc21571dd8f59670 [id=37bcf611810c4cd8b8a42926c1064705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.710656 3709 tablet_service.cc:1467] Processing CreateTablet for tablet fae33c49dab3444988738f31180de35a (DEFAULT_TABLE table=default.Table_d584e1cc7d8b45be912d8610e391f4f7 [id=b0d36b86f3114722acc3d45b30e041bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.711047 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae33c49dab3444988738f31180de35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.699589 4169 ts_tablet_manager.cc:1905] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.717737 3745 raft_consensus.cc:2454] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.718278 3824 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.719453 5334 raft_consensus.cc:2781] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.720230 5334 raft_consensus.cc:686] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.720553 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4013b1905ea74bc6a06f9468d9025570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.720830 5334 consensus_queue.cc:237] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:31.721385 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4013b1905ea74bc6a06f9468d9025570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.734701 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 20182ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:31.734668 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1eb30e64e86040098dcee5fc2e9b7266
I20260811 18:57:31.735757 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3418766144ec4b91bb802faa2ad8808e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.736508 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3418766144ec4b91bb802faa2ad8808e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.737203 3864 tablet_service.cc:1467] Processing CreateTablet for tablet aafc113d84a44fe9b291fa2efb69ef2e (DEFAULT_TABLE table=default.Table_aaf507832a664e6ebcdafffc433bbac7 [id=b3c972d878764259adfd841f2ddda03f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.738036 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 35ebc32eb5d5443eaaeba245b566f98a
I20260811 18:57:31.738945 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.738878 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a273b6a16afa44d196b4654b067735a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.739270 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.739414 4162 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:31.739534 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a273b6a16afa44d196b4654b067735a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.739562 4162 ts_tablet_manager.cc:1397] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.144s user 0.002s sys 0.001s
I20260811 18:57:31.740655 4162 raft_consensus.cc:348] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.741065 4162 raft_consensus.cc:374] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.741134 4162 raft_consensus.cc:729] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.741456 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_98f9637c9f7a47559887a4c73c52daea" } new_table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" modify_external_catalogs: true
I20260811 18:57:31.741369 4162 consensus_queue.cc:260] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.742264 4162 ts_tablet_manager.cc:1428] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:31.742509 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae33c49dab3444988738f31180de35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.742575 4162 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.742936 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3dd4ec30acde4585a1c3451927217cb3
I20260811 18:57:31.743567 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 3dd4ec30acde4585a1c3451927217cb3 deleted and replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4
I20260811 18:57:31.744060 4162 tablet_bootstrap.cc:654] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.743957 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_79f5b2394f094ecc85733e272dcfe369"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.744645 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79f5b2394f094ecc85733e272dcfe369 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.746510 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75af8f21dcc7443fabd13e06061d19a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.747826 3714 tablet_service.cc:1467] Processing CreateTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb (DEFAULT_TABLE table=default.Table_4133a1bbf6474492bbf3683ca9608a52 [id=5d4c64c60c974852bceb5a5a71174e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.749589 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.750141 3722 tablet_service.cc:1467] Processing CreateTablet for tablet aafc113d84a44fe9b291fa2efb69ef2e (DEFAULT_TABLE table=default.Table_aaf507832a664e6ebcdafffc433bbac7 [id=b3c972d878764259adfd841f2ddda03f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.750501 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aafc113d84a44fe9b291fa2efb69ef2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.750605 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aafc113d84a44fe9b291fa2efb69ef2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.754763 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 862a3101a9b8498693851d4034826ca3 (DEFAULT_TABLE table=default.Table_b2a92f77f0074806a1b56a31c303dd69 [id=6553759cad994e019ddc88e3915cecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.755489 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 72b2ebaee8d84fb1aae11d3160ad712b (DEFAULT_TABLE table=default.Table_827736b8218e48e086bf05354914d450 [id=a6444c78db944c00bf604e3e662c5cb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.755573 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757381 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 9b931eca7ba04c11af6571cb418f882e (DEFAULT_TABLE table=default.Table_ebca8dd587b144d7909761c052084855 [id=d51a676c300d49678703b388baddb9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757511 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 72b2ebaee8d84fb1aae11d3160ad712b (DEFAULT_TABLE table=default.Table_827736b8218e48e086bf05354914d450 [id=a6444c78db944c00bf604e3e662c5cb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757510 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 75af8f21dcc7443fabd13e06061d19a6 (DEFAULT_TABLE table=default.Table_949ccbca1e8b44b79578377f67f2422c [id=2764171b8d1a4494bd0d30ecfd267a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757949 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0c50ef3021404d30a09396237ea30627 (DEFAULT_TABLE table=default.Table_80a7a76a26ab4fc9919bfef0a312f217 [id=2764e110843341bc99f1898c1445eeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758002 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b2ebaee8d84fb1aae11d3160ad712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.757921 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 862a3101a9b8498693851d4034826ca3 (DEFAULT_TABLE table=default.Table_b2a92f77f0074806a1b56a31c303dd69 [id=6553759cad994e019ddc88e3915cecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758108 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.758276 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862a3101a9b8498693851d4034826ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.758566 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 62c7286685244f558bfa80a9fc8abaf4 (DEFAULT_TABLE table=default.Table_3fbd6111333f4635a274f38a20902738 [id=068eb6b595f144789e9ad90c9a8f1bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758940 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c7286685244f558bfa80a9fc8abaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.759294 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:31.762616 3678 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.372s (SENT)
I20260811 18:57:31.762617 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.763017 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60091bd891a94cc4aac6ef47673a4ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.766588 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9b931eca7ba04c11af6571cb418f882e (DEFAULT_TABLE table=default.Table_ebca8dd587b144d7909761c052084855 [id=d51a676c300d49678703b388baddb9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.766958 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b931eca7ba04c11af6571cb418f882e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.757565 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 0c50ef3021404d30a09396237ea30627 (DEFAULT_TABLE table=default.Table_80a7a76a26ab4fc9919bfef0a312f217 [id=2764e110843341bc99f1898c1445eeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.770825 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c50ef3021404d30a09396237ea30627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774573 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862a3101a9b8498693851d4034826ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774796 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75af8f21dcc7443fabd13e06061d19a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774940 5091 raft_consensus.cc:482] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.775076 5091 raft_consensus.cc:504] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.775674 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b2ebaee8d84fb1aae11d3160ad712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.775789 5091 leader_election.cc:288] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.776207 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c50ef3021404d30a09396237ea30627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.778646 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b931eca7ba04c11af6571cb418f882e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.778623 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c07150e31994bdf9a00ac2e7a954691" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:31.779543 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60091bd891a94cc4aac6ef47673a4ee5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.782552 3676 leader_election.cc:341] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:31.783285 3678 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.544s (SENT)
I20260811 18:57:31.783406 3678 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:31.783707 3678 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.141s (SENT)
I20260811 18:57:31.784173 5091 raft_consensus.cc:2726] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:31.785662 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.791925 3824 consensus_peers.cc:487] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0b199f4c9e14b468cb26cbd61fc8586. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:31.771768 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 701) took 4709 ms. Trace:
I20260811 18:57:31.792588 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdab1c3c431c4dceb08b00eabcfa1ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.796331 4162 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.793063 4167 rpcz_store.cc:276] 0811 18:57:27.061614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.061700 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:31.674592 (+4612892us) ts_tablet_manager.cc:1145] Deleting tablet 37be4f7579fa4ab99ddbacfa80282b42
0811 18:57:31.674607 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.771201 (+ 96594us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4612505}
W20260811 18:57:31.797776 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 37be4f7579fa4ab99ddbacfa80282b42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37be4f7579fa4ab99ddbacfa80282b42 already in progress: creating tablet
I20260811 18:57:31.798548 3548 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c] alter complete (version 2)
I20260811 18:57:31.799486 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b702aef88e124443888002d8ccd47b2f
I20260811 18:57:31.800184 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet c21eb02db3de4c97878fa8badccaa6e3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.800905 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f1d0ca0ad934be687840ccd25db713a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.801563 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f1d0ca0ad934be687840ccd25db713a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.802507 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825b6743798144deb24330304d6b5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.802644 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1800e9e2b064a6fb546b9d33cefa68b
I20260811 18:57:31.803333 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.803956 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b30b503bf0a841d6a0061c6430b70b1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.805356 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f2f31331974f42c3a22b3ac21b690f82
I20260811 18:57:31.806201 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4e3a3cb2f24408ba3e2e8510459eee4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.806691 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 703) took 4738 ms. Trace:
I20260811 18:57:31.806787 4167 rpcz_store.cc:276] 0811 18:57:27.067918 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.068068 (+ 150us) service_pool.cc:227] Handling call
0811 18:57:31.806515 (+4738447us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:31.806530 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.806667 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4738188}
W20260811 18:57:31.806931 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.807224 4162 ts_tablet_manager.cc:1397] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.000s
I20260811 18:57:31.807977 4162 raft_consensus.cc:348] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.808187 4162 raft_consensus.cc:374] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.808233 4162 raft_consensus.cc:729] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.809250 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91bea743ec0b491299f45f3ac23cb734"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.809819 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91bea743ec0b491299f45f3ac23cb734 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.810379 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 704) took 4742 ms. Trace:
I20260811 18:57:31.812657 4167 rpcz_store.cc:276] 0811 18:57:27.068024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.068374 (+ 350us) service_pool.cc:227] Handling call
0811 18:57:31.809711 (+4741337us) ts_tablet_manager.cc:1145] Deleting tablet 9722572533de436cb9149e6230570382
0811 18:57:31.809726 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.810263 (+ 537us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4741083}
I20260811 18:57:31.815263 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f31331974f42c3a22b3ac21b690f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.818022 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_efc46b70936f4bd292f775b723c1fbff [id=0fee6d656a1b486cb1fb77abf4042dd8] alter complete (version 2)
I20260811 18:57:31.819098 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_badc11f1b0554e629ddb3f8a20e151e2" } new_table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" modify_external_catalogs: true
W20260811 18:57:31.819787 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7251204325dd424192a6f58e18e71c34 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7251204325dd424192a6f58e18e71c34 already in progress: creating tablet
W20260811 18:57:31.820138 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9722572533de436cb9149e6230570382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9722572533de436cb9149e6230570382 already in progress: creating tablet
I20260811 18:57:31.821632 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.822258 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.823033 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.823336 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.823546 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.823791 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.821906 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet b702aef88e124443888002d8ccd47b2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.824244 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f31331974f42c3a22b3ac21b690f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.824506 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.824612 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b702aef88e124443888002d8ccd47b2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.813650 4162 consensus_queue.cc:260] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.825488 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.827095 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5 (DEFAULT_TABLE table=default.Table_5b0e7b307dc84d88aff7e744fe4ed97b [id=cc52e710b7b64b74aca455760f151a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.827541 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.829046 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b426bc8778444ab3959ce53532b50959 (DEFAULT_TABLE table=default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b [id=868ded1d55df48ebb3116ca5e939599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.829486 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b426bc8778444ab3959ce53532b50959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.829640 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5 (DEFAULT_TABLE table=default.Table_5b0e7b307dc84d88aff7e744fe4ed97b [id=cc52e710b7b64b74aca455760f151a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.831187 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc967770de4ed486b638fb6b174db8 (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.831728 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.833710 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 35b3ddaf14db47438ec396689cdb0d41 (DEFAULT_TABLE table=default.Table_634c196a72864956aedd6ceb1757173a [id=35751da09b674aebb70ab932d6f9dfcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.834079 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b3ddaf14db47438ec396689cdb0d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.834282 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc967770de4ed486b638fb6b174db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.836226 3717 tablet_service.cc:1467] Processing CreateTablet for tablet ee5b7d1e32514ad48177371582e85241 (DEFAULT_TABLE table=default.Table_fd5ee099cbc94fd285f1b71eea0cde80 [id=cffd7424406a4d50be8d46cf45d6e6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.836318 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 112de1a11805406b9183100c6c66f720 (DEFAULT_TABLE table=default.Table_93b2871d4c454120a6b609ae91e096e6 [id=7873608f194c42bc9b11d26cca8dafc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.836575 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5b7d1e32514ad48177371582e85241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.836740 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112de1a11805406b9183100c6c66f720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.838145 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 112de1a11805406b9183100c6c66f720 (DEFAULT_TABLE table=default.Table_93b2871d4c454120a6b609ae91e096e6 [id=7873608f194c42bc9b11d26cca8dafc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.838616 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112de1a11805406b9183100c6c66f720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.839881 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ee5b7d1e32514ad48177371582e85241 (DEFAULT_TABLE table=default.Table_fd5ee099cbc94fd285f1b71eea0cde80 [id=cffd7424406a4d50be8d46cf45d6e6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.840229 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5b7d1e32514ad48177371582e85241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.842634 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 35b3ddaf14db47438ec396689cdb0d41 (DEFAULT_TABLE table=default.Table_634c196a72864956aedd6ceb1757173a [id=35751da09b674aebb70ab932d6f9dfcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.843027 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b3ddaf14db47438ec396689cdb0d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.846097 3872 tablet_service.cc:1467] Processing CreateTablet for tablet b426bc8778444ab3959ce53532b50959 (DEFAULT_TABLE table=default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b [id=868ded1d55df48ebb3116ca5e939599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.846508 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b426bc8778444ab3959ce53532b50959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.847090 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc967770de4ed486b638fb6b174db8 (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.847431 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc967770de4ed486b638fb6b174db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.826154 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 705) took 4754 ms. Trace:
I20260811 18:57:31.857717 4167 rpcz_store.cc:276] 0811 18:57:27.071363 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.071415 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:31.825675 (+4754260us) ts_tablet_manager.cc:1145] Deleting tablet 28af2d2c61254b0ca399dddb05e9ee70
0811 18:57:31.825696 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.826105 (+ 409us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4745081}
I20260811 18:57:31.826858 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 62c7286685244f558bfa80a9fc8abaf4 (DEFAULT_TABLE table=default.Table_3fbd6111333f4635a274f38a20902738 [id=068eb6b595f144789e9ad90c9a8f1bcd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:31.859858 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 28af2d2c61254b0ca399dddb05e9ee70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28af2d2c61254b0ca399dddb05e9ee70 already in progress: creating tablet
I20260811 18:57:31.860560 4162 ts_tablet_manager.cc:1428] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.053s user 0.001s sys 0.001s
I20260811 18:57:31.860817 4163 tablet_bootstrap.cc:654] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:31.888747 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 09366eccf38a4876840cc5ddddbbb103 (table default.Table_1314176a6f3346d7a808360af2c1cfd2 [id=307c31b5346949adb9b0c0db5da22ea0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:31.889093 3855 tablet_service.cc:1467] Processing CreateTablet for tablet bf1c037a37504b3fad9612f5a734dd33 (DEFAULT_TABLE table=default.Table_6f13efa6fe524f5bbd7154aa2f991d38 [id=1387459576db42fa870b89b1dab61400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.889954 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a05b9d2817ff41d7a1d516f34dad88d2 (DEFAULT_TABLE table=default.Table_2076434270fa4c9c99b4d12f1aa924c9 [id=173131c0dc16407d8634e08b2580c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.890391 3710 tablet_service.cc:1467] Processing CreateTablet for tablet a05b9d2817ff41d7a1d516f34dad88d2 (DEFAULT_TABLE table=default.Table_2076434270fa4c9c99b4d12f1aa924c9 [id=173131c0dc16407d8634e08b2580c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.890393 3709 tablet_service.cc:1467] Processing CreateTablet for tablet bf1c037a37504b3fad9612f5a734dd33 (DEFAULT_TABLE table=default.Table_6f13efa6fe524f5bbd7154aa2f991d38 [id=1387459576db42fa870b89b1dab61400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.894011 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c7286685244f558bfa80a9fc8abaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.898371 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 702) took 4830 ms. Trace:
I20260811 18:57:31.898579 4167 rpcz_store.cc:276] 0811 18:57:27.067773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.071735 (+ 3962us) service_pool.cc:227] Handling call
0811 18:57:31.860257 (+4788522us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:31.898116 (+ 37859us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.898314 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4788270}
I20260811 18:57:31.899420 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 712) took 4678 ms. Trace:
I20260811 18:57:31.899595 4167 rpcz_store.cc:276] 0811 18:57:27.220742 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.226578 (+ 5836us) service_pool.cc:227] Handling call
0811 18:57:31.899292 (+4672714us) ts_tablet_manager.cc:1145] Deleting tablet 90576c068ce4468a9c572307dcb79b10
0811 18:57:31.899305 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.899401 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4668816}
I20260811 18:57:31.900179 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 717) took 4377 ms. Trace:
I20260811 18:57:31.900321 4167 rpcz_store.cc:276] 0811 18:57:27.523107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.524301 (+ 1194us) service_pool.cc:227] Handling call
0811 18:57:31.900040 (+4375739us) ts_tablet_manager.cc:1145] Deleting tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
0811 18:57:31.900055 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.900165 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4375426}
I20260811 18:57:31.900887 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 718) took 4362 ms. Trace:
I20260811 18:57:31.901052 4167 rpcz_store.cc:276] 0811 18:57:27.538044 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.538125 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:31.900736 (+4362611us) ts_tablet_manager.cc:1145] Deleting tablet a49b8b344177487fb5bd096ee9cdea2f
0811 18:57:31.900748 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.900867 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4362188}
I20260811 18:57:31.901641 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 719) took 4352 ms. Trace:
I20260811 18:57:31.901800 4167 rpcz_store.cc:276] 0811 18:57:27.548798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.549039 (+ 241us) service_pool.cc:227] Handling call
0811 18:57:31.901522 (+4352483us) ts_tablet_manager.cc:1145] Deleting tablet c21eb02db3de4c97878fa8badccaa6e3
0811 18:57:31.901535 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.901623 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4352125}
I20260811 18:57:31.902319 4162 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:31.903834 3824 consensus_peers.cc:487] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4687b95728fc485fa0e842bc447eb83f. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:31.904979 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05b9d2817ff41d7a1d516f34dad88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.905093 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf1c037a37504b3fad9612f5a734dd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.905405 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05b9d2817ff41d7a1d516f34dad88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.906742 4162 tablet_bootstrap.cc:654] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.907856 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 720) took 4358 ms. Trace:
I20260811 18:57:31.907939 4167 rpcz_store.cc:276] 0811 18:57:27.548968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.549459 (+ 491us) service_pool.cc:227] Handling call
0811 18:57:31.907738 (+4358279us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:31.907750 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.907838 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4358040}
I20260811 18:57:31.908380 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 731) took 4263 ms. Trace:
I20260811 18:57:31.908452 4167 rpcz_store.cc:276] 0811 18:57:27.644992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.645061 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:31.908289 (+4263228us) ts_tablet_manager.cc:1145] Deleting tablet f287040b879642448367abe2b1c73cd9
0811 18:57:31.908298 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.908367 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4262970}
I20260811 18:57:31.909049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 732) took 4263 ms. Trace:
I20260811 18:57:31.909211 4167 rpcz_store.cc:276] 0811 18:57:27.645516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.645565 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:31.908848 (+4263283us) ts_tablet_manager.cc:1145] Deleting tablet 2be6762e326448d7b677a5641e2b3103
0811 18:57:31.908890 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.909032 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4263112}
I20260811 18:57:31.909499 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf1c037a37504b3fad9612f5a734dd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.909731 4167 tablet_replica.cc:330] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.910063 4167 raft_consensus.cc:2229] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.910419 4167 raft_consensus.cc:2258] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.917747 4163 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.917999 4163 ts_tablet_manager.cc:1397] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.239s user 0.003s sys 0.002s
W20260811 18:57:31.919718 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0aee6beb2e70421083841520733087f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aee6beb2e70421083841520733087f8 already in progress: creating tablet
W20260811 18:57:31.920100 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 90576c068ce4468a9c572307dcb79b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90576c068ce4468a9c572307dcb79b10 already in progress: creating tablet
W20260811 18:57:31.920383 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 already in progress: creating tablet
W20260811 18:57:31.920611 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a49b8b344177487fb5bd096ee9cdea2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49b8b344177487fb5bd096ee9cdea2f already in progress: creating tablet
W20260811 18:57:31.921144 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c21eb02db3de4c97878fa8badccaa6e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c21eb02db3de4c97878fa8badccaa6e3 already in progress: creating tablet
W20260811 18:57:31.921414 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0cccebfc54484a9b9abffcce4e459fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cccebfc54484a9b9abffcce4e459fb5 already in progress: creating tablet
I20260811 18:57:31.921605 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a8483f83b5cd4a65a0fe9097c2c80cb3 (DEFAULT_TABLE table=default.Table_aadfbb276f80467d884601ea6c92a401 [id=d8682a306ecd4a9f83cd49f3c358a536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.922295 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8483f83b5cd4a65a0fe9097c2c80cb3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.934931 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f287040b879642448367abe2b1c73cd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f287040b879642448367abe2b1c73cd9 already in progress: creating tablet
W20260811 18:57:31.935400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2be6762e326448d7b677a5641e2b3103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be6762e326448d7b677a5641e2b3103 already in progress: creating tablet
I20260811 18:57:31.939574 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 97d5c52e361c4f08904475acf62dae2b (DEFAULT_TABLE table=default.Table_5727f2e5abe54ecaaeec9a52f1bd78da [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.940739 3712 tablet_service.cc:1467] Processing CreateTablet for tablet a8483f83b5cd4a65a0fe9097c2c80cb3 (DEFAULT_TABLE table=default.Table_aadfbb276f80467d884601ea6c92a401 [id=d8682a306ecd4a9f83cd49f3c358a536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.941694 3705 tablet_service.cc:1467] Processing CreateTablet for tablet cd04ae1903034a97a0bd086c1bd39a7d (DEFAULT_TABLE table=default.Table_6204a77bcccf427db16e32981a5621fb [id=0df5378171774f75b312a9205dae21dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.943202 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f89eebbc40234794a081734f834294ab (DEFAULT_TABLE table=default.Table_24c4140a00244c8d8446d91705a8d4a8 [id=d2508b5d391a441197b92e7f71fde5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.943466 4163 raft_consensus.cc:348] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.943815 4163 raft_consensus.cc:374] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.943970 4163 raft_consensus.cc:729] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.944384 4163 consensus_queue.cc:260] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.949278 4162 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.949434 4162 ts_tablet_manager.cc:1397] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.001s
I20260811 18:57:31.954689 3855 tablet_service.cc:1467] Processing CreateTablet for tablet cd04ae1903034a97a0bd086c1bd39a7d (DEFAULT_TABLE table=default.Table_6204a77bcccf427db16e32981a5621fb [id=0df5378171774f75b312a9205dae21dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.955240 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd04ae1903034a97a0bd086c1bd39a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.957549 4162 raft_consensus.cc:348] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.957803 4162 raft_consensus.cc:374] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.957871 4162 raft_consensus.cc:729] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.958112 4162 consensus_queue.cc:260] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.960970 4169 ts_tablet_manager.cc:1918] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.961159 4169 log.cc:1196] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:31.967326 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8483f83b5cd4a65a0fe9097c2c80cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.968508 4169 ts_tablet_manager.cc:1939] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.970597 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd04ae1903034a97a0bd086c1bd39a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.975135 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89eebbc40234794a081734f834294ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.981935 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 722) took 4195 ms. Trace:
I20260811 18:57:31.982162 4169 rpcz_store.cc:276] 0811 18:57:27.786745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.786811 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:31.696655 (+3909844us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:31.696668 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.915315 (+218647us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.960855 (+ 45540us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.981899 (+ 21044us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3908775}
I20260811 18:57:31.992046 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b6f36ab6bb774257b5c2692c64eff63d (table default.Table_e0c0cbe2c2f846c2b85cda84670f448c [id=22529e4acc82410a93bca07dfda7d1b9]) successfully deleted
I20260811 18:57:31.993135 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 97d5c52e361c4f08904475acf62dae2b (DEFAULT_TABLE table=default.Table_5727f2e5abe54ecaaeec9a52f1bd78da [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.993741 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d5c52e361c4f08904475acf62dae2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.995333 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:31.997381 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d5c52e361c4f08904475acf62dae2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.992422 4169 tablet_replica.cc:330] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.998664 4169 raft_consensus.cc:2229] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.998886 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f89eebbc40234794a081734f834294ab (DEFAULT_TABLE table=default.Table_24c4140a00244c8d8446d91705a8d4a8 [id=d2508b5d391a441197b92e7f71fde5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.999050 4169 raft_consensus.cc:2258] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.999437 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89eebbc40234794a081734f834294ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.000576 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2e58ff1b0c44ee8b71812ae6399b4da" } new_table_name: "default.Table_738fb827c1e14e659c918e66b8f3d079" modify_external_catalogs: true
I20260811 18:57:32.001914 4169 ts_tablet_manager.cc:1905] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.016832 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.709s (SENT) [suppressed 27 similar messages]
I20260811 18:57:32.024920 4167 ts_tablet_manager.cc:1905] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.027009 4169 ts_tablet_manager.cc:1918] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.028321 4169 log.cc:1196] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:32.028990 4169 ts_tablet_manager.cc:1939] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.029863 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 724) took 4221 ms. Trace:
I20260811 18:57:32.030692 4163 ts_tablet_manager.cc:1428] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.113s user 0.002s sys 0.000s
I20260811 18:57:31.959146 4162 ts_tablet_manager.cc:1428] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:32.030527 4169 rpcz_store.cc:276] 0811 18:57:27.808665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.808732 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:31.982851 (+4174119us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:31.992365 (+ 9514us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.005773 (+ 13408us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.026885 (+ 21112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.029842 (+ 2957us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4173826}
I20260811 18:57:32.037240 4162 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.038846 4162 tablet_bootstrap.cc:654] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.040110 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d2d7d986900244ddbc05dbe73a708dcf (table default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]) successfully deleted
I20260811 18:57:32.040215 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 723) took 4231 ms. Trace:
I20260811 18:57:32.040462 4169 rpcz_store.cc:276] 0811 18:57:27.808526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.818592 (+ 10066us) service_pool.cc:227] Handling call
0811 18:57:32.040032 (+4221440us) ts_tablet_manager.cc:1145] Deleting tablet 4093564a88264ed59c945523c9ddc9b3
0811 18:57:32.040046 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.040192 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4204355}
W20260811 18:57:32.041226 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4093564a88264ed59c945523c9ddc9b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4093564a88264ed59c945523c9ddc9b3 already in progress: creating tablet
I20260811 18:57:32.041225 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 725) took 3902 ms. Trace:
I20260811 18:57:32.041520 4169 rpcz_store.cc:276] 0811 18:57:28.139272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.139486 (+ 214us) service_pool.cc:227] Handling call
0811 18:57:32.041073 (+3901587us) ts_tablet_manager.cc:1145] Deleting tablet ae7da0868b114052ba421801b76fb33c
0811 18:57:32.041084 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.041197 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3901325}
I20260811 18:57:32.042464 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 726) took 3903 ms. Trace:
W20260811 18:57:32.042409 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ae7da0868b114052ba421801b76fb33c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae7da0868b114052ba421801b76fb33c already in progress: creating tablet
I20260811 18:57:32.042656 4169 rpcz_store.cc:276] 0811 18:57:28.139406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.139804 (+ 398us) service_pool.cc:227] Handling call
0811 18:57:32.042143 (+3902339us) ts_tablet_manager.cc:1145] Deleting tablet 6d988ef7e9dd43158d107998ea4d82b9
0811 18:57:32.042153 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.042445 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3902189}
W20260811 18:57:32.043213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6d988ef7e9dd43158d107998ea4d82b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d988ef7e9dd43158d107998ea4d82b9 already in progress: creating tablet
I20260811 18:57:32.043426 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 727) took 3898 ms. Trace:
I20260811 18:57:32.043548 4169 rpcz_store.cc:276] 0811 18:57:28.145398 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.145455 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:32.043132 (+3897677us) ts_tablet_manager.cc:1145] Deleting tablet b6c70b4163634984bd02dd9b47ffb472
0811 18:57:32.043144 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.043404 (+ 260us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3897473}
W20260811 18:57:32.044332 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b6c70b4163634984bd02dd9b47ffb472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6c70b4163634984bd02dd9b47ffb472 already in progress: creating tablet
I20260811 18:57:32.036933 4163 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.047806 4163 tablet_bootstrap.cc:654] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.044443 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 728) took 3872 ms. Trace:
I20260811 18:57:32.050469 4169 rpcz_store.cc:276] 0811 18:57:28.172188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.172328 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:32.044149 (+3871821us) ts_tablet_manager.cc:1145] Deleting tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
0811 18:57:32.044175 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.044409 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3871469}
I20260811 18:57:32.051134 4169 tablet_replica.cc:330] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:32.051225 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca already in progress: creating tablet
I20260811 18:57:32.051541 4169 raft_consensus.cc:2229] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:32.052237 3798 tablet.cc:2369] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:32.052008 4169 raft_consensus.cc:2258] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.054068 4169 ts_tablet_manager.cc:1905] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.063015 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46196.1us
I20260811 18:57:32.063495 4162 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.063661 4162 ts_tablet_manager.cc:1397] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
W20260811 18:57:32.067624 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 15dd14e2b4bd4a08b5b08f532a55291b has no consensus state
W20260811 18:57:32.068348 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4af28c051e8243eaa73d4a5ed1abcea6 consensus state has no leader
I20260811 18:57:32.069221 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4133a1bbf6474492bbf3683ca9608a52" } new_table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" modify_external_catalogs: true
I20260811 18:57:32.069846 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0da5d975ff884b9bb82d239828666af7" } modify_external_catalogs: true
I20260811 18:57:32.070672 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" } modify_external_catalogs: true
I20260811 18:57:32.071094 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" } modify_external_catalogs: true
I20260811 18:57:32.072196 4163 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.064529 4162 raft_consensus.cc:348] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.073112 4163 ts_tablet_manager.cc:1397] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.007s
I20260811 18:57:32.073381 4162 raft_consensus.cc:374] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.074362 4163 raft_consensus.cc:348] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.074759 4163 raft_consensus.cc:374] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.074913 4163 raft_consensus.cc:729] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.075263 4163 consensus_queue.cc:260] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.075553 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" } modify_external_catalogs: true
I20260811 18:57:32.075940 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" } modify_external_catalogs: true
I20260811 18:57:32.078470 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" } modify_external_catalogs: true
I20260811 18:57:32.078768 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" } modify_external_catalogs: true
I20260811 18:57:32.079259 4163 ts_tablet_manager.cc:1428] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:57:32.079627 4163 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.074290 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_73bf9f47862a4f03ac5e35afacfc2463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.083878 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73bf9f47862a4f03ac5e35afacfc2463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.082295 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0da5d975ff884b9bb82d239828666af7" } modify_external_catalogs: true
I20260811 18:57:32.068348 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c7a814de93aa4d49a3ab5c7d2b167976" } new_table_name: "default.Table_0edbfd08288d4926a353671dfbf84fd5" modify_external_catalogs: true
I20260811 18:57:32.091142 4163 tablet_bootstrap.cc:654] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.098786 4162 raft_consensus.cc:729] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.099253 4162 consensus_queue.cc:260] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.100858 3942 tablet.cc:2369] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:32.106451 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fdab1c3c431c4dceb08b00eabcfa1ceb consensus state has no leader
I20260811 18:57:32.104300 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b752cb750e9447b092ecac296b5f836e" } modify_external_catalogs: true
I20260811 18:57:32.107519 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b752cb750e9447b092ecac296b5f836e" } modify_external_catalogs: true
#8 0xae894d in kudu::tools::(anonymous namespace)::TServerRun(kudu::tools::RunnerContext const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action_tserver.cc:114:10
I20260811 18:57:32.110563 4162 ts_tablet_manager.cc:1428] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260811 18:57:32.114296 4167 ts_tablet_manager.cc:1918] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.114478 4167 log.cc:1196] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:32.114953 4167 ts_tablet_manager.cc:1939] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.116884 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.117386 4162 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.119105 4162 tablet_bootstrap.cc:654] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.120949 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 734) took 4466 ms. Trace:
I20260811 18:57:32.121068 4167 rpcz_store.cc:276] 0811 18:57:27.654614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.654677 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:31.909664 (+4254987us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:31.909674 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.029235 (+119561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.090342 (+ 61107us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.120921 (+ 30579us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4254722}
I20260811 18:57:32.121973 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cbf6ce167fb348be8d61372791f0cbca (table default.Table_1558f156b5eb46a1858ca7673e4a9747 [id=dda7fb95dc1549ddbc88d8525ff023e8]) successfully deleted
I20260811 18:57:32.121510 4167 tablet_replica.cc:330] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.122471 4167 raft_consensus.cc:2229] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.122679 4167 raft_consensus.cc:2258] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.124862 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1dc240fd92f941489de07a83cdafb797"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.121585 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.125769 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1dc240fd92f941489de07a83cdafb797 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.128226 4167 ts_tablet_manager.cc:1905] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.131543 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:32.152248 4162 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.152463 4162 ts_tablet_manager.cc:1397] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.003s
I20260811 18:57:32.153865 4162 raft_consensus.cc:348] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.154158 4162 raft_consensus.cc:374] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.154235 4162 raft_consensus.cc:729] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.154752 4162 consensus_queue.cc:260] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.161288 4162 ts_tablet_manager.cc:1428] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260811 18:57:32.152851 5376 raft_consensus.cc:482] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.171770 4162 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.176730 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cdb92902c2584751af77d3117f52cb4e
I20260811 18:57:32.177019 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ccc63b46a4e346f79d1e03a82490794c
I20260811 18:57:32.177749 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet e1ad3860d03f46baadd5ede854f1a909 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:32.181684 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc63b46a4e346f79d1e03a82490794c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.181977 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb92902c2584751af77d3117f52cb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.185613 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1bfaa8ab4fcd40efb60e1718e424bccf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.189450 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.191916 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.193615 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 99de8ce45e514ff291cb9e5c0a4c00d1
I20260811 18:57:32.194037 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 14799a6f61a946008acee8a09d050c1e
W20260811 18:57:32.194947 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f06e5b6a30764caa978ba0022ca1d6bc consensus state has no leader
I20260811 18:57:32.195636 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 55f97f37fa7c483cacaeed54f93e7a24
W20260811 18:57:32.196542 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5a184b83373a4c089f2ca0169cbfcf66 consensus state has no leader
I20260811 18:57:32.197496 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.197773 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 14799a6f61a946008acee8a09d050c1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.198587 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.199213 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bfaa8ab4fcd40efb60e1718e424bccf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.168560 5376 raft_consensus.cc:504] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.209338 5376 leader_election.cc:288] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.181675 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb92902c2584751af77d3117f52cb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.212792 4162 tablet_bootstrap.cc:654] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.215606 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc63b46a4e346f79d1e03a82490794c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.217167 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 14799a6f61a946008acee8a09d050c1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.219563 4169 ts_tablet_manager.cc:1918] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.219738 4169 log.cc:1196] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:32.220103 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.226568 4169 ts_tablet_manager.cc:1939] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.227124 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 734) took 3784 ms. Trace:
I20260811 18:57:32.227315 4169 rpcz_store.cc:276] 0811 18:57:28.442482 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.443442 (+ 960us) service_pool.cc:227] Handling call
0811 18:57:32.051063 (+3607621us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:32.051077 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.068521 (+ 17444us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.219427 (+150906us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.227096 (+ 7669us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3607381}
I20260811 18:57:32.228842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 735) took 3784 ms. Trace:
I20260811 18:57:32.229036 4169 rpcz_store.cc:276] 0811 18:57:28.444764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.444818 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:32.227792 (+3782974us) ts_tablet_manager.cc:1145] Deleting tablet d8146f47d9ae48889ac7efc954d4a6b7
0811 18:57:32.227804 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.228817 (+ 1013us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3782788}
I20260811 18:57:32.229604 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 736) took 3738 ms. Trace:
I20260811 18:57:32.229763 4169 rpcz_store.cc:276] 0811 18:57:28.491405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.491516 (+ 111us) service_pool.cc:227] Handling call
0811 18:57:32.229427 (+3737911us) ts_tablet_manager.cc:1145] Deleting tablet a38c172a9fef4a3ab8ee188335a7c97e
0811 18:57:32.229441 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.229574 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3737631}
I20260811 18:57:32.230288 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 737) took 3737 ms. Trace:
I20260811 18:57:32.230496 4169 rpcz_store.cc:276] 0811 18:57:28.493048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.493229 (+ 181us) service_pool.cc:227] Handling call
0811 18:57:32.230160 (+3736931us) ts_tablet_manager.cc:1145] Deleting tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
0811 18:57:32.230174 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.230267 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3736695}
I20260811 18:57:32.230860 5091 raft_consensus.cc:482] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.230973 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 738) took 3723 ms. Trace:
I20260811 18:57:32.231081 4169 rpcz_store.cc:276] 0811 18:57:28.507960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.508068 (+ 108us) service_pool.cc:227] Handling call
0811 18:57:32.230844 (+3722776us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:32.230858 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.230955 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3722533}
I20260811 18:57:32.231057 5091 raft_consensus.cc:504] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.231642 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 741) took 3604 ms. Trace:
I20260811 18:57:32.231763 4169 rpcz_store.cc:276] 0811 18:57:28.627533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.630131 (+ 2598us) service_pool.cc:227] Handling call
0811 18:57:32.231483 (+3601352us) ts_tablet_manager.cc:1145] Deleting tablet f13a14bd64704798bf6de4fc05201a31
0811 18:57:32.231497 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.231624 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3601036}
I20260811 18:57:32.232123 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 751) took 3514 ms. Trace:
I20260811 18:57:32.232293 5091 leader_election.cc:288] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.232211 4169 rpcz_store.cc:276] 0811 18:57:28.717828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.719682 (+ 1854us) service_pool.cc:227] Handling call
0811 18:57:32.231983 (+3512301us) ts_tablet_manager.cc:1145] Deleting tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
0811 18:57:32.231995 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.232107 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3512046}
I20260811 18:57:32.232569 4169 tablet_replica.cc:330] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.233268 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d4c26da86b49b8a37f2289f4ef6614" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.234688 3676 leader_election.cc:341] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:32.235952 3824 leader_election.cc:341] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c08436f33db043b1bc6395cd92102e59
I20260811 18:57:32.236225 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.236794 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 799ba9a044e145ec89d0025c03f391ea (table default.Table_56b89b2b2c4b45768b2866c74d9f1390 [id=7736472828a3499c8ad8026e164b9759]) successfully deleted
W20260811 18:57:32.237213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d8146f47d9ae48889ac7efc954d4a6b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8146f47d9ae48889ac7efc954d4a6b7 already in progress: creating tablet
W20260811 18:57:32.237537 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a38c172a9fef4a3ab8ee188335a7c97e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38c172a9fef4a3ab8ee188335a7c97e already in progress: creating tablet
W20260811 18:57:32.237823 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c already in progress: creating tablet
W20260811 18:57:32.238157 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cf6c8d542ae2489ba466ffadd518db12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf6c8d542ae2489ba466ffadd518db12 already in progress: creating tablet
W20260811 18:57:32.238791 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f13a14bd64704798bf6de4fc05201a31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13a14bd64704798bf6de4fc05201a31 already in progress: creating tablet
W20260811 18:57:32.239143 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 already in progress: creating tablet
I20260811 18:57:32.244575 4167 ts_tablet_manager.cc:1918] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.245649 4167 log.cc:1196] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f335e50ae57547ed80b144129faaffbb
I20260811 18:57:32.247768 4167 ts_tablet_manager.cc:1939] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.233352 4169 raft_consensus.cc:2229] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.251678 4169 raft_consensus.cc:2258] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.252516 4169 ts_tablet_manager.cc:1905] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.252257 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 735) took 4597 ms. Trace:
I20260811 18:57:32.256919 4162 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.257094 4162 ts_tablet_manager.cc:1397] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.086s user 0.006s sys 0.001s
I20260811 18:57:32.258458 4162 raft_consensus.cc:348] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:32.258903 4162 raft_consensus.cc:374] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.259033 4162 raft_consensus.cc:729] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.259500 4162 consensus_queue.cc:260] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:32.262276 3678 leader_election.cc:334] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.166s (SENT)
I20260811 18:57:32.264660 3678 leader_election.cc:302] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:32.264948 3678 raft_consensus.cc:2670] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.253592 4167 rpcz_store.cc:276] 0811 18:57:27.654741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.654792 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.121459 (+4466667us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:32.121473 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.190206 (+ 68733us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.244040 (+ 53834us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.251918 (+ 7878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4466482}
I20260811 18:57:32.268556 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f335e50ae57547ed80b144129faaffbb (table default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]) successfully deleted
I20260811 18:57:32.270200 4163 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.270673 4163 ts_tablet_manager.cc:1397] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.191s user 0.004s sys 0.011s
I20260811 18:57:32.271730 4163 raft_consensus.cc:348] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.271546 4167 tablet_replica.cc:330] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.272151 4163 raft_consensus.cc:374] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.272298 4163 raft_consensus.cc:729] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.272670 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e1ad3860d03f46baadd5ede854f1a909
I20260811 18:57:32.272711 4163 consensus_queue.cc:260] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.274298 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ad3860d03f46baadd5ede854f1a909 with delete_type TABLET_DATA_DELETED (replaced by tablet 084b60eeb6b64e1da7336203d75770f4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.274324 3553 catalog_manager.cc:6565] ReplaceTablet: tablet e1ad3860d03f46baadd5ede854f1a909 deleted and replaced by tablet 084b60eeb6b64e1da7336203d75770f4
I20260811 18:57:32.277391 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:32.277190 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4e35d031a62148d582b7bdb0240f63da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.278111 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e35d031a62148d582b7bdb0240f63da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.280308 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4dc767eb645648aebd0aacf2d46f9c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.281075 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dc767eb645648aebd0aacf2d46f9c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.286652 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d584e1cc7d8b45be912d8610e391f4f7" } new_table_name: "default.Table_cf2da094466b4bf597833b58e3402047" modify_external_catalogs: true
I20260811 18:57:32.289326 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6531b518884b4b9b8ecfdc09c2a0e15f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.278646 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.278705 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ad3860d03f46baadd5ede854f1a909 with delete_type TABLET_DATA_DELETED (replaced by tablet 084b60eeb6b64e1da7336203d75770f4) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.290162 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6531b518884b4b9b8ecfdc09c2a0e15f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.272470 4162 ts_tablet_manager.cc:1428] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:32.289319 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ad9f3fa268174f9aa74bcee2360bf4d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.291059 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.294867 4167 raft_consensus.cc:2229] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.298116 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f980c9b98994253b590548d86f6d145" } new_table_name: "default.Table_022048402d3f4ad2906b4c19cb3882bb" modify_external_catalogs: true
I20260811 18:57:32.297667 4162 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.297797 4167 raft_consensus.cc:2258] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.302763 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3309d84857df4970a9891fd64c181bf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.303686 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3309d84857df4970a9891fd64c181bf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.304538 4167 ts_tablet_manager.cc:1905] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#9 0x921350 in std::_Function_handler<kudu::Status (kudu::tools::RunnerContext const&), kudu::Status (*)(kudu::tools::RunnerContext const&)>::_M_invoke(std::_Any_data const&, kudu::tools::RunnerContext const&) ../../../include/c++/8/bits/std_function.h:282:9
I20260811 18:57:32.313313 4163 ts_tablet_manager.cc:1428] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20260811 18:57:32.315510 4163 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.317090 4163 tablet_bootstrap.cc:654] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.320792 4162 tablet_bootstrap.cc:654] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.324882 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a09ba72daf1440ba3921b736aeb429e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.325008 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_949ccbca1e8b44b79578377f67f2422c" } new_table_name: "default.Table_292cbd0196af4a9a8164ead40c13b401" modify_external_catalogs: true
W20260811 18:57:32.325677 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a09ba72daf1440ba3921b736aeb429e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:32.327600 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae33c49dab3444988738f31180de35a consensus state has no leader
I20260811 18:57:32.330806 5091 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.331025 5091 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.331877 5091 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.332690 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.333269 3676 leader_election.cc:341] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.337491 5376 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.337652 5376 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.338476 5376 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.338979 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.339325 3740 raft_consensus.cc:2454] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.339969 3824 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.340302 5376 raft_consensus.cc:2781] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.340405 5376 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.340485 5376 raft_consensus.cc:3037] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.343870 5376 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.344681 5376 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.347019 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.347268 3740 raft_consensus.cc:3037] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.350303 3740 raft_consensus.cc:2454] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.351128 3824 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.351476 5376 raft_consensus.cc:2781] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.352133 5376 raft_consensus.cc:686] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.352972 5376 consensus_queue.cc:237] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.354789 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a255a0922a92419191f18730d8b335bc consensus state has no leader
W20260811 18:57:32.362318 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75af8f21dcc7443fabd13e06061d19a6 consensus state has no leader
I20260811 18:57:32.370150 4163 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.370313 4163 ts_tablet_manager.cc:1397] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.002s
I20260811 18:57:32.371093 4163 raft_consensus.cc:348] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.373142 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet bf2816cc2dfe45f0815ce50a5a929324 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:32.377439 5283 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.377620 5283 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.378051 4163 raft_consensus.cc:374] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.378152 4163 raft_consensus.cc:729] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.378398 4163 consensus_queue.cc:260] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.378412 5283 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.383893 4169 ts_tablet_manager.cc:1918] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.384198 4169 log.cc:1196] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/06b8f2a97c0f46ea8c56b3de2b97e60c
I20260811 18:57:32.385030 4169 ts_tablet_manager.cc:1939] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.385679 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 752) took 3595 ms. Trace:
I20260811 18:57:32.385716 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25" } modify_external_catalogs: true
I20260811 18:57:32.385871 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a13d9e49bc748e6b820a614e55ba4c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.386178 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25" } modify_external_catalogs: true
I20260811 18:57:32.385876 4169 rpcz_store.cc:276] 0811 18:57:28.789815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.790940 (+ 1125us) service_pool.cc:227] Handling call
0811 18:57:32.232510 (+3441570us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:32.232523 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.325458 (+ 92935us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.329092 (+ 3634us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.385642 (+ 56550us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3435186}
W20260811 18:57:32.386869 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a13d9e49bc748e6b820a614e55ba4c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.387269 4169 tablet_replica.cc:330] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.387172 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d" } modify_external_catalogs: true
I20260811 18:57:32.384754 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.387636 3740 raft_consensus.cc:2454] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.387687 4169 raft_consensus.cc:2229] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.388053 3824 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.388741 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (table default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]) successfully deleted
I20260811 18:57:32.391515 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d" } modify_external_catalogs: true
I20260811 18:57:32.397936 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a09ba72daf1440ba3921b736aeb429e" } modify_external_catalogs: true
I20260811 18:57:32.397914 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_57453b6f4942454da731710b8510f4a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.398371 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a09ba72daf1440ba3921b736aeb429e" } modify_external_catalogs: true
W20260811 18:57:32.399040 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57453b6f4942454da731710b8510f4a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.400795 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4" } modify_external_catalogs: true
I20260811 18:57:32.401166 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4" } modify_external_catalogs: true
I20260811 18:57:32.404904 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_07499669fc704a31830a06152cd615cf" } modify_external_catalogs: true
I20260811 18:57:32.405180 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_07499669fc704a31830a06152cd615cf" } modify_external_catalogs: true
I20260811 18:57:32.406615 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a0329aed6694593bc21571dd8f59670" } new_table_name: "default.Table_809eb8f582f14652bb5f9cd7301260c0" modify_external_catalogs: true
I20260811 18:57:32.409732 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5eb58df7cc2042f69300d10a7999719b" } new_table_name: "default.Table_8a69fc9e8b464e1cb431dcb6b30858e2" modify_external_catalogs: true
I20260811 18:57:32.419229 5283 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.419229 5286 raft_consensus.cc:2781] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.419422 5286 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.419529 5286 raft_consensus.cc:3037] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.419657 4169 pending_rounds.cc:62] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:32.419714 4169 pending_rounds.cc:68] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400161362292736 op_type: NO_OP noop_request { }
I20260811 18:57:32.419858 4169 raft_consensus.cc:2866] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:32.419888 4163 ts_tablet_manager.cc:1428] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.049s user 0.002s sys 0.000s
I20260811 18:57:32.419955 4169 raft_consensus.cc:2258] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.420182 4163 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.420704 4169 ts_tablet_manager.cc:1905] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.420742 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1a83eafb62054d789e5cf28ea8380f0a
I20260811 18:57:32.421995 4163 tablet_bootstrap.cc:654] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.419425 5283 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.423249 5283 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.424115 5286 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.426260 5286 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.426931 3551 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 79c67bc70d0d4ff892734ca757ada5a8
I20260811 18:57:32.426880 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.427114 3743 raft_consensus.cc:3037] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.427696 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.428550 3740 raft_consensus.cc:2454] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.430601 3824 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.430995 5286 raft_consensus.cc:2781] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.431152 5286 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:32.431201 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 825b6743798144deb24330304d6b5d2b consensus state has no leader
I20260811 18:57:32.431293 5286 raft_consensus.cc:3037] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.431665 3743 raft_consensus.cc:2454] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:32.431522 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0ff928a633e4cae9d49229b989216d8 consensus state has no leader
I20260811 18:57:32.432479 3824 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.432827 5283 raft_consensus.cc:2781] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.432969 5283 raft_consensus.cc:686] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.433307 5283 consensus_queue.cc:237] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.436359 5286 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.438921 5286 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.439550 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.439847 3743 raft_consensus.cc:3037] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.447130 3743 raft_consensus.cc:2454] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.448176 3824 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.448541 5286 raft_consensus.cc:2781] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.448822 5286 raft_consensus.cc:686] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.449142 5286 consensus_queue.cc:237] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.470605 5091 raft_consensus.cc:482] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.470803 5091 raft_consensus.cc:504] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.471535 5091 leader_election.cc:288] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.472286 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9bdc606be40558dfa33e1e7a8ebc7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.472883 3676 leader_election.cc:341] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.488894 4169 ts_tablet_manager.cc:1918] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.489172 4169 log.cc:1196] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:32.489727 4169 ts_tablet_manager.cc:1939] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.490387 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 753) took 3698 ms. Trace:
I20260811 18:57:32.496580 4163 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.496886 4163 ts_tablet_manager.cc:1397] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.011s sys 0.000s
I20260811 18:57:32.495286 4167 ts_tablet_manager.cc:1918] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.498026 4163 raft_consensus.cc:348] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.498785 4167 log.cc:1196] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:32.502436 4163 raft_consensus.cc:374] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.502835 4163 raft_consensus.cc:729] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.503351 4163 consensus_queue.cc:260] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.504784 3678 leader_election.cc:334] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.218s (SENT)
I20260811 18:57:32.509227 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e691dcf4a764665aa9275bfa441105a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.510080 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e691dcf4a764665aa9275bfa441105a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.512475 4163 ts_tablet_manager.cc:1428] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260811 18:57:32.512828 4163 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.522421 4163 tablet_bootstrap.cc:654] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.492487 4169 rpcz_store.cc:276] 0811 18:57:28.791606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.791662 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:32.387212 (+3595550us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:32.387228 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.447789 (+ 60561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.488771 (+ 40982us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.490359 (+ 1588us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31013,"tablet-delete.queue_time_us":3589796}
I20260811 18:57:32.523322 4169 tablet_replica.cc:330] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.523325 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ad424c8843ff434ba0449a86849bb5a0 (table default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]) successfully deleted
I20260811 18:57:32.523554 4169 raft_consensus.cc:2229] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.523767 4169 raft_consensus.cc:2258] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.524516 4169 ts_tablet_manager.cc:1905] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.532513 4163 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.533684 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7075f060dbd942cca40c448af9eb5825" } new_table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" modify_external_catalogs: true
I20260811 18:57:32.533845 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a17ac8f677ce46efaade718a851a016d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.534790 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a17ac8f677ce46efaade718a851a016d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.536583 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e97d2b678784537a3e986a742e9f88b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.535104 5376 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.537473 5376 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.537711 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.538117 5376 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.538393 4163 ts_tablet_manager.cc:1397] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
W20260811 18:57:32.538450 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:32.540445 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e97d2b678784537a3e986a742e9f88b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.542296 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.542632 3743 raft_consensus.cc:2454] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.544066 3824 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.544345 5376 raft_consensus.cc:2781] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.544452 5376 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.544533 5376 raft_consensus.cc:3037] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.548220 5376 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.548821 5376 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.548659 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a7bc3ed83682403b84a5a23d14cf1c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.549518 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7bc3ed83682403b84a5a23d14cf1c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.551380 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.551620 3743 raft_consensus.cc:3037] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.553025 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db613e17da86441bb1ab37b46e730db9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.553519 4169 ts_tablet_manager.cc:1918] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.553647 4169 log.cc:1196] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/515d5b5afbd74ec5ab90a503aef2f4fb
W20260811 18:57:32.553714 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db613e17da86441bb1ab37b46e730db9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.554085 4169 ts_tablet_manager.cc:1939] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.556671 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:32.557010 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:32.562383 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.564533 3743 raft_consensus.cc:2454] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.566319 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4e412149fc55445bb720ffed59a05618" } new_table_name: "default.Table_0f2efce46eed4e0b9626c257cbfe711c" modify_external_catalogs: true
I20260811 18:57:32.570466 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 7538ee8c4da948d480f907de8953bb9a (DEFAULT_TABLE table=default.Table_a273b6a16afa44d196b4654b067735a5 [id=babc4abc6e5c4bb0806b988aa2f5a7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.571019 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7538ee8c4da948d480f907de8953bb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.575691 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 0c108daf7ba848e585b07b3f0ba78bdc (DEFAULT_TABLE table=default.Table_4013b1905ea74bc6a06f9468d9025570 [id=e25121ff4ff6437786d67ff375ccc427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.576229 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c108daf7ba848e585b07b3f0ba78bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.587445 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 60fe0c1500c143c3a5fcefeaed303991 (DEFAULT_TABLE table=default.Table_2e0b184f30d7464fbb8120d29bd0a80b [id=292b0341d47f4ad29a78a33fe7f02a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.587958 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fe0c1500c143c3a5fcefeaed303991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.558634 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.562130 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 7538ee8c4da948d480f907de8953bb9a (DEFAULT_TABLE table=default.Table_a273b6a16afa44d196b4654b067735a5 [id=babc4abc6e5c4bb0806b988aa2f5a7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.592902 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7538ee8c4da948d480f907de8953bb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.565196 3824 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.594249 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: OK, instance_id { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" instance_seqno: 1786474650422925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:32.594631 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:32.594614 5376 raft_consensus.cc:2781] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.594743 5376 raft_consensus.cc:686] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.595031 5376 consensus_queue.cc:237] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.597996 3678 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.640s (SENT)
I20260811 18:57:32.598153 3678 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:32.598759 5091 raft_consensus.cc:2726] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:32.561875 4163 raft_consensus.cc:348] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.598917 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 60fe0c1500c143c3a5fcefeaed303991 (DEFAULT_TABLE table=default.Table_2e0b184f30d7464fbb8120d29bd0a80b [id=292b0341d47f4ad29a78a33fe7f02a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.599400 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fe0c1500c143c3a5fcefeaed303991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.603473 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0c108daf7ba848e585b07b3f0ba78bdc (DEFAULT_TABLE table=default.Table_4013b1905ea74bc6a06f9468d9025570 [id=e25121ff4ff6437786d67ff375ccc427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.603873 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c108daf7ba848e585b07b3f0ba78bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.606976 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2460d58bb949e4a3a2c9f74dd800a8 (DEFAULT_TABLE table=default.Table_619d1e7c0e384107962b41e86e45c45b [id=1f4ab61624c84bde9f6689e2716a8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.607391 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2460d58bb949e4a3a2c9f74dd800a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.610390 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 756) took 3768 ms. Trace:
I20260811 18:57:32.610580 4169 rpcz_store.cc:276] 0811 18:57:28.841629 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.844781 (+ 3152us) service_pool.cc:227] Handling call
0811 18:57:32.523260 (+3678479us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:32.523275 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.550153 (+ 26878us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.553421 (+ 3268us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.610368 (+ 56947us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3678184}
I20260811 18:57:32.611192 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 515d5b5afbd74ec5ab90a503aef2f4fb (table default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]) successfully deleted
I20260811 18:57:32.611848 4169 tablet_replica.cc:330] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.612128 4169 raft_consensus.cc:2229] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.598917 4163 raft_consensus.cc:374] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.618291 4163 raft_consensus.cc:729] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:32.620924 3824 consensus_peers.cc:487] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f335e50ae57547ed80b144129faaffbb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:32.621232 5286 consensus_queue.cc:1035] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:32.612473 4169 raft_consensus.cc:2258] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:32.622488 3678 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.138s (SENT)
I20260811 18:57:32.622637 3678 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:32.622805 3678 raft_consensus.cc:2670] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.623489 4169 ts_tablet_manager.cc:1905] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.620450 4163 consensus_queue.cc:260] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.635442 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfa85fca4bd41dc83acb02ac610cf97 (DEFAULT_TABLE table=default.Table_2698f49dd4b64fce9a411b1c3307abf2 [id=c30db61b225d42e491bc9529e2b91cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.635990 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfa85fca4bd41dc83acb02ac610cf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.638480 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 844bcd16f06c4788af9c0d7d320464aa
I20260811 18:57:32.639493 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2033e26987554c839ae47bd350d82580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.640264 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2033e26987554c839ae47bd350d82580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.640060 3547 catalog_manager.cc:5495] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
W20260811 18:57:32.642067 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3195082643264e0d90f70e1007ab9b24 consensus state has no leader
I20260811 18:57:32.642710 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_58b2359b8659428190fe3cd7688c1d4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.643088 4163 ts_tablet_manager.cc:1428] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.105s user 0.003s sys 0.000s
I20260811 18:57:32.643498 4163 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:32.643522 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58b2359b8659428190fe3cd7688c1d4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.645462 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0a5b06fa572045f3a5fc5a0b123d77cc
I20260811 18:57:32.646327 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c4384a8ba27a4a7696a6051325ddede2" } new_table_name: "default.Table_2ce145eabbad4a958c6799476314823c" modify_external_catalogs: true
I20260811 18:57:32.650650 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b (DEFAULT_TABLE table=default.Table_19742da9decc46818ae2cd5debeee68c [id=8f3414e814ed42cca911730ff2785356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.651069 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a3973dfdaac746a6af7a402277d2711b (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.651309 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15dd14e2b4bd4a08b5b08f532a55291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.651453 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 844bcd16f06c4788af9c0d7d320464aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.650784 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.651722 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3973dfdaac746a6af7a402277d2711b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.651618 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fae8f41d600f4fbfaed39f695cf6672a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.651871 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b50efab3c782447cad8e45b1fa51aa1f (DEFAULT_TABLE table=default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 [id=1f5525b9c44f471dbabaed8159de8784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.652218 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50efab3c782447cad8e45b1fa51aa1f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:32.652432 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fae8f41d600f4fbfaed39f695cf6672a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.654709 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.655424 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.658726 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b (DEFAULT_TABLE table=default.Table_19742da9decc46818ae2cd5debeee68c [id=8f3414e814ed42cca911730ff2785356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.658962 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b50efab3c782447cad8e45b1fa51aa1f (DEFAULT_TABLE table=default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 [id=1f5525b9c44f471dbabaed8159de8784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.659204 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15dd14e2b4bd4a08b5b08f532a55291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.659518 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50efab3c782447cad8e45b1fa51aa1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.663306 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a3973dfdaac746a6af7a402277d2711b (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.663750 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3973dfdaac746a6af7a402277d2711b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.666551 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.668939 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 844bcd16f06c4788af9c0d7d320464aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.685328 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21828ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:32.689956 5091 raft_consensus.cc:482] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.689916 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4abd6e9a89814dbaa84316475bbe1ae5 (DEFAULT_TABLE table=default.Table_04e3e316fa374868a4be1497035d91e8 [id=dcf9bb82d0794200a6f82208c97b2532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.690156 5091 raft_consensus.cc:504] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.690943 5091 leader_election.cc:288] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.693892 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfa85fca4bd41dc83acb02ac610cf97 (DEFAULT_TABLE table=default.Table_2698f49dd4b64fce9a411b1c3307abf2 [id=c30db61b225d42e491bc9529e2b91cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.694777 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfa85fca4bd41dc83acb02ac610cf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.684165 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b278420f8d2f4be8b973de3eb756f69a (DEFAULT_TABLE table=default.Table_c284a387f502480c8faa36ea0e40a8f7 [id=bc157b21c5424118bc3d36507b8d4a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.696789 3717 tablet_service.cc:1467] Processing CreateTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f (DEFAULT_TABLE table=default.Table_ddf22cbb415c485ebc1407d9104b715d [id=e01f868b4ac2491ea511820aa199319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.702286 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 4abd6e9a89814dbaa84316475bbe1ae5 (DEFAULT_TABLE table=default.Table_04e3e316fa374868a4be1497035d91e8 [id=dcf9bb82d0794200a6f82208c97b2532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.702741 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abd6e9a89814dbaa84316475bbe1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.702827 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f (DEFAULT_TABLE table=default.Table_ddf22cbb415c485ebc1407d9104b715d [id=e01f868b4ac2491ea511820aa199319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.703168 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86bfa65a8774b51bc7e7fe59442bf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.703099 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1374a80d15444cb0faff339275aa8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:32.703737 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b9bdd685e23249119bbeac4997cbbf9c" } new_table_name: "default.Table_78bb9ed48dc34a4ea11b23eec0467238" modify_external_catalogs: true
I20260811 18:57:32.706127 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" } modify_external_catalogs: true
I20260811 18:57:32.706444 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" } modify_external_catalogs: true
I20260811 18:57:32.707315 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b278420f8d2f4be8b973de3eb756f69a (DEFAULT_TABLE table=default.Table_c284a387f502480c8faa36ea0e40a8f7 [id=bc157b21c5424118bc3d36507b8d4a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.708015 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278420f8d2f4be8b973de3eb756f69a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:32.708799 3676 leader_election.cc:341] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.709666 4167 ts_tablet_manager.cc:1939] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.714035 4169 ts_tablet_manager.cc:1918] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.714391 4169 log.cc:1196] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:32.714912 4169 ts_tablet_manager.cc:1939] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.725315 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2460d58bb949e4a3a2c9f74dd800a8 (DEFAULT_TABLE table=default.Table_619d1e7c0e384107962b41e86e45c45b [id=1f4ab61624c84bde9f6689e2716a8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.734184 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86bfa65a8774b51bc7e7fe59442bf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.737161 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278420f8d2f4be8b973de3eb756f69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.737447 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 (DEFAULT_TABLE table=default.Table_aadee0d920fc4be5aa7f965bc9db93e4 [id=3886fb421ea4409585d7020e74f2221e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.718189 4163 tablet_bootstrap.cc:654] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.731600 5334 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.738627 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abd6e9a89814dbaa84316475bbe1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.738447 5334 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.739486 5334 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.738057 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 (DEFAULT_TABLE table=default.Table_aadee0d920fc4be5aa7f965bc9db93e4 [id=3886fb421ea4409585d7020e74f2221e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.742831 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.743137 3745 raft_consensus.cc:2454] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.746374 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2460d58bb949e4a3a2c9f74dd800a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.746645 3824 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.746990 5334 raft_consensus.cc:2781] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.747095 5334 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.747185 5334 raft_consensus.cc:3037] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.750763 5334 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.751957 5334 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.754750 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.755738 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 736) took 5100 ms. Trace:
I20260811 18:57:32.755857 4167 rpcz_store.cc:276] 0811 18:57:27.655124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.655175 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.270874 (+4615699us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:32.271193 (+ 319us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.449810 (+178617us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.493485 (+ 43675us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.755604 (+262119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4613561}
I20260811 18:57:32.756691 4167 tablet_replica.cc:330] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.756985 4167 raft_consensus.cc:2229] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.757309 4167 raft_consensus.cc:2258] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.758891 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a902ad70a25546be926ff8f7bf2ed64f (table default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]) successfully deleted
I20260811 18:57:32.760986 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.761263 3745 raft_consensus.cc:3037] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.764796 3745 raft_consensus.cc:2454] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.765379 3824 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.765771 5334 raft_consensus.cc:2781] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.765905 5334 raft_consensus.cc:686] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.766213 5334 consensus_queue.cc:237] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.777338 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.781170 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b609177908d04a829b1fe744ef0f53e1 (DEFAULT_TABLE table=default.Table_03b64ac0b80f46288fcf2b34397efcaa [id=4845fddf6562470ebb7275b67fad076e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.781668 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b609177908d04a829b1fe744ef0f53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.786603 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 758) took 3863 ms. Trace:
I20260811 18:57:32.786787 4169 rpcz_store.cc:276] 0811 18:57:28.922855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.922916 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:32.611800 (+3688884us) ts_tablet_manager.cc:1145] Deleting tablet 0ba6246e6d29418cbca5f751b34f4ef1
0811 18:57:32.611812 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.670414 (+ 58602us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.713894 (+ 43480us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.786565 (+ 72671us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3688638}
I20260811 18:57:32.787499 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 759) took 3864 ms. Trace:
I20260811 18:57:32.787657 4169 rpcz_store.cc:276] 0811 18:57:28.923346 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.923410 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:32.787373 (+3863963us) ts_tablet_manager.cc:1145] Deleting tablet b4dc2df4c6464d71bb13fe25e0046758
0811 18:57:32.787386 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.787482 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3863752}
I20260811 18:57:32.788220 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 760) took 3864 ms. Trace:
I20260811 18:57:32.788369 4169 rpcz_store.cc:276] 0811 18:57:28.923777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.923822 (+ 45us) service_pool.cc:227] Handling call
0811 18:57:32.788093 (+3864271us) ts_tablet_manager.cc:1145] Deleting tablet 2474ae972e9242b69f488b7a2fdbc481
0811 18:57:32.788106 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.788200 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3864098}
I20260811 18:57:32.788412 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ba6246e6d29418cbca5f751b34f4ef1 (table default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]) successfully deleted
W20260811 18:57:32.788899 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b4dc2df4c6464d71bb13fe25e0046758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4dc2df4c6464d71bb13fe25e0046758 already in progress: creating tablet
I20260811 18:57:32.789086 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 757) took 3866 ms. Trace:
W20260811 18:57:32.789438 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2474ae972e9242b69f488b7a2fdbc481 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2474ae972e9242b69f488b7a2fdbc481 already in progress: creating tablet
I20260811 18:57:32.789796 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b609177908d04a829b1fe744ef0f53e1 (DEFAULT_TABLE table=default.Table_03b64ac0b80f46288fcf2b34397efcaa [id=4845fddf6562470ebb7275b67fad076e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.790706 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b609177908d04a829b1fe744ef0f53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.795146 4169 rpcz_store.cc:276] 0811 18:57:28.922554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924383 (+ 1829us) service_pool.cc:227] Handling call
0811 18:57:32.788894 (+3864511us) ts_tablet_manager.cc:1145] Deleting tablet b203df4124364737839bf4c1ae072988
0811 18:57:32.788908 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.788999 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3864309}
W20260811 18:57:32.798942 3824 leader_election.cc:334] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.146s (SENT)
W20260811 18:57:32.801517 3824 consensus_peers.cc:487] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a902ad70a25546be926ff8f7bf2ed64f. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:32.801496 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b203df4124364737839bf4c1ae072988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b203df4124364737839bf4c1ae072988 already in progress: creating tablet
I20260811 18:57:32.803648 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 761) took 3875 ms. Trace:
I20260811 18:57:32.804241 4169 rpcz_store.cc:276] 0811 18:57:28.928622 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.928902 (+ 280us) service_pool.cc:227] Handling call
0811 18:57:32.803059 (+3874157us) ts_tablet_manager.cc:1145] Deleting tablet 8e120a45539f49bb82ab0fb95eac36f9
0811 18:57:32.803079 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.803619 (+ 540us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3873958}
I20260811 18:57:32.806171 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 0b07f6b410654740aed79e58fb281007 (DEFAULT_TABLE table=default.Table_9183f700c0544cd5942abebaaa8a121f [id=af4584f5e44548e3a40317d2301eef07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.806910 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b07f6b410654740aed79e58fb281007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.806170 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 0b07f6b410654740aed79e58fb281007 (DEFAULT_TABLE table=default.Table_9183f700c0544cd5942abebaaa8a121f [id=af4584f5e44548e3a40317d2301eef07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.807621 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b07f6b410654740aed79e58fb281007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.807782 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 762) took 3741 ms. Trace:
W20260811 18:57:32.807967 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8e120a45539f49bb82ab0fb95eac36f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e120a45539f49bb82ab0fb95eac36f9 already in progress: creating tablet
I20260811 18:57:32.809928 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 10e441b5df3d40fd93cf2dafe33bc2f4 (DEFAULT_TABLE table=default.Table_0db19ab70f764e00b17ee2e0f4742b8b [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.810467 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e441b5df3d40fd93cf2dafe33bc2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.821727 4169 rpcz_store.cc:276] 0811 18:57:29.066239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.066311 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:32.807608 (+3741297us) ts_tablet_manager.cc:1145] Deleting tablet c77882ff8d974057a61f037f1943f58f
0811 18:57:32.807626 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.807760 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3741024}
I20260811 18:57:32.826850 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 10e441b5df3d40fd93cf2dafe33bc2f4 (DEFAULT_TABLE table=default.Table_0db19ab70f764e00b17ee2e0f4742b8b [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.830652 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e441b5df3d40fd93cf2dafe33bc2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.837330 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 77ae7711d3ce4d769e8e7fa66d285329 (DEFAULT_TABLE table=default.Table_160e44b6b513432c89082037a071fbb4 [id=421aa2aad7ed4696b94c748b9b7f9d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.837894 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ae7711d3ce4d769e8e7fa66d285329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.838274 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 77ae7711d3ce4d769e8e7fa66d285329 (DEFAULT_TABLE table=default.Table_160e44b6b513432c89082037a071fbb4 [id=421aa2aad7ed4696b94c748b9b7f9d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.838773 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ae7711d3ce4d769e8e7fa66d285329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.839425 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 772) took 3179 ms. Trace:
I20260811 18:57:32.840962 4169 rpcz_store.cc:276] 0811 18:57:29.660448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.660646 (+ 198us) service_pool.cc:227] Handling call
0811 18:57:32.826238 (+3165592us) ts_tablet_manager.cc:1145] Deleting tablet 1d721b0e55384d8a8efd88a0e47b1504
0811 18:57:32.828227 (+ 1989us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.839406 (+ 11179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3165408}
W20260811 18:57:32.843047 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1d721b0e55384d8a8efd88a0e47b1504 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d721b0e55384d8a8efd88a0e47b1504 already in progress: creating tablet
I20260811 18:57:32.843391 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 773) took 3182 ms. Trace:
I20260811 18:57:32.845436 4169 rpcz_store.cc:276] 0811 18:57:29.661360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.661491 (+ 131us) service_pool.cc:227] Handling call
0811 18:57:32.843243 (+3181752us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:32.843259 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.843371 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3181553}
W20260811 18:57:32.846079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1e1374a80d15444cb0faff339275aa8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e1374a80d15444cb0faff339275aa8d already in progress: creating tablet
I20260811 18:57:32.847842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 774) took 3185 ms. Trace:
I20260811 18:57:32.847951 4169 rpcz_store.cc:276] 0811 18:57:29.662481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.662610 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:32.847703 (+3185093us) ts_tablet_manager.cc:1145] Deleting tablet 47a782b8470b4d14ac89b3b7d463a125
0811 18:57:32.847716 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.847822 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3184902}
W20260811 18:57:32.848553 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 47a782b8470b4d14ac89b3b7d463a125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47a782b8470b4d14ac89b3b7d463a125 already in progress: creating tablet
I20260811 18:57:32.852102 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 775) took 3171 ms. Trace:
I20260811 18:57:32.852221 4169 rpcz_store.cc:276] 0811 18:57:29.680714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.680920 (+ 206us) service_pool.cc:227] Handling call
0811 18:57:32.851958 (+3171038us) ts_tablet_manager.cc:1145] Deleting tablet 1c74abe9dcf642e9b4d40df6ff3639ef
0811 18:57:32.851974 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.852081 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170727}
W20260811 18:57:32.853380 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1c74abe9dcf642e9b4d40df6ff3639ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c74abe9dcf642e9b4d40df6ff3639ef already in progress: creating tablet
I20260811 18:57:32.853663 3852 tablet_service.cc:1467] Processing CreateTablet for tablet bcfad3da366e4554b06bfa51a2535419 (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.854128 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfad3da366e4554b06bfa51a2535419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.758320 4167 ts_tablet_manager.cc:1905] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.857671 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e7d75d8468fc4c938fde0b2951d84562
I20260811 18:57:32.858968 4163 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.859122 4163 ts_tablet_manager.cc:1397] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.216s user 0.003s sys 0.007s
I20260811 18:57:32.859933 4163 raft_consensus.cc:348] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.859969 5327 log.cc:824] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Log is configured to *not* fsync() on all Append() calls
W20260811 18:57:32.860097 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:32.860124 4163 raft_consensus.cc:374] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.860174 4163 raft_consensus.cc:729] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.860337 4163 consensus_queue.cc:260] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.860561 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_706c784347b146089c032d699f6a9abe" } new_table_name: "default.Table_25c8439e034242f7bc3d1c546188fc8e" modify_external_catalogs: true
I20260811 18:57:32.860877 4163 ts_tablet_manager.cc:1428] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:32.861955 5376 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.862164 5376 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.862300 3824 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.384s (SENT)
I20260811 18:57:32.862721 4163 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.862865 5376 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.865048 4163 tablet_bootstrap.cc:654] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.865208 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" } modify_external_catalogs: true
I20260811 18:57:32.865482 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" } modify_external_catalogs: true
I20260811 18:57:32.874814 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d75d8468fc4c938fde0b2951d84562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.875569 5091 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.880887 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 776) took 3199 ms. Trace:
I20260811 18:57:32.882511 4169 rpcz_store.cc:276] 0811 18:57:29.681979 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.687792 (+ 5813us) service_pool.cc:227] Handling call
0811 18:57:32.879943 (+3192151us) ts_tablet_manager.cc:1145] Deleting tablet 6880ad27a3eb4599b407edeb2969a08f
0811 18:57:32.880729 (+ 786us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.880862 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3191835}
I20260811 18:57:32.884363 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 (DEFAULT_TABLE table=default.Table_b79839172cff41c3a8235675d44c21fa [id=8033ac0465864ef3870d27a6cf52244d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.885010 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fb15b72c1d49d2ac4961bc04655a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.886902 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 785) took 2877 ms. Trace:
I20260811 18:57:32.888366 4169 rpcz_store.cc:276] 0811 18:57:30.009454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.009577 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:32.886752 (+2877175us) ts_tablet_manager.cc:1145] Deleting tablet f3670f8908d54ebb82f698cdd207e48f
0811 18:57:32.886774 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.886881 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2859157}
I20260811 18:57:32.891156 4169 tablet_replica.cc:330] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.891551 4169 raft_consensus.cc:2229] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.875725 5091 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.892436 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:32.892542 5091 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.892941 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb (DEFAULT_TABLE table=default.Table_6743ac70503f4bbc8811494daa73361f [id=6a0643492e8847e4883fa191f98d5f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.895733 3719 tablet_service.cc:1467] Processing CreateTablet for tablet a0a12cb16855433eaf447c3bca2f4bad (DEFAULT_TABLE table=default.Table_96bf47f7003740cba4c66f780597a121 [id=782aced1be1a402980d996e519b959df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896520 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 (DEFAULT_TABLE table=default.Table_b79839172cff41c3a8235675d44c21fa [id=8033ac0465864ef3870d27a6cf52244d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896664 3714 tablet_service.cc:1467] Processing CreateTablet for tablet bcfad3da366e4554b06bfa51a2535419 (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896859 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 922c6fcbcb184e99bf396b6e69e84299 (DEFAULT_TABLE table=default.Table_bf4fa780a3ba4824b3317ae7e9a379fc [id=66d2227c128e4a41ab2754c4f788c0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896914 3853 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb (DEFAULT_TABLE table=default.Table_6743ac70503f4bbc8811494daa73361f [id=6a0643492e8847e4883fa191f98d5f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897204 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fb15b72c1d49d2ac4961bc04655a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.897425 3855 tablet_service.cc:1467] Processing CreateTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897430 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 80b31661fc59476496371e71ed1a21e1 (DEFAULT_TABLE table=default.Table_5b76a052465c4749a059cbf9171ab6be [id=d82bca795c574e6896ed5e93201570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897814 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25b1eddd4a64847b51ad3cdc0238ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.899363 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.900053 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfad3da366e4554b06bfa51a2535419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.901324 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.901569 3745 raft_consensus.cc:2454] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.902683 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 922c6fcbcb184e99bf396b6e69e84299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.902642 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 80b31661fc59476496371e71ed1a21e1 (DEFAULT_TABLE table=default.Table_5b76a052465c4749a059cbf9171ab6be [id=d82bca795c574e6896ed5e93201570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.903000 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b31661fc59476496371e71ed1a21e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.903139 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b31661fc59476496371e71ed1a21e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.904839 3824 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.905279 5376 consensus_queue.cc:1035] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:32.905715 5376 raft_consensus.cc:2781] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.905795 5376 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.905859 5376 raft_consensus.cc:3037] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.906630 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.908591 5376 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.909152 5376 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.910944 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a0a12cb16855433eaf447c3bca2f4bad (DEFAULT_TABLE table=default.Table_96bf47f7003740cba4c66f780597a121 [id=782aced1be1a402980d996e519b959df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.911355 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0a12cb16855433eaf447c3bca2f4bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.912214 4169 pending_rounds.cc:62] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:32.912299 4169 pending_rounds.cc:68] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400169131261952 op_type: NO_OP noop_request { }
I20260811 18:57:32.912483 4169 raft_consensus.cc:2866] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:32.912596 4169 raft_consensus.cc:2258] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.913323 4169 ts_tablet_manager.cc:1905] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.914399 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6880ad27a3eb4599b407edeb2969a08f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6880ad27a3eb4599b407edeb2969a08f already in progress: creating tablet
W20260811 18:57:32.914934 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f3670f8908d54ebb82f698cdd207e48f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3670f8908d54ebb82f698cdd207e48f already in progress: creating tablet
I20260811 18:57:32.912314 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 922c6fcbcb184e99bf396b6e69e84299 (DEFAULT_TABLE table=default.Table_bf4fa780a3ba4824b3317ae7e9a379fc [id=66d2227c128e4a41ab2754c4f788c0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.917536 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 922c6fcbcb184e99bf396b6e69e84299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.918533 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d75d8468fc4c938fde0b2951d84562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.918777 5283 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.918957 5283 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.919878 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.920406 5283 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.927886 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 81c02b97930249d18b1e6fd3fd695e4b (DEFAULT_TABLE table=default.Table_3418766144ec4b91bb802faa2ad8808e [id=d22eb4f13976484a86cdbe8f85f55fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.927985 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 81c02b97930249d18b1e6fd3fd695e4b (DEFAULT_TABLE table=default.Table_3418766144ec4b91bb802faa2ad8808e [id=d22eb4f13976484a86cdbe8f85f55fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.928716 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c02b97930249d18b1e6fd3fd695e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.929980 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e06e3db1c605429895482ac91b4b8053 (DEFAULT_TABLE table=default.Table_0c05dd720298449e9aa85faeadb50b96 [id=cb3bbf69e7b14f54badc8a9f679b7550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.930773 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06e3db1c605429895482ac91b4b8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.933364 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25b1eddd4a64847b51ad3cdc0238ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.933673 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c02b97930249d18b1e6fd3fd695e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.931367 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.946852 3744 raft_consensus.cc:2454] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.946827 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.947090 3745 raft_consensus.cc:3037] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.947381 3824 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.947669 5283 raft_consensus.cc:2781] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.947747 5283 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.947814 5283 raft_consensus.cc:3037] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.951349 5283 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.952586 5283 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.975025 5283 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.975343 5283 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.976680 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.977248 5283 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.980270 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:32.980904 3678 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.739s (SENT)
I20260811 18:57:32.981043 3678 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:32.981566 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.981887 3743 raft_consensus.cc:3037] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.986229 4163 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.986485 4163 ts_tablet_manager.cc:1397] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.004s sys 0.002s
I20260811 18:57:32.987440 4163 raft_consensus.cc:348] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.987836 4163 raft_consensus.cc:374] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.987954 4163 raft_consensus.cc:729] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.988257 4163 consensus_queue.cc:260] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.988958 4163 ts_tablet_manager.cc:1428] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:32.993856 4163 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:32.981216 3678 raft_consensus.cc:2670] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.995747 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e06e3db1c605429895482ac91b4b8053 (DEFAULT_TABLE table=default.Table_0c05dd720298449e9aa85faeadb50b96 [id=cb3bbf69e7b14f54badc8a9f679b7550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.003852 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0a12cb16855433eaf447c3bca2f4bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.009902 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06e3db1c605429895482ac91b4b8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.010624 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.010948 3744 raft_consensus.cc:2454] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.011898 3824 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.012667 5283 raft_consensus.cc:2781] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.012872 5283 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.013031 5283 raft_consensus.cc:3037] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.023237 3745 raft_consensus.cc:2454] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.025441 3824 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.025852 5334 raft_consensus.cc:2781] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.026007 5334 raft_consensus.cc:686] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.026259 5334 consensus_queue.cc:237] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.028584 3743 raft_consensus.cc:2454] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.029212 3824 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.029558 5376 raft_consensus.cc:2781] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.029701 5376 raft_consensus.cc:686] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.029933 5376 consensus_queue.cc:237] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.047895 3676 leader_election.cc:341] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:33.048596 5407 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.048877 5407 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.049849 5407 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.057433 4167 ts_tablet_manager.cc:1918] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.057614 4167 log.cc:1196] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:33.058140 4167 ts_tablet_manager.cc:1939] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.058774 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 737) took 5403 ms. Trace:
I20260811 18:57:33.058915 4167 rpcz_store.cc:276] 0811 18:57:27.655613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.655664 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.756607 (+5100943us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:32.756620 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.024147 (+267527us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.052956 (+ 28809us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.058750 (+ 5794us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5100717}
I20260811 18:57:33.059803 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 745) took 5346 ms. Trace:
I20260811 18:57:33.059922 4167 rpcz_store.cc:276] 0811 18:57:27.713460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.713527 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:33.059617 (+5346090us) ts_tablet_manager.cc:1145] Deleting tablet 4059e5907b1e488ab9e1d04e563f96cd
0811 18:57:33.059633 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.059765 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5345696}
I20260811 18:57:33.060465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 746) took 5079 ms. Trace:
I20260811 18:57:33.060570 4167 rpcz_store.cc:276] 0811 18:57:27.980794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.984328 (+ 3534us) service_pool.cc:227] Handling call
0811 18:57:33.060338 (+5076010us) ts_tablet_manager.cc:1145] Deleting tablet 4093564a88264ed59c945523c9ddc9b3
0811 18:57:33.060351 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.060448 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5073227}
I20260811 18:57:33.020853 5283 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.063416 5283 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:33.065805 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 100091us
I20260811 18:57:33.069216 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.075932 5418 tablet.cc:1722] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:33.081126 4167 tablet_replica.cc:330] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.081574 4167 raft_consensus.cc:2229] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.081908 4167 raft_consensus.cc:2258] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.082120 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d2d7d986900244ddbc05dbe73a708dcf (table default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]) successfully deleted
W20260811 18:57:33.082793 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4059e5907b1e488ab9e1d04e563f96cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4059e5907b1e488ab9e1d04e563f96cd already in progress: creating tablet
I20260811 18:57:33.082923 4167 ts_tablet_manager.cc:1905] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.083241 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4093564a88264ed59c945523c9ddc9b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4093564a88264ed59c945523c9ddc9b3 already in progress: creating tablet
W20260811 18:57:33.083829 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73f05e1d5cb947fe9dbcc2e876a23788 consensus state has no leader
I20260811 18:57:33.080457 4162 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.084383 4162 ts_tablet_manager.cc:1397] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.789s user 0.028s sys 0.000s
I20260811 18:57:33.085253 4162 raft_consensus.cc:348] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.085537 4162 raft_consensus.cc:374] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.085613 4162 raft_consensus.cc:729] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.085862 4162 consensus_queue.cc:260] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.086598 4162 ts_tablet_manager.cc:1428] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.088552 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7cabaea271024c3daf3ffc96c5825f1e" } new_table_name: "default.Table_f5e5c61d6ae446fdbe466f320a17ed6c" modify_external_catalogs: true
W20260811 18:57:33.088749 3676 consensus_peers.cc:487] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 515d5b5afbd74ec5ab90a503aef2f4fb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.089713 4163 tablet_bootstrap.cc:654] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.090258 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1afc7507935345b2a081a64ed01106da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.091357 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edd393d9c4d24aab8a170e1329400656 consensus state has no leader
I20260811 18:57:33.092876 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" } modify_external_catalogs: true
I20260811 18:57:33.093182 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" } modify_external_catalogs: true
I20260811 18:57:33.091682 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c515c153df4317b3fb427983ee15e0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:33.095176 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f868e00d47d34ad995459fb265d9157a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.096586 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1afc7507935345b2a081a64ed01106da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:33.097393 3676 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:33.098284 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f868e00d47d34ad995459fb265d9157a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.099164 4162 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.100378 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6080085ba7594b18b5594e8868f0b80e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.104146 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6080085ba7594b18b5594e8868f0b80e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.107323 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a4328549b8d4ad08d0135327263c811" } modify_external_catalogs: true
I20260811 18:57:33.107609 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a4328549b8d4ad08d0135327263c811" } modify_external_catalogs: true
I20260811 18:57:33.112727 4162 tablet_bootstrap.cc:654] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.117805 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:33.118245 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 60688us
I20260811 18:57:33.119323 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.119573 3740 raft_consensus.cc:3037] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.122763 3740 raft_consensus.cc:2454] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.131449 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.132221 3824 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.132503 5376 raft_consensus.cc:2781] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.132741 5376 raft_consensus.cc:686] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.133118 5376 consensus_queue.cc:237] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.136451 4167 ts_tablet_manager.cc:1918] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.136636 4167 log.cc:1196] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/06b8f2a97c0f46ea8c56b3de2b97e60c
I20260811 18:57:33.137081 4167 ts_tablet_manager.cc:1939] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.137022 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2fc380eb763b488e815a72be20c8f563"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.138537 5422 tablet.cc:1722] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:33.144275 4162 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.144584 4162 ts_tablet_manager.cc:1397] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.000s
I20260811 18:57:33.145155 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6df5545532e346f7abcdc4cd228b84da
I20260811 18:57:33.145545 4162 raft_consensus.cc:348] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.145825 4162 raft_consensus.cc:374] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.145887 4162 raft_consensus.cc:729] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.146139 4162 consensus_queue.cc:260] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.146793 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df5545532e346f7abcdc4cd228b84da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.152844 4162 ts_tablet_manager.cc:1428] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
W20260811 18:57:33.159797 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fc380eb763b488e815a72be20c8f563 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.163645 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 747) took 5060 ms. Trace:
I20260811 18:57:33.163791 4167 rpcz_store.cc:276] 0811 18:57:28.103105 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.103163 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:33.081054 (+4977891us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:33.081071 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.114992 (+ 33921us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.136305 (+ 21313us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.163614 (+ 27309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4977572}
I20260811 18:57:33.164305 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (table default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]) successfully deleted
W20260811 18:57:33.185596 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae75d6652c704e1995966c6dd4c06307 consensus state has no leader
I20260811 18:57:33.188793 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f414c31b8cff4e889368e4207afb1b53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.189416 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f414c31b8cff4e889368e4207afb1b53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.193493 4163 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.193818 4163 ts_tablet_manager.cc:1397] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.200s user 0.008s sys 0.000s
I20260811 18:57:33.194291 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df5545532e346f7abcdc4cd228b84da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.194849 4163 raft_consensus.cc:348] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.195068 4163 raft_consensus.cc:374] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.195113 4163 raft_consensus.cc:729] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.195314 4163 consensus_queue.cc:260] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.195996 4163 ts_tablet_manager.cc:1428] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.198144 5407 raft_consensus.cc:482] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.198303 5407 raft_consensus.cc:504] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:33.198511 3554 catalog_manager.cc:5495] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:33.199031 5407 leader_election.cc:288] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.199077 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:33.200062 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:33.202936 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.207504 4162 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.209781 4162 tablet_bootstrap.cc:654] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.210938 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 748) took 4965 ms. Trace:
I20260811 18:57:33.211063 4167 rpcz_store.cc:276] 0811 18:57:28.245441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.245540 (+ 99us) service_pool.cc:227] Handling call
0811 18:57:33.165932 (+4920392us) ts_tablet_manager.cc:1145] Deleting tablet ae7da0868b114052ba421801b76fb33c
0811 18:57:33.210760 (+ 44828us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.210911 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4920074}
I20260811 18:57:33.211560 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 749) took 4965 ms. Trace:
I20260811 18:57:33.211666 4167 rpcz_store.cc:276] 0811 18:57:28.245647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.245918 (+ 271us) service_pool.cc:227] Handling call
0811 18:57:33.211430 (+4965512us) ts_tablet_manager.cc:1145] Deleting tablet 6d988ef7e9dd43158d107998ea4d82b9
0811 18:57:33.211445 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.211541 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957400}
I20260811 18:57:33.212041 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 750) took 4957 ms. Trace:
I20260811 18:57:33.212126 4167 rpcz_store.cc:276] 0811 18:57:28.254289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.254369 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:33.211922 (+4957553us) ts_tablet_manager.cc:1145] Deleting tablet b6c70b4163634984bd02dd9b47ffb472
0811 18:57:33.211935 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.212023 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957350}
I20260811 18:57:33.212427 4167 tablet_replica.cc:330] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.212711 4167 raft_consensus.cc:2229] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.212947 4167 raft_consensus.cc:2258] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.213729 4167 ts_tablet_manager.cc:1905] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.203657 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6432abd4fc5c4f9ca95971ca4795e1cf" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.218897 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ae7da0868b114052ba421801b76fb33c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae7da0868b114052ba421801b76fb33c already in progress: creating tablet
W20260811 18:57:33.219095 3676 leader_election.cc:341] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.206453 5286 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:33.219255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6d988ef7e9dd43158d107998ea4d82b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d988ef7e9dd43158d107998ea4d82b9 already in progress: creating tablet
I20260811 18:57:33.219362 5286 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.219570 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b6c70b4163634984bd02dd9b47ffb472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6c70b4163634984bd02dd9b47ffb472 already in progress: creating tablet
I20260811 18:57:33.220072 5286 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.220463 4163 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.220942 5407 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.221076 5407 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.221644 5407 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.222379 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7251204325dd424192a6f58e18e71c34" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.222929 3676 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:33.224912 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
I20260811 18:57:33.209756 4169 ts_tablet_manager.cc:1918] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.217684 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.229417 4169 log.cc:1196] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/4687b95728fc485fa0e842bc447eb83f
I20260811 18:57:33.229295 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.229595 3740 raft_consensus.cc:2454] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.229930 4169 ts_tablet_manager.cc:1939] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.230106 3824 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.230548 5286 raft_consensus.cc:2781] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.230685 5286 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.230813 5286 raft_consensus.cc:3037] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.230778 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 790) took 3105 ms. Trace:
I20260811 18:57:33.230953 4169 rpcz_store.cc:276] 0811 18:57:30.125771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.273241 (+147470us) service_pool.cc:227] Handling call
0811 18:57:32.890747 (+2617506us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:32.891103 (+ 356us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.071413 (+180310us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.209634 (+138221us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.230736 (+ 21102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2617289}
I20260811 18:57:33.231567 4169 tablet_replica.cc:330] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.232400 4169 raft_consensus.cc:2229] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.233039 4169 raft_consensus.cc:2258] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.233585 4167 ts_tablet_manager.cc:1918] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.233755 4167 log.cc:1196] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/867259e37fe44c33a35823ec2d896754
I20260811 18:57:33.233952 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 4687b95728fc485fa0e842bc447eb83f (table default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]) successfully deleted
I20260811 18:57:33.234247 4167 ts_tablet_manager.cc:1939] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.235468 5286 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.237732 4163 tablet_bootstrap.cc:654] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.238113 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:33.239550 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e71d261fb704c1296dfc437453d6f38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.240296 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e71d261fb704c1296dfc437453d6f38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.249704 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4af28c051e8243eaa73d4a5ed1abcea6
I20260811 18:57:33.251550 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.252286 4162 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.252539 4162 ts_tablet_manager.cc:1397] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.000s
I20260811 18:57:33.253422 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.253788 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:33.254719 4162 raft_consensus.cc:348] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.255143 4162 raft_consensus.cc:374] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.255280 4162 raft_consensus.cc:729] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.255676 4162 consensus_queue.cc:260] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.259362 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.259603 3740 raft_consensus.cc:3037] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.260582 4162 ts_tablet_manager.cc:1428] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.262764 3740 raft_consensus.cc:2454] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.263386 3824 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.263787 5376 raft_consensus.cc:2781] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.263916 5376 raft_consensus.cc:686] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.264145 5376 consensus_queue.cc:237] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.265545 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 19962ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:33.267428 5286 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.268824 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" } modify_external_catalogs: true
I20260811 18:57:33.269136 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" } modify_external_catalogs: true
I20260811 18:57:33.280232 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 751) took 5025 ms. Trace:
I20260811 18:57:33.280429 4167 rpcz_store.cc:276] 0811 18:57:28.254737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.254793 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:33.212375 (+4957582us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:33.212387 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.225492 (+ 13105us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.233434 (+ 7942us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.280193 (+ 46759us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957393}
I20260811 18:57:33.280936 4167 tablet_replica.cc:330] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.281199 4167 raft_consensus.cc:2229] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.281421 4167 raft_consensus.cc:2258] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.281955 4162 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.284346 4162 tablet_bootstrap.cc:654] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.280920 4163 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.286844 4163 ts_tablet_manager.cc:1397] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.002s
I20260811 18:57:33.287741 4163 raft_consensus.cc:348] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.287973 4163 raft_consensus.cc:374] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.288040 4163 raft_consensus.cc:729] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.288411 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 867259e37fe44c33a35823ec2d896754 (table default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]) successfully deleted
I20260811 18:57:33.290200 4167 ts_tablet_manager.cc:1905] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.293762 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_972a064a812a441bb4068503de32341e" } new_table_name: "default.Table_227406eaa3694731a78479da6280faac" modify_external_catalogs: true
I20260811 18:57:33.301700 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.302657 4167 ts_tablet_manager.cc:1918] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.302838 4167 log.cc:1196] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/490f34d7a403416fbab64e46973746de
I20260811 18:57:33.303328 4167 ts_tablet_manager.cc:1939] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.305712 4163 consensus_queue.cc:260] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.307442 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 752) took 5052 ms. Trace:
I20260811 18:57:33.307607 4167 rpcz_store.cc:276] 0811 18:57:28.255116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255165 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:33.280878 (+5025713us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:33.280892 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.295090 (+ 14198us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.302500 (+ 7410us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.307407 (+ 4907us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5025551}
I20260811 18:57:33.308290 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 490f34d7a403416fbab64e46973746de (table default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]) successfully deleted
I20260811 18:57:33.308498 4163 ts_tablet_manager.cc:1428] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260811 18:57:33.308835 4163 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.310721 4162 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.310983 4162 ts_tablet_manager.cc:1397] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
I20260811 18:57:33.312465 4162 raft_consensus.cc:348] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.312938 4162 raft_consensus.cc:374] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.313073 4162 raft_consensus.cc:729] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.313416 4162 consensus_queue.cc:260] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.317925 4162 ts_tablet_manager.cc:1428] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.331637 4163 tablet_bootstrap.cc:654] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.334638 4167 tablet_replica.cc:330] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.334981 4167 raft_consensus.cc:2229] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.335202 4167 raft_consensus.cc:2258] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.335729 4162 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.341403 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.342314 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.343119 4167 ts_tablet_manager.cc:1905] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.346707 4162 tablet_bootstrap.cc:654] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.322491 4169 ts_tablet_manager.cc:1905] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.354113 4167 ts_tablet_manager.cc:1918] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.358754 4167 log.cc:1196] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:33.359946 4167 ts_tablet_manager.cc:1939] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.362851 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 753) took 5107 ms. Trace:
I20260811 18:57:33.363807 4167 rpcz_store.cc:276] 0811 18:57:28.255452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255499 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:33.309785 (+5054286us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:33.334543 (+ 24758us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.349755 (+ 15212us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.353968 (+ 4213us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.362820 (+ 8852us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5054114}
I20260811 18:57:33.365088 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 02808605ee6c4634af47fc3fd57e9cd8 (table default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]) successfully deleted
I20260811 18:57:33.365849 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 754) took 5110 ms. Trace:
I20260811 18:57:33.366827 4167 rpcz_store.cc:276] 0811 18:57:28.255798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255846 (+ 48us) service_pool.cc:227] Handling call
0811 18:57:33.365670 (+5109824us) ts_tablet_manager.cc:1145] Deleting tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
0811 18:57:33.365690 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.365825 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5109644}
W20260811 18:57:33.368121 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca already in progress: creating tablet
I20260811 18:57:33.368707 4167 tablet_replica.cc:330] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.371001 4167 raft_consensus.cc:2229] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.371748 4167 raft_consensus.cc:2258] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.373011 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9608d6ddcad749cd9e72dd6f265db0da
I20260811 18:57:33.373049 4167 ts_tablet_manager.cc:1905] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.374698 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.375136 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.375811 4162 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.376042 4162 ts_tablet_manager.cc:1397] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.010s
I20260811 18:57:33.377257 4162 raft_consensus.cc:348] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.377511 5376 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.377655 4162 raft_consensus.cc:374] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.377738 4162 raft_consensus.cc:729] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.377677 5376 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.378024 4162 consensus_queue.cc:260] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.378754 5376 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.378852 4162 ts_tablet_manager.cc:1428] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:33.379453 4162 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.379654 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.379974 3740 raft_consensus.cc:2454] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.380805 3824 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.381129 5376 raft_consensus.cc:2781] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.381219 5376 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.381289 5376 raft_consensus.cc:3037] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.382191 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aa50e46e061b41f79d5d6d048ea15132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.383122 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa50e46e061b41f79d5d6d048ea15132 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.386998 5376 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.388186 5376 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.388525 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.388760 3740 raft_consensus.cc:3037] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.390146 4169 ts_tablet_manager.cc:1918] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.390321 4169 log.cc:1196] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/867259e37fe44c33a35823ec2d896754
I20260811 18:57:33.390899 4169 ts_tablet_manager.cc:1939] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.391513 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 791) took 3265 ms. Trace:
I20260811 18:57:33.391647 4169 rpcz_store.cc:276] 0811 18:57:30.125903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.273472 (+147569us) service_pool.cc:227] Handling call
0811 18:57:33.231470 (+2957998us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:33.231486 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.355666 (+124180us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.390009 (+ 34343us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.391469 (+ 1460us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2957820}
I20260811 18:57:33.392702 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 867259e37fe44c33a35823ec2d896754 (table default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]) successfully deleted
#10 0x7fc7c6b82128 in std::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7c6b7dcc5 in kudu::tools::Action::Run(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action.cc:245:10
I20260811 18:57:33.410027 4162 tablet_bootstrap.cc:654] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.418668 4169 tablet_replica.cc:330] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.418989 4169 raft_consensus.cc:2229] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.419404 4169 pending_rounds.cc:62] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:33.419521 4169 pending_rounds.cc:68] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400169550319616 op_type: NO_OP noop_request { }
I20260811 18:57:33.419749 4169 raft_consensus.cc:2866] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.420368 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b0759818f6cc4cd8b2747ae532c19ccc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.421231 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0759818f6cc4cd8b2747ae532c19ccc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.423039 3740 raft_consensus.cc:2454] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.423664 3824 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.425930 5376 raft_consensus.cc:2781] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.426719 5376 raft_consensus.cc:686] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.427057 5376 consensus_queue.cc:237] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.431197 4167 ts_tablet_manager.cc:1918] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.431386 4167 log.cc:1196] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:33.431780 4167 ts_tablet_manager.cc:1939] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.439646 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be8ef75ca590418486730a869708b637"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.440487 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be8ef75ca590418486730a869708b637 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.443853 4169 raft_consensus.cc:2258] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.445369 4162 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.445608 4162 ts_tablet_manager.cc:1397] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.066s user 0.004s sys 0.009s
I20260811 18:57:33.446763 4162 raft_consensus.cc:348] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.447132 4162 raft_consensus.cc:374] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.445016 4169 ts_tablet_manager.cc:1905] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.447271 4162 raft_consensus.cc:729] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.447602 4162 consensus_queue.cc:260] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.448450 4162 ts_tablet_manager.cc:1428] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:33.449563 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 755) took 5192 ms. Trace:
I20260811 18:57:33.449759 4167 rpcz_store.cc:276] 0811 18:57:28.256963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.257015 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:33.368645 (+5111630us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:33.368660 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.386257 (+ 17597us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.430305 (+ 44048us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.449536 (+ 19231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5111403}
I20260811 18:57:33.450183 4167 tablet_replica.cc:330] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.450482 4167 raft_consensus.cc:2229] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.450661 4167 raft_consensus.cc:2258] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.451303 4167 ts_tablet_manager.cc:1905] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.458204 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4b6ec419ed7747c3b24dc11992518e88 (table default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]) successfully deleted
I20260811 18:57:33.459093 4167 ts_tablet_manager.cc:1918] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.459239 4167 log.cc:1196] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0b3ec38f9b55489ba00ed76f9e4456f5
I20260811 18:57:33.459543 4167 ts_tablet_manager.cc:1939] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.460115 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 761) took 5001 ms. Trace:
I20260811 18:57:33.460253 4167 rpcz_store.cc:276] 0811 18:57:28.459059 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.459178 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:33.450124 (+4990946us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:33.450139 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.455868 (+ 5729us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.458989 (+ 3121us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.460092 (+ 1103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4990642}
I20260811 18:57:33.460793 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
I20260811 18:57:33.460915 4162 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.462208 4163 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.462498 4163 ts_tablet_manager.cc:1397] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.154s user 0.010s sys 0.001s
I20260811 18:57:33.462764 4162 tablet_bootstrap.cc:654] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.463587 4163 raft_consensus.cc:348] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.463963 4163 raft_consensus.cc:374] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.464242 4163 raft_consensus.cc:729] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.465067 4163 consensus_queue.cc:260] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.466909 4163 ts_tablet_manager.cc:1428] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:33.470249 4163 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.463171 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 762) took 5003 ms. Trace:
I20260811 18:57:33.479283 4167 rpcz_store.cc:276] 0811 18:57:28.459254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.459503 (+ 249us) service_pool.cc:227] Handling call
0811 18:57:33.462997 (+5003494us) ts_tablet_manager.cc:1145] Deleting tablet d8146f47d9ae48889ac7efc954d4a6b7
0811 18:57:33.463012 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.463146 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5002805}
W20260811 18:57:33.480631 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d8146f47d9ae48889ac7efc954d4a6b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8146f47d9ae48889ac7efc954d4a6b7 already in progress: creating tablet
I20260811 18:57:33.480929 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 764) took 5007 ms. Trace:
I20260811 18:57:33.481062 4167 rpcz_store.cc:276] 0811 18:57:28.473894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.474478 (+ 584us) service_pool.cc:227] Handling call
0811 18:57:33.480730 (+5006252us) ts_tablet_manager.cc:1145] Deleting tablet a38c172a9fef4a3ab8ee188335a7c97e
0811 18:57:33.480751 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.480903 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4999848}
W20260811 18:57:33.481734 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a38c172a9fef4a3ab8ee188335a7c97e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38c172a9fef4a3ab8ee188335a7c97e already in progress: creating tablet
I20260811 18:57:33.487018 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 765) took 5002 ms. Trace:
I20260811 18:57:33.492159 4163 tablet_bootstrap.cc:654] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.494534 4167 rpcz_store.cc:276] 0811 18:57:28.484583 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.485058 (+ 475us) service_pool.cc:227] Handling call
0811 18:57:33.486807 (+5001749us) ts_tablet_manager.cc:1145] Deleting tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
0811 18:57:33.486830 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.486983 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5001095}
W20260811 18:57:33.495693 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c already in progress: creating tablet
I20260811 18:57:33.495927 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 766) took 4995 ms. Trace:
I20260811 18:57:33.497040 4167 rpcz_store.cc:276] 0811 18:57:28.500080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.500954 (+ 874us) service_pool.cc:227] Handling call
0811 18:57:33.495750 (+4994796us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:33.495770 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.495904 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4994199}
W20260811 18:57:33.499393 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cf6c8d542ae2489ba466ffadd518db12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf6c8d542ae2489ba466ffadd518db12 already in progress: creating tablet
I20260811 18:57:33.499502 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 769) took 4877 ms. Trace:
I20260811 18:57:33.499609 4167 rpcz_store.cc:276] 0811 18:57:28.622502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.622583 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:33.499342 (+4876759us) ts_tablet_manager.cc:1145] Deleting tablet f13a14bd64704798bf6de4fc05201a31
0811 18:57:33.499358 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.499482 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4876511}
W20260811 18:57:33.500232 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f13a14bd64704798bf6de4fc05201a31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13a14bd64704798bf6de4fc05201a31 already in progress: creating tablet
I20260811 18:57:33.498728 4162 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.500466 4162 ts_tablet_manager.cc:1397] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260811 18:57:33.501868 4162 raft_consensus.cc:348] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.506153 4162 raft_consensus.cc:374] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.510542 4162 raft_consensus.cc:729] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.514739 4162 consensus_queue.cc:260] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.519073 4162 ts_tablet_manager.cc:1428] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260811 18:57:33.519778 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c3c124426696404c9e9cb5200f63055e" } new_table_name: "default.Table_c12987db235f4f4fb949580479275ec2" modify_external_catalogs: true
I20260811 18:57:33.520273 4162 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.502707 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 779) took 4782 ms. Trace:
I20260811 18:57:33.520532 4167 rpcz_store.cc:276] 0811 18:57:28.720461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.720825 (+ 364us) service_pool.cc:227] Handling call
0811 18:57:33.502496 (+4781671us) ts_tablet_manager.cc:1145] Deleting tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
0811 18:57:33.502513 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.502681 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4781477}
I20260811 18:57:33.521147 4167 tablet_replica.cc:330] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.521436 4167 raft_consensus.cc:2229] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:33.521539 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 already in progress: creating tablet
I20260811 18:57:33.521835 4167 raft_consensus.cc:2258] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.522702 4167 ts_tablet_manager.cc:1905] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.524720 3678 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.417s (SENT)
I20260811 18:57:33.524858 3678 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:33.525027 3678 raft_consensus.cc:2670] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
#12 0xb1ad42 in kudu::tools::DispatchCommand(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::deque<std::string, std::allocator<std::string> > const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:138:15
#13 0xb1c3b9 in kudu::tools::RunTool(int, char**, bool) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:210:16
#14 0xb1d0c2 in main /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:305:10
#15 0x7fc7af40fc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
Thread T137 (tablet-delete [) created by T59 (rpc worker-4011) here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b32befd7 in kudu::ThreadPool::CreateThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:848:10
#4 0x7fc7b32babea in kudu::ThreadPool::DoSubmit(std::function<void ()>, kudu::ThreadPoolToken*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:643:21
I20260811 18:57:33.550895 5376 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.551236 5376 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.552475 5376 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:33.553284 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:33.554071 4162 tablet_bootstrap.cc:654] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.554224 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet f7d4c26da86b49b8a37f2289f4ef6614 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.555148 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.555449 3740 raft_consensus.cc:2454] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.556123 3824 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.556692 5286 raft_consensus.cc:2781] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.556811 5286 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.556888 5286 raft_consensus.cc:3037] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.559114 4162 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.559270 4162 ts_tablet_manager.cc:1397] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.000s
I20260811 18:57:33.560313 4162 raft_consensus.cc:348] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.560591 4162 raft_consensus.cc:374] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.560663 4162 raft_consensus.cc:729] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.560812 5286 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.561564 5286 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.563465 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.563874 3743 raft_consensus.cc:3037] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.565260 4163 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.565423 4163 ts_tablet_manager.cc:1397] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.095s user 0.000s sys 0.005s
I20260811 18:57:33.566329 4163 raft_consensus.cc:348] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.566587 4163 raft_consensus.cc:374] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.566685 4163 raft_consensus.cc:729] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.567001 4163 consensus_queue.cc:260] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.567689 5376 consensus_queue.cc:1035] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.568746 3743 raft_consensus.cc:2454] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.571332 3824 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.571658 5286 raft_consensus.cc:2781] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.571779 5286 raft_consensus.cc:686] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.572008 5286 consensus_queue.cc:237] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.577669 4162 consensus_queue.cc:260] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.579190 4162 ts_tablet_manager.cc:1428] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20260811 18:57:33.498208 4169 ts_tablet_manager.cc:1918] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.581185 4169 log.cc:1196] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:33.581928 4162 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.582118 4169 ts_tablet_manager.cc:1939] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.582906 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 797) took 3438 ms. Trace:
I20260811 18:57:33.581071 4167 ts_tablet_manager.cc:1918] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.587164 4167 log.cc:1196] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:33.588024 4167 ts_tablet_manager.cc:1939] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.589118 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 780) took 4783 ms. Trace:
I20260811 18:57:33.594558 4167 rpcz_store.cc:276] 0811 18:57:28.805314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.805387 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:33.521084 (+4715697us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:33.521100 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.547667 (+ 26567us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.580929 (+ 33262us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.589096 (+ 8167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4715507}
#5 0x7fc7b32bfcee in kudu::ThreadPool::Submit(std::function<void ()>) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:525:10
#6 0x7fc7c23e5265 in kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1121:35
I20260811 18:57:33.599836 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 515d5b5afbd74ec5ab90a503aef2f4fb (table default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]) successfully deleted
I20260811 18:57:33.600036 4167 tablet_replica.cc:330] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.600346 4167 raft_consensus.cc:2229] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.601272 4167 raft_consensus.cc:2258] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.596465 4162 tablet_bootstrap.cc:654] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#7 0x7fc7c23337bb in kudu::tserver::TabletServiceAdminImpl::DeleteTablet(kudu::tserver::DeleteTabletRequestPB const*, kudu::tserver::DeleteTabletResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1532:30
I20260811 18:57:33.609658 4167 ts_tablet_manager.cc:1905] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#8 0x7fc7bc95b926 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_3::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:215:13
#9 0x7fc7bc95b655 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_3>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
#10 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
#12 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#13 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#14 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Thread T59 (rpc worker-4011) created by T0 here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b9db98c6 in kudu::rpc::ServicePool::Init(int) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#4 0x7fc7bc1faa67 in kudu::RpcServer::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:238:3
#5 0x7fc7bc21b0b7 in kudu::server::ServerBase::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:1171:23
#6 0x7fc7c23171a3 in kudu::tserver::TabletServer::Start() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server.cc:157:3
#7 0x7fc7c2326380 in kudu::tserver::RunTabletServer() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server_runner.cc:86:3
#8 0xae894d in kudu::tools::(anonymous namespace)::TServerRun(kudu::tools::RunnerContext const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action_tserver.cc:114:10
#9 0x921350 in std::_Function_handler<kudu::Status (kudu::tools::RunnerContext const&), kudu::Status (*)(kudu::tools::RunnerContext const&)>::_M_invoke(std::_Any_data const&, kudu::tools::RunnerContext const&) ../../../include/c++/8/bits/std_function.h:282:9
#10 0x7fc7c6b82128 in std::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7c6b7dcc5 in kudu::tools::Action::Run(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action.cc:245:10
#12 0xb1ad42 in kudu::tools::DispatchCommand(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::deque<std::string, std::allocator<std::string> > const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:138:15
#13 0xb1c3b9 in kudu::tools::RunTool(int, char**, bool) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:210:16
#14 0xb1d0c2 in main /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:305:10
#15 0x7fc7af40fc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
Thread T135 (tablet-open [wo) created by T59 (rpc worker-4011) here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
I20260811 18:57:33.624686 4162 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b32befd7 in kudu::ThreadPool::CreateThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:848:10
#4 0x7fc7b32babea in kudu::ThreadPool::DoSubmit(std::function<void ()>, kudu::ThreadPoolToken*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:643:21
I20260811 18:57:33.625213 4162 ts_tablet_manager.cc:1397] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.002s
#5 0x7fc7b32bfcee in kudu::ThreadPool::Submit(std::function<void ()>) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:525:10
#6 0x7fc7c23d9e06 in kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:703:3
#7 0x7fc7c233140b in kudu::tserver::TabletServiceAdminImpl::CreateTablet(kudu::tserver::CreateTabletRequestPB const*, kudu::tserver::CreateTabletResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1474:34
#8 0x7fc7bc95aac6 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:189:13
#9 0x7fc7bc95a7f5 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:33.626631 4162 raft_consensus.cc:348] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#10 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:33.584052 5334 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.627296 5334 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.628051 5334 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:33.588132 3824 consensus_peers.cc:487] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b3ec38f9b55489ba00ed76f9e4456f5. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.627202 4169 rpcz_store.cc:276] 0811 18:57:30.144329 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.330528 (+186199us) service_pool.cc:227] Handling call
0811 18:57:33.416641 (+3086113us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:33.416659 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.481235 (+ 64576us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.498045 (+ 16810us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.582878 (+ 84833us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3018892}
I20260811 18:57:33.632076 5334 consensus_queue.cc:1035] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.632826 4169 tablet_replica.cc:330] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.632772 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b0b199f4c9e14b468cb26cbd61fc8586 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:33.633165 4169 raft_consensus.cc:2229] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.633476 4169 raft_consensus.cc:2258] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.606462 4163 ts_tablet_manager.cc:1428] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.041s user 0.002s sys 0.001s
I20260811 18:57:33.634573 4163 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.640077 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" } modify_external_catalogs: true
I20260811 18:57:33.640378 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" } modify_external_catalogs: true
I20260811 18:57:33.640599 4162 raft_consensus.cc:374] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.640730 4162 raft_consensus.cc:729] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.630622 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.641141 3745 raft_consensus.cc:2454] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.641364 4162 consensus_queue.cc:260] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.641705 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9409f5b4b0034412aedc183b302556a3" } new_table_name: "default.Table_7b329f3d8a5543659bfb5de93453f46b" modify_external_catalogs: true
W20260811 18:57:33.643556 3678 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.582s (SENT)
I20260811 18:57:33.643925 3824 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.644281 5376 raft_consensus.cc:2781] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.644376 5376 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.644474 5376 raft_consensus.cc:3037] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.644950 4162 ts_tablet_manager.cc:1428] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:33.645263 4162 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.645404 4169 ts_tablet_manager.cc:1905] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.647699 5376 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.656306 5376 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:33.657438 3824 leader_election.cc:334] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.511s (SENT)
I20260811 18:57:33.657620 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7065b527f8104608b1efc6787f0b2b77" } new_table_name: "default.Table_ed17e76c67444161b978d62bf0bef64f" modify_external_catalogs: true
I20260811 18:57:33.659535 4163 tablet_bootstrap.cc:654] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.659780 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f" } modify_external_catalogs: true
I20260811 18:57:33.660073 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f" } modify_external_catalogs: true
I20260811 18:57:33.662159 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.663923 3745 raft_consensus.cc:3037] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.667488 3745 raft_consensus.cc:2454] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.667968 4167 ts_tablet_manager.cc:1918] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.668862 3824 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.669116 4167 log.cc:1196] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:33.669243 5376 raft_consensus.cc:2781] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.669528 5376 raft_consensus.cc:686] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.669750 4167 ts_tablet_manager.cc:1939] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.669926 5376 consensus_queue.cc:237] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.670799 5334 consensus_queue.cc:1035] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.674865 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 784) took 4754 ms. Trace:
I20260811 18:57:33.677819 5443 raft_consensus.cc:482] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.678632 5443 raft_consensus.cc:504] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.679821 5443 leader_election.cc:288] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.680074 4167 rpcz_store.cc:276] 0811 18:57:28.920764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.922506 (+ 1742us) service_pool.cc:227] Handling call
0811 18:57:33.599959 (+4677453us) ts_tablet_manager.cc:1145] Deleting tablet 0ba6246e6d29418cbca5f751b34f4ef1
0811 18:57:33.599979 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.630331 (+ 30352us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.667831 (+ 37500us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.674842 (+ 7011us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4660485}
I20260811 18:57:33.680220 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fde25bcb40493aafb083e161aa6932" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.681089 3676 leader_election.cc:341] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.681097 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ba6246e6d29418cbca5f751b34f4ef1 (table default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]) successfully deleted
I20260811 18:57:33.681694 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 786) took 4757 ms. Trace:
I20260811 18:57:33.683720 4169 ts_tablet_manager.cc:1918] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.685214 4167 rpcz_store.cc:276] 0811 18:57:28.924040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924102 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:33.681522 (+4757420us) ts_tablet_manager.cc:1145] Deleting tablet 2474ae972e9242b69f488b7a2fdbc481
0811 18:57:33.681540 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.681670 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4741920}
W20260811 18:57:33.686237 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2474ae972e9242b69f488b7a2fdbc481 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2474ae972e9242b69f488b7a2fdbc481 already in progress: creating tablet
I20260811 18:57:33.686389 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 785) took 4765 ms. Trace:
I20260811 18:57:33.686491 4167 rpcz_store.cc:276] 0811 18:57:28.921008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924270 (+ 3262us) service_pool.cc:227] Handling call
0811 18:57:33.686223 (+4761953us) ts_tablet_manager.cc:1145] Deleting tablet b4dc2df4c6464d71bb13fe25e0046758
0811 18:57:33.686239 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.686368 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4746373}
I20260811 18:57:33.687228 4169 log.cc:1196] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:33.688666 4169 ts_tablet_manager.cc:1939] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.688844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 787) took 4763 ms. Trace:
I20260811 18:57:33.689357 4167 rpcz_store.cc:276] 0811 18:57:28.925472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.925537 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:33.688690 (+4763153us) ts_tablet_manager.cc:1145] Deleting tablet 8e120a45539f49bb82ab0fb95eac36f9
0811 18:57:33.688704 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.688820 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4748649}
I20260811 18:57:33.690233 4167 tablet_replica.cc:330] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.692018 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 800) took 3187 ms. Trace:
W20260811 18:57:33.692826 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b4dc2df4c6464d71bb13fe25e0046758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4dc2df4c6464d71bb13fe25e0046758 already in progress: creating tablet
W20260811 18:57:33.693235 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8e120a45539f49bb82ab0fb95eac36f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e120a45539f49bb82ab0fb95eac36f9 already in progress: creating tablet
I20260811 18:57:33.696329 4169 rpcz_store.cc:276] 0811 18:57:30.504847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.504903 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:33.632760 (+3127857us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:33.632778 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.651359 (+ 18581us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.683601 (+ 32242us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.691989 (+ 8388us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3080401}
I20260811 18:57:33.698413 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 02808605ee6c4634af47fc3fd57e9cd8 (table default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]) successfully deleted
I20260811 18:57:33.698290 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 799) took 3193 ms. Trace:
I20260811 18:57:33.700527 4169 rpcz_store.cc:276] 0811 18:57:30.504654 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.504728 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:33.698141 (+3193413us) ts_tablet_manager.cc:1145] Deleting tablet 85140a3800f34a23b716703fd4e73d17
0811 18:57:33.698156 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.698271 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34,"tablet-delete.queue_time_us":3145774}
I20260811 18:57:33.700984 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 798) took 3196 ms. Trace:
I20260811 18:57:33.701064 4169 rpcz_store.cc:276] 0811 18:57:30.504451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.505484 (+ 1033us) service_pool.cc:227] Handling call
0811 18:57:33.700853 (+3195369us) ts_tablet_manager.cc:1145] Deleting tablet 98ad733726594d2a8087f857f155dbe2
0811 18:57:33.700865 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.700967 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3144586}
W20260811 18:57:33.701231 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 85140a3800f34a23b716703fd4e73d17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85140a3800f34a23b716703fd4e73d17 already in progress: creating tablet
I20260811 18:57:33.701303 4169 tablet_replica.cc:330] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.701483 4169 raft_consensus.cc:2229] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.701838 4169 raft_consensus.cc:2258] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:33.702039 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 98ad733726594d2a8087f857f155dbe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98ad733726594d2a8087f857f155dbe2 already in progress: creating tablet
I20260811 18:57:33.703768 4167 raft_consensus.cc:2229] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.704353 4167 pending_rounds.cc:62] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:33.704480 4167 pending_rounds.cc:68] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400177733836800 op_type: NO_OP noop_request { }
I20260811 18:57:33.704732 4167 raft_consensus.cc:2866] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.704922 4167 raft_consensus.cc:2258] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.706498 4167 ts_tablet_manager.cc:1905] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.710299 3824 consensus_peers.cc:487] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.710600 4169 ts_tablet_manager.cc:1905] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.715847 4163 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.716164 4163 ts_tablet_manager.cc:1397] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.000s
I20260811 18:57:33.716956 4163 raft_consensus.cc:348] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.717583 4163 raft_consensus.cc:374] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.717724 4163 raft_consensus.cc:729] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.718031 4163 consensus_queue.cc:260] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.722553 4163 ts_tablet_manager.cc:1428] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:57:33.723166 4163 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
#11 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
#12 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#13 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#14 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
SUMMARY: AddressSanitizer: heap-use-after-free /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/atomicops-internals-x86.h:200:10 in base::subtle::NoBarrier_Load(int const volatile*)
Shadow bytes around the buggy address:
0x0c328000fee0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000fef0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c328000ff20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c328000ff30: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
W20260811 18:57:33.725891 3966 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 28.062s user 1.075s sys 2.859s
W20260811 18:57:33.726989 4378 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:05.663892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:38889 (local address 127.3.72.67:39173)
0811 18:57:33.726206 (+28062314us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:33.726395 (+ 189us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.72.125:38889: Timeout exceeded waiting to connect
Metrics: {"client-negotiator.queue_time_us":28062104,"threads_started":1}
W20260811 18:57:33.727398 3966 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 28.063s user 1.075s sys 2.859s
I20260811 18:57:33.733657 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc16d2310b294ccb8c18945b693c6fce" } new_table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" modify_external_catalogs: true
I20260811 18:57:33.735702 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870] alter complete (version 2)
I20260811 18:57:33.737002 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6" } modify_external_catalogs: true
I20260811 18:57:33.737331 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6" } modify_external_catalogs: true
I20260811 18:57:33.739962 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 2c0f6df7a291456d80538e2b76417873 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.740785 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ee627ac9abf47889b5fed972cecd97e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.741433 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ee627ac9abf47889b5fed972cecd97e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.743157 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.745592 4163 tablet_bootstrap.cc:654] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
==3947==ABORTING
W20260811 18:57:33.764217 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae consensus state has no leader
I20260811 18:57:33.779249 4167 ts_tablet_manager.cc:1918] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.779585 4167 log.cc:1196] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:33.780247 4167 ts_tablet_manager.cc:1939] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.781833 4163 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.782011 4163 ts_tablet_manager.cc:1397] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.059s user 0.007s sys 0.002s
I20260811 18:57:33.783020 4163 raft_consensus.cc:348] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.783288 4163 raft_consensus.cc:374] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.783368 4163 raft_consensus.cc:729] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.783587 4163 consensus_queue.cc:260] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.783990 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 788) took 4854 ms. Trace:
I20260811 18:57:33.784150 4167 rpcz_store.cc:276] 0811 18:57:28.929636 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.929697 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:33.690171 (+4760474us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:33.690187 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.704242 (+ 14055us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x6150001453a4. stack: 00007fcdd297d66e 00007fcdd297d430 00007fcdd1a045ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fcdddeb504d 00007fcddf1595ec 00007fcde1ae82e6 00007fcde1b1a404 00007fcde1afb870 00007fcde1afb5fd 00000000009edb7c 00007fcdd29c3344 00007fcdd29c691b 00007fcdd29c6798
0811 18:57:33.728949 (+ 24707us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.779130 (+ 50181us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.783968 (+ 4838us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":29714176,"tablet-delete.queue_time_us":4750062}
I20260811 18:57:33.784976 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 783) took 4864 ms. Trace:
I20260811 18:57:33.784976 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet fe203387bc9c45ba82afe6b058aa5e5f (table default.Table_e67c791ed9cd4da88ae7ce53ba362301 [id=14b4d412d33c43fdb3d20c85b82c1670]) successfully deleted
I20260811 18:57:33.785112 4167 rpcz_store.cc:276] 0811 18:57:28.920626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924679 (+ 4053us) service_pool.cc:227] Handling call
0811 18:57:33.784860 (+4860181us) ts_tablet_manager.cc:1145] Deleting tablet b203df4124364737839bf4c1ae072988
0811 18:57:33.784871 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.784961 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4844665}
I20260811 18:57:33.785674 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 789) took 4713 ms. Trace:
W20260811 18:57:33.785645 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b203df4124364737839bf4c1ae072988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b203df4124364737839bf4c1ae072988 already in progress: creating tablet
I20260811 18:57:33.785751 4167 rpcz_store.cc:276] 0811 18:57:29.072014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.072389 (+ 375us) service_pool.cc:227] Handling call
0811 18:57:33.785579 (+4713190us) ts_tablet_manager.cc:1145] Deleting tablet c77882ff8d974057a61f037f1943f58f
0811 18:57:33.785589 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.785661 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1540,"tablet-delete.queue_time_us":4712927}
I20260811 18:57:33.786248 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 801) took 4278 ms. Trace:
I20260811 18:57:33.786331 4167 rpcz_store.cc:276] 0811 18:57:29.508132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.508423 (+ 291us) service_pool.cc:227] Handling call
0811 18:57:33.786154 (+4277731us) ts_tablet_manager.cc:1145] Deleting tablet 6880ad27a3eb4599b407edeb2969a08f
0811 18:57:33.786164 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.786232 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4277474}
W20260811 18:57:33.788066 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6880ad27a3eb4599b407edeb2969a08f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6880ad27a3eb4599b407edeb2969a08f already in progress: creating tablet
I20260811 18:57:33.788067 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 802) took 4279 ms. Trace:
I20260811 18:57:33.788172 4167 rpcz_store.cc:276] 0811 18:57:29.508228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.508308 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:33.787962 (+4279654us) ts_tablet_manager.cc:1145] Deleting tablet 1d721b0e55384d8a8efd88a0e47b1504
0811 18:57:33.787972 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.788050 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4279259}
W20260811 18:57:33.788700 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1d721b0e55384d8a8efd88a0e47b1504 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d721b0e55384d8a8efd88a0e47b1504 already in progress: creating tablet
I20260811 18:57:33.789578 4163 ts_tablet_manager.cc:1428] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.791754 4163 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.792513 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 800) took 4284 ms. Trace:
I20260811 18:57:33.794464 4162 tablet_bootstrap.cc:654] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.792608 4167 rpcz_store.cc:276] 0811 18:57:29.508029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.509625 (+ 1596us) service_pool.cc:227] Handling call
0811 18:57:33.792408 (+4282783us) ts_tablet_manager.cc:1145] Deleting tablet 1c74abe9dcf642e9b4d40df6ff3639ef
0811 18:57:33.792418 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.792497 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4282447}
I20260811 18:57:33.796656 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 799) took 4288 ms. Trace:
I20260811 18:57:33.796757 4167 rpcz_store.cc:276] 0811 18:57:29.507902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.510017 (+ 2115us) service_pool.cc:227] Handling call
0811 18:57:33.796536 (+4286519us) ts_tablet_manager.cc:1145] Deleting tablet 47a782b8470b4d14ac89b3b7d463a125
0811 18:57:33.796549 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.796638 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4286265}
W20260811 18:57:33.796880 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1c74abe9dcf642e9b4d40df6ff3639ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c74abe9dcf642e9b4d40df6ff3639ef already in progress: creating tablet
I20260811 18:57:33.797160 4167 tablet_replica.cc:330] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:33.797266 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 47a782b8470b4d14ac89b3b7d463a125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47a782b8470b4d14ac89b3b7d463a125 already in progress: creating tablet
I20260811 18:57:33.797380 4167 raft_consensus.cc:2229] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.797544 4167 raft_consensus.cc:2258] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.798103 4167 ts_tablet_manager.cc:1905] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.802093 5443 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.802228 5443 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.802842 5443 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.803591 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.806191 3676 leader_election.cc:341] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.806236 5376 consensus_queue.cc:1035] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.808905 4163 tablet_bootstrap.cc:654] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:33.812261 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.667s (SENT) [suppressed 29 similar messages]
W20260811 18:57:33.820340 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
I20260811 18:57:33.828589 4163 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.829063 4163 ts_tablet_manager.cc:1397] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.003s
I20260811 18:57:33.810165 4169 ts_tablet_manager.cc:1918] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.830673 4169 log.cc:1196] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/490f34d7a403416fbab64e46973746de
I20260811 18:57:33.831197 4162 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.831341 4162 ts_tablet_manager.cc:1397] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.186s user 0.022s sys 0.005s
I20260811 18:57:33.831964 4162 raft_consensus.cc:348] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.832154 4162 raft_consensus.cc:374] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.832204 4162 raft_consensus.cc:729] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.832396 4162 consensus_queue.cc:260] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.832902 4162 ts_tablet_manager.cc:1428] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 18:57:33.833145 4162 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.833536 4167 ts_tablet_manager.cc:1918] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.833635 4167 log.cc:1196] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:33.833925 4167 ts_tablet_manager.cc:1939] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.834395 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 798) took 4326 ms. Trace:
I20260811 18:57:33.834479 4167 rpcz_store.cc:276] 0811 18:57:29.507635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.510310 (+ 2675us) service_pool.cc:227] Handling call
0811 18:57:33.797099 (+4286789us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:33.797110 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.801333 (+ 4223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.833439 (+ 32106us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.834374 (+ 935us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4286523}
I20260811 18:57:33.835143 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 1e1374a80d15444cb0faff339275aa8d (table default.Table_647ae82e63464976abb7f26c9526a349 [id=e0ef202b868f47269e4f2a63cb5a390a]) successfully deleted
I20260811 18:57:33.836016 4162 tablet_bootstrap.cc:654] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:33.840632 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae96e10a2ed4407c9d42a359d82d389e consensus state has no leader
I20260811 18:57:33.841249 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4464846117e74da2b263ff545f9937b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.841977 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4464846117e74da2b263ff545f9937b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.852567 4163 raft_consensus.cc:348] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.852831 4163 raft_consensus.cc:374] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.852893 4163 raft_consensus.cc:729] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.852772 4169 ts_tablet_manager.cc:1939] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.855803 4163 consensus_queue.cc:260] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.858438 4163 ts_tablet_manager.cc:1428] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260811 18:57:33.859513 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 801) took 3137 ms. Trace:
I20260811 18:57:33.860569 4163 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.860381 4169 rpcz_store.cc:276] 0811 18:57:30.720438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.720499 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:33.701264 (+2980765us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:33.701272 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.715816 (+ 14544us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.810026 (+ 94210us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.858236 (+ 48210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2966965}
I20260811 18:57:33.863093 4167 tablet_replica.cc:330] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.863423 4167 raft_consensus.cc:2229] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.863612 4167 raft_consensus.cc:2258] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.864020 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 490f34d7a403416fbab64e46973746de (table default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]) successfully deleted
I20260811 18:57:33.864307 4167 ts_tablet_manager.cc:1905] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.866159 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 825) took 2983 ms. Trace:
I20260811 18:57:33.870428 4163 tablet_bootstrap.cc:654] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.866896 4169 rpcz_store.cc:276] 0811 18:57:30.882816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.020544 (+137728us) service_pool.cc:227] Handling call
0811 18:57:33.865060 (+2844516us) ts_tablet_manager.cc:1145] Deleting tablet 22abc6f8e9ef4c32ab69af31bdaad872
0811 18:57:33.865075 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.865863 (+ 788us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2843896}
I20260811 18:57:33.872179 4162 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:33.872330 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 22abc6f8e9ef4c32ab69af31bdaad872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22abc6f8e9ef4c32ab69af31bdaad872 already in progress: creating tablet
I20260811 18:57:33.872416 4162 ts_tablet_manager.cc:1397] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.007s
I20260811 18:57:33.873219 4162 raft_consensus.cc:348] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.873567 4162 raft_consensus.cc:374] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.873694 4162 raft_consensus.cc:729] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.874054 4162 consensus_queue.cc:260] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.875982 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e50906e16414e8b99c43763a538d6be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.876745 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e50906e16414e8b99c43763a538d6be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:33.877959 3824 leader_election.cc:334] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.733s (SENT)
I20260811 18:57:33.877995 4162 ts_tablet_manager.cc:1428] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:33.878278 4162 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.879966 4162 tablet_bootstrap.cc:654] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.884321 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 826) took 3001 ms. Trace:
I20260811 18:57:33.885830 4167 ts_tablet_manager.cc:1918] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.885957 4167 log.cc:1196] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6f3060822021431584a494e6941be756
I20260811 18:57:33.885433 4169 rpcz_store.cc:276] 0811 18:57:30.882927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.020952 (+138025us) service_pool.cc:227] Handling call
0811 18:57:33.881640 (+2860688us) ts_tablet_manager.cc:1145] Deleting tablet 3030c31bc2b24ee296f29b5d5aa402ff
0811 18:57:33.881936 (+ 296us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.884094 (+ 2158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2848775}
W20260811 18:57:33.890143 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3030c31bc2b24ee296f29b5d5aa402ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3030c31bc2b24ee296f29b5d5aa402ff already in progress: creating tablet
I20260811 18:57:33.891891 4163 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.892095 4163 ts_tablet_manager.cc:1397] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.000s
I20260811 18:57:33.892359 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 828) took 3009 ms. Trace:
I20260811 18:57:33.893006 4163 raft_consensus.cc:348] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.893302 4163 raft_consensus.cc:374] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.893415 4163 raft_consensus.cc:729] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.892791 4169 rpcz_store.cc:276] 0811 18:57:30.883203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.029533 (+146330us) service_pool.cc:227] Handling call
0811 18:57:33.891164 (+2861631us) ts_tablet_manager.cc:1145] Deleting tablet 90fde17971ea49aaad212a5836661b08
0811 18:57:33.891568 (+ 404us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.892177 (+ 609us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2860849}
I20260811 18:57:33.893728 4163 consensus_queue.cc:260] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.894284 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 90fde17971ea49aaad212a5836661b08 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90fde17971ea49aaad212a5836661b08 already in progress: creating tablet
I20260811 18:57:33.897133 4163 ts_tablet_manager.cc:1428] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:33.897446 4163 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.899220 4163 tablet_bootstrap.cc:654] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.895689 4162 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.901515 4162 ts_tablet_manager.cc:1397] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
W20260811 18:57:33.902199 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae96e10a2ed4407c9d42a359d82d389e (table default.Table_60eefdf4cac142ff98c75bfd637b21af [id=b28c814fee894ee7a12d05872eea8e92], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae96e10a2ed4407c9d42a359d82d389e consensus state has no leader
I20260811 18:57:33.902187 4162 raft_consensus.cc:348] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.902539 4162 raft_consensus.cc:374] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.902667 4162 raft_consensus.cc:729] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.902935 4162 consensus_queue.cc:260] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.886435 4167 ts_tablet_manager.cc:1939] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.907119 4162 ts_tablet_manager.cc:1428] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 18:57:33.907516 4162 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.909569 4162 tablet_bootstrap.cc:654] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.910290 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 808) took 4381 ms. Trace:
I20260811 18:57:33.910422 4167 rpcz_store.cc:276] 0811 18:57:29.529231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.529290 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:33.863032 (+4333742us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:33.863047 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.879744 (+ 16697us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.885728 (+ 5984us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.908221 (+ 22493us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x61500011f6a4. stack: 00007fcdd297d66e 00007fcdd297d430 00007fcdd1a045ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fcdddeb4edd 00007fcddde74de4 00007fcddf15b426 00007fcde1ae86e3 00007fcde1b1a404 00007fcde1afb870 00007fcde1afb5fd 00000000009edb7c 00007fcdd29c3344 00007fcdd29c691b
0811 18:57:33.910269 (+ 2048us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5452672,"tablet-delete.queue_time_us":4333516}
I20260811 18:57:33.910976 4167 tablet_replica.cc:330] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.910925 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6f3060822021431584a494e6941be756 (table default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]) successfully deleted
I20260811 18:57:33.911229 4167 raft_consensus.cc:2229] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.911417 4167 raft_consensus.cc:2258] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.911990 4167 ts_tablet_manager.cc:1905] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.914057 4163 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.914213 4163 ts_tablet_manager.cc:1397] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260811 18:57:33.915020 4163 raft_consensus.cc:348] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.915257 4163 raft_consensus.cc:374] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.915323 4163 raft_consensus.cc:729] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.915544 4163 consensus_queue.cc:260] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.916285 4163 ts_tablet_manager.cc:1428] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.915647 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 829) took 3024 ms. Trace:
I20260811 18:57:33.920020 3548 catalog_manager.cc:5495] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:33.919145 4169 rpcz_store.cc:276] 0811 18:57:30.883305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.030513 (+147208us) service_pool.cc:227] Handling call
0811 18:57:33.896111 (+2865598us) ts_tablet_manager.cc:1145] Deleting tablet 7d747f03ad234805a4ea4ef60840d426
0811 18:57:33.896124 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.908033 (+ 11909us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2864808}
I20260811 18:57:33.920805 4163 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:33.921797 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c6852aaef1d5486b87e5462eae532719 consensus state has no leader
I20260811 18:57:33.922799 4163 tablet_bootstrap.cc:654] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.923959 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f08236e0af834f29abafce228034a18a (DEFAULT_TABLE table=default.Table_a17ac8f677ce46efaade718a851a016d [id=41b250c044d94c37879aa50e57415f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924014 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0128fecf972b48c2973a5c36e997f71b (DEFAULT_TABLE table=default.Table_a7bc3ed83682403b84a5a23d14cf1c3b [id=4278f7ec09d147ae8f8441f33bc4a6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924304 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0128fecf972b48c2973a5c36e997f71b (DEFAULT_TABLE table=default.Table_a7bc3ed83682403b84a5a23d14cf1c3b [id=4278f7ec09d147ae8f8441f33bc4a6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924343 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 82b04e37fc264fc08e0ed9517356a17d (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924459 3855 tablet_service.cc:1467] Processing CreateTablet for tablet fc967ce6d10b49f09731f6326cbf066c (DEFAULT_TABLE table=default.Table_7e97d2b678784537a3e986a742e9f88b [id=1bdec829404a41ad84ab102c515ac0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924695 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08236e0af834f29abafce228034a18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924727 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0128fecf972b48c2973a5c36e997f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924907 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc967ce6d10b49f09731f6326cbf066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924863 3720 tablet_service.cc:1467] Processing CreateTablet for tablet f08236e0af834f29abafce228034a18a (DEFAULT_TABLE table=default.Table_a17ac8f677ce46efaade718a851a016d [id=41b250c044d94c37879aa50e57415f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.925153 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0128fecf972b48c2973a5c36e997f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.925212 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08236e0af834f29abafce228034a18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.926587 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 82b04e37fc264fc08e0ed9517356a17d (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.926908 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b04e37fc264fc08e0ed9517356a17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.930109 3719 tablet_service.cc:1467] Processing CreateTablet for tablet fc967ce6d10b49f09731f6326cbf066c (DEFAULT_TABLE table=default.Table_7e97d2b678784537a3e986a742e9f88b [id=1bdec829404a41ad84ab102c515ac0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.930481 5443 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.930612 5443 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.931236 5443 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.931545 5417 raft_consensus.cc:482] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.931676 5417 raft_consensus.cc:504] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.922255 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9d18547a0114b83bfbed5110e565661" } modify_external_catalogs: true
I20260811 18:57:33.932085 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9d18547a0114b83bfbed5110e565661" } modify_external_catalogs: true
I20260811 18:57:33.932211 5417 leader_election.cc:288] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.932413 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b04e37fc264fc08e0ed9517356a17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.937976 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.938644 3676 leader_election.cc:341] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.939127 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3263da75f0d84528b7c8cc55dcaeaa46" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.939343 3743 raft_consensus.cc:2454] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.940604 3724 tablet_service.cc:1467] Processing CreateTablet for tablet de6245131a3a4513b88611fd4c2e031c (DEFAULT_TABLE table=default.Table_db613e17da86441bb1ab37b46e730db9 [id=d0c9376b40d34652839aec9e4e711def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.930490 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc967ce6d10b49f09731f6326cbf066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.944483 4167 ts_tablet_manager.cc:1918] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.944605 4167 log.cc:1196] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:33.945339 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e1922c7317c24aa984945aaaa6ce8003 (DEFAULT_TABLE table=default.Table_3309d84857df4970a9891fd64c181bf5 [id=24a68c86a16a494a8d4b2108c7b06c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.946237 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1922c7317c24aa984945aaaa6ce8003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.946593 3705 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4d5464594c0eb19f1739c29c0b45 (DEFAULT_TABLE table=default.Table_57453b6f4942454da731710b8510f4a7 [id=c1e2053ce9d340659a6dd593465dea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.946919 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4d5464594c0eb19f1739c29c0b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.948735 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 827) took 3065 ms. Trace:
I20260811 18:57:33.948836 4169 rpcz_store.cc:276] 0811 18:57:30.883024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.024982 (+141958us) service_pool.cc:227] Handling call
0811 18:57:33.948559 (+2923577us) ts_tablet_manager.cc:1145] Deleting tablet 6009c9f6bbb0493ebe125120a1a47f3f
0811 18:57:33.948574 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.948709 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2914587}
I20260811 18:57:33.949154 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 836) took 2690 ms. Trace:
I20260811 18:57:33.949216 4169 rpcz_store.cc:276] 0811 18:57:31.259151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.259756 (+ 605us) service_pool.cc:227] Handling call
0811 18:57:33.949068 (+2689312us) ts_tablet_manager.cc:1145] Deleting tablet c944d5cfd9b3405b8151abfc9f2fd998
0811 18:57:33.949077 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949141 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2685784}
I20260811 18:57:33.949487 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 835) took 2690 ms. Trace:
I20260811 18:57:33.949558 4169 rpcz_store.cc:276] 0811 18:57:31.259055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.261289 (+ 2234us) service_pool.cc:227] Handling call
0811 18:57:33.949390 (+2688101us) ts_tablet_manager.cc:1145] Deleting tablet 01a5fdab68694761b7493b42a0bf7c5b
0811 18:57:33.949399 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949472 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659116}
I20260811 18:57:33.945346 4167 ts_tablet_manager.cc:1939] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.949862 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 834) took 2690 ms. Trace:
I20260811 18:57:33.949930 4169 rpcz_store.cc:276] 0811 18:57:31.258934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.260802 (+ 1868us) service_pool.cc:227] Handling call
0811 18:57:33.949760 (+2688958us) ts_tablet_manager.cc:1145] Deleting tablet c0f68d33e1b14144b34c17e3c7201408
0811 18:57:33.949771 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949846 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659260}
I20260811 18:57:33.950193 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 837) took 2655 ms. Trace:
I20260811 18:57:33.950255 4169 rpcz_store.cc:276] 0811 18:57:31.294282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.294528 (+ 246us) service_pool.cc:227] Handling call
0811 18:57:33.950111 (+2655583us) ts_tablet_manager.cc:1145] Deleting tablet 4e0747ef88574c68ae074732cb7095b9
0811 18:57:33.950119 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.950180 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2655361}
I20260811 18:57:33.950517 4169 tablet_replica.cc:330] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.950721 4169 raft_consensus.cc:2229] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.951023 4169 pending_rounds.cc:62] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:33.951076 4169 pending_rounds.cc:68] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400172410499072 op_type: NO_OP noop_request { }
W20260811 18:57:33.951136 3678 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.149s (SENT)
I20260811 18:57:33.951215 4169 raft_consensus.cc:2866] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.951304 4169 raft_consensus.cc:2258] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.951251 3678 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:33.951897 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6245131a3a4513b88611fd4c2e031c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.950253 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 809) took 4420 ms. Trace:
I20260811 18:57:33.952517 4167 rpcz_store.cc:276] 0811 18:57:29.529410 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.529463 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:33.910924 (+4381461us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:33.910938 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.923606 (+ 12668us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.944406 (+ 20800us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.950231 (+ 5825us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4380950}
I20260811 18:57:33.952865 4167 tablet_replica.cc:330] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.953058 4167 raft_consensus.cc:2229] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.954037 4167 raft_consensus.cc:2258] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.954735 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 221d0d8d69f142a29ec7065a82b3f769 (table default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]) successfully deleted
W20260811 18:57:33.955199 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7d747f03ad234805a4ea4ef60840d426 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d747f03ad234805a4ea4ef60840d426 already in progress: creating tablet
I20260811 18:57:33.955214 3824 leader_election.cc:302] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:33.955407 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6009c9f6bbb0493ebe125120a1a47f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6009c9f6bbb0493ebe125120a1a47f3f already in progress: creating tablet
W20260811 18:57:33.955602 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c944d5cfd9b3405b8151abfc9f2fd998 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c944d5cfd9b3405b8151abfc9f2fd998 already in progress: creating tablet
W20260811 18:57:33.955770 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 01a5fdab68694761b7493b42a0bf7c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01a5fdab68694761b7493b42a0bf7c5b already in progress: creating tablet
W20260811 18:57:33.955940 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c0f68d33e1b14144b34c17e3c7201408 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f68d33e1b14144b34c17e3c7201408 already in progress: creating tablet
I20260811 18:57:33.955920 3871 tablet_service.cc:1467] Processing CreateTablet for tablet de6245131a3a4513b88611fd4c2e031c (DEFAULT_TABLE table=default.Table_db613e17da86441bb1ab37b46e730db9 [id=d0c9376b40d34652839aec9e4e711def]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:33.956140 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4e0747ef88574c68ae074732cb7095b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e0747ef88574c68ae074732cb7095b9 already in progress: creating tablet
I20260811 18:57:33.956357 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6245131a3a4513b88611fd4c2e031c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.956355 3714 tablet_service.cc:1467] Processing CreateTablet for tablet c44eee516d634d9a986107b55410dfc3 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.956686 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44eee516d634d9a986107b55410dfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.956976 3709 tablet_service.cc:1467] Processing CreateTablet for tablet bc5105a88ad540adb326a15271237b5c (DEFAULT_TABLE table=default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 [id=30255d6b39ce43e6b7e98a2f2d9b210a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.957297 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5105a88ad540adb326a15271237b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.958580 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 683f257f18054935adcf17b00470d784 (DEFAULT_TABLE table=default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 [id=539b401b507c4ef9ad1fe5579b13dcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958626 3867 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4d5464594c0eb19f1739c29c0b45 (DEFAULT_TABLE table=default.Table_57453b6f4942454da731710b8510f4a7 [id=c1e2053ce9d340659a6dd593465dea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958626 3855 tablet_service.cc:1467] Processing CreateTablet for tablet bc5105a88ad540adb326a15271237b5c (DEFAULT_TABLE table=default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 [id=30255d6b39ce43e6b7e98a2f2d9b210a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958899 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683f257f18054935adcf17b00470d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.959043 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5105a88ad540adb326a15271237b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.962631 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c44eee516d634d9a986107b55410dfc3 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.962800 5417 raft_consensus.cc:2781] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.962896 5417 raft_consensus.cc:482] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.962961 5417 raft_consensus.cc:3037] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.962984 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44eee516d634d9a986107b55410dfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.964697 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6b28d478364f4057a8c2619e5d90b850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.964895 4167 ts_tablet_manager.cc:1905] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.965271 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b28d478364f4057a8c2619e5d90b850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.965478 5417 raft_consensus.cc:504] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.966118 5417 leader_election.cc:288] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.966598 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 683f257f18054935adcf17b00470d784 (DEFAULT_TABLE table=default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 [id=539b401b507c4ef9ad1fe5579b13dcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.966945 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683f257f18054935adcf17b00470d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.968086 5443 raft_consensus.cc:2726] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:33.976804 3853 tablet_service.cc:1467] Processing CreateTablet for tablet dae10844d9574b1da443358b37c7ab38 (DEFAULT_TABLE table=default.Table_73bf9f47862a4f03ac5e35afacfc2463 [id=685d668b3f3f42549c9e228ea9cdb981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.976982 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe (DEFAULT_TABLE table=default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c [id=7247bf0c19b6435795201ca8b085cb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.977265 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae10844d9574b1da443358b37c7ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977490 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe (DEFAULT_TABLE table=default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c [id=7247bf0c19b6435795201ca8b085cb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.977666 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3263da75f0d84528b7c8cc55dcaeaa46" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.977859 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977977 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977869 3743 raft_consensus.cc:3037] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.982657 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f86ba123808f41669c4e400da7c639ce (DEFAULT_TABLE table=default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 [id=dff3f101d29e4ce180b8afa24d7b248d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.983021 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86ba123808f41669c4e400da7c639ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.985396 3743 raft_consensus.cc:2454] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.987996 3824 leader_election.cc:302] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.991561 5417 raft_consensus.cc:2781] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.991819 5417 raft_consensus.cc:686] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.992065 5417 consensus_queue.cc:237] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.992903 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_489a043a9e89488ca2fbcd69855c4e5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.993476 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4d5464594c0eb19f1739c29c0b45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:33.993547 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_489a043a9e89488ca2fbcd69855c4e5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.994997 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 05cba6b392b642e59321570aac3541d0
I20260811 18:57:33.995739 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_33ef706171fc47669bf54e9a71d90905"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.996356 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33ef706171fc47669bf54e9a71d90905 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.999545 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cba6b392b642e59321570aac3541d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.999817 4169 ts_tablet_manager.cc:1905] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.000769 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e1922c7317c24aa984945aaaa6ce8003 (DEFAULT_TABLE table=default.Table_3309d84857df4970a9891fd64c181bf5 [id=24a68c86a16a494a8d4b2108c7b06c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.001192 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1922c7317c24aa984945aaaa6ce8003. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.001312 3678 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.001448 3678 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.001552 3678 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.002007 3678 leader_election.cc:334] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.002219 3678 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.002728 5443 raft_consensus.cc:2726] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.002303 3678 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.003621 3678 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.003721 3678 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.003867 3678 raft_consensus.cc:2670] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.004455 3678 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.004555 3678 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.004688 3678 raft_consensus.cc:2670] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.005153 3678 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.005247 3678 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.005357 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cba6b392b642e59321570aac3541d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.005442 5407 raft_consensus.cc:2726] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.005625 5407 raft_consensus.cc:2726] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.005548 3678 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.006203 3678 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.006786 3678 leader_election.cc:334] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.006868 5407 raft_consensus.cc:2726] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.005843 3720 tablet_service.cc:1467] Processing CreateTablet for tablet f86ba123808f41669c4e400da7c639ce (DEFAULT_TABLE table=default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 [id=dff3f101d29e4ce180b8afa24d7b248d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:34.007324 3678 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.007481 3678 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.007593 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86ba123808f41669c4e400da7c639ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.007671 3678 raft_consensus.cc:2670] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.008836 3678 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.008864 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 101994cc708a4fadb46fc8c5915a5587
I20260811 18:57:34.009023 3678 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.009397 3678 raft_consensus.cc:2670] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.010532 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 101994cc708a4fadb46fc8c5915a5587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.013255 3824 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.013716 3824 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.014002 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 101994cc708a4fadb46fc8c5915a5587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.014113 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_897c0f5693354801afbbaa4de2ad5252" } new_table_name: "default.Table_667271a3af6a4ab489e6ef9a2a2769a0" modify_external_catalogs: true
W20260811 18:57:34.014514 3824 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.016453 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:34.017345 3824 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.017562 3824 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.017711 3824 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.019729 3824 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020193 3824 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020608 3824 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020814 3824 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.022132 3541 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.022424 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 3 similar messages]
I20260811 18:57:34.030720 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7
I20260811 18:57:34.031137 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 deleted and replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8
I20260811 18:57:34.031486 3709 tablet_service.cc:1467] Processing CreateTablet for tablet dae10844d9574b1da443358b37c7ab38 (DEFAULT_TABLE table=default.Table_73bf9f47862a4f03ac5e35afacfc2463 [id=685d668b3f3f42549c9e228ea9cdb981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.032970 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28350d4a6baf4232bef35569f82e0d4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.033665 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28350d4a6baf4232bef35569f82e0d4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.039458 4163 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:34.039459 3678 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.039616 3678 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.039655 4163 ts_tablet_manager.cc:1397] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.119s user 0.003s sys 0.000s
W20260811 18:57:34.039851 3678 raft_consensus.cc:2670] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.040618 4163 raft_consensus.cc:348] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.040930 4163 raft_consensus.cc:374] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.041005 4163 raft_consensus.cc:729] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.041215 4163 consensus_queue.cc:260] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.047186 4163 ts_tablet_manager.cc:1428] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
W20260811 18:57:34.047189 3824 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.048488 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7216c095cc03424bbbe97a52a9600ebb
I20260811 18:57:34.049083 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 40898ceb18cc4f2d9104818798049ae2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.050402 4162 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.050733 4162 ts_tablet_manager.cc:1397] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.143s user 0.004s sys 0.001s
I20260811 18:57:34.051579 4162 raft_consensus.cc:348] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.051880 4162 raft_consensus.cc:374] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.051985 4162 raft_consensus.cc:729] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.052261 4162 consensus_queue.cc:260] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.052703 4163 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.052987 4162 ts_tablet_manager.cc:1428] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:34.054888 4163 tablet_bootstrap.cc:654] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.058983 3824 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.059950 3824 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.060909 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae10844d9574b1da443358b37c7ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.063287 4162 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.064960 4162 tablet_bootstrap.cc:654] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.066095 3824 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.067482 4167 ts_tablet_manager.cc:1918] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.067624 4167 log.cc:1196] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fbba1923ce0049448fe4e28f2dd258dc
W20260811 18:57:34.070537 3678 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.071558 3824 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.072353 3678 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.070602 4167 ts_tablet_manager.cc:1939] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.073510 3824 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.073588 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 810) took 4543 ms. Trace:
I20260811 18:57:34.073673 4167 rpcz_store.cc:276] 0811 18:57:29.529948 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.530006 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:33.952814 (+4422808us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:33.952828 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.967830 (+ 15002us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.045734 (+ 77904us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.073571 (+ 27837us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4422630}
W20260811 18:57:34.074270 3678 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.074673 3824 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.075001 3678 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.079843 5407 raft_consensus.cc:482] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:34.080016 3678 raft_consensus.cc:2670] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.079995 5407 raft_consensus.cc:504] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.080368 5417 raft_consensus.cc:482] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.080526 5407 leader_election.cc:288] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.080518 5417 raft_consensus.cc:504] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:34.080873 3824 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.081100 3824 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081151 5417 leader_election.cc:288] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:34.080535 3678 leader_election.cc:334] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081632 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968fa8b8912b40a8bdba77db1d39c5c0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.081753 3678 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081847 3678 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.081984 3678 raft_consensus.cc:2670] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.082628 3824 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.082904 3824 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.082224 3676 leader_election.cc:341] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.088044 3678 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.088289 3824 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.088451 3678 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.089039 3824 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.090539 4167 tablet_replica.cc:330] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.091290 4167 raft_consensus.cc:2229] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.091522 4167 raft_consensus.cc:2258] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.092108 4167 ts_tablet_manager.cc:1905] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.095772 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7be6d9578e7847c2822278486119390c
I20260811 18:57:34.096614 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet a25b1eddd4a64847b51ad3cdc0238ff0 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:34.098519 3678 raft_consensus.cc:2670] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.099215 4162 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.099994 4162 ts_tablet_manager.cc:1397] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.001s
I20260811 18:57:34.100843 4162 raft_consensus.cc:348] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.101145 4162 raft_consensus.cc:374] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.101390 4162 raft_consensus.cc:729] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.101742 4162 consensus_queue.cc:260] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.102499 4162 ts_tablet_manager.cc:1428] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:57:34.103363 3824 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.103809 4162 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:34.105906 3678 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.106139 3678 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.106386 3678 raft_consensus.cc:2670] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.107769 3824 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.108655 3824 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.109414 4162 tablet_bootstrap.cc:654] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.109885 3678 leader_election.cc:334] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.111543 3678 leader_election.cc:302] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.111616 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
W20260811 18:57:34.111780 3678 raft_consensus.cc:2670] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.112298 3678 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.113413 3824 leader_election.cc:334] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.114423 3678 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.117547 3678 raft_consensus.cc:2670] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.117699 3824 leader_election.cc:302] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.118384 3678 leader_election.cc:334] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.118837 3824 raft_consensus.cc:2670] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.122826 3824 leader_election.cc:334] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.123062 3678 leader_election.cc:302] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.123482 3678 raft_consensus.cc:2670] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.123307 4167 ts_tablet_manager.cc:1918] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.124264 4167 log.cc:1196] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:34.124714 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c031c1fe18014450aa4cffbc9cd0f573"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.125291 4167 ts_tablet_manager.cc:1939] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.125393 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c031c1fe18014450aa4cffbc9cd0f573 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.125667 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 807) took 4596 ms. Trace:
I20260811 18:57:34.125754 4167 rpcz_store.cc:276] 0811 18:57:29.528947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.530011 (+ 1064us) service_pool.cc:227] Handling call
0811 18:57:34.090477 (+4560466us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:34.090494 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.095328 (+ 4834us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.098290 (+ 2962us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.125643 (+ 27353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4560219}
I20260811 18:57:34.126811 4163 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.127003 4163 ts_tablet_manager.cc:1397] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.002s
I20260811 18:57:34.126999 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.127604 3678 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.127763 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.127840 4163 raft_consensus.cc:348] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.128094 4163 raft_consensus.cc:374] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.128180 4163 raft_consensus.cc:729] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.128422 4163 consensus_queue.cc:260] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.128711 3678 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.128549 3824 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.129037 3678 raft_consensus.cc:2670] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.129177 3824 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.129364 3824 raft_consensus.cc:2670] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.130038 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 277a245243d14de3be26ce3962e1a9b1 (DEFAULT_TABLE table=default.Table_1dc240fd92f941489de07a83cdafb797 [id=3921e9add8ac4b0caa95a18563adb2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.129547 4163 ts_tablet_manager.cc:1428] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260811 18:57:34.131271 3678 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.131743 3678 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.132373 3678 raft_consensus.cc:2670] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.133229 3678 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.135840 3678 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.136637 5407 raft_consensus.cc:2726] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.137327 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 816) took 4553 ms. Trace:
I20260811 18:57:34.137488 4167 rpcz_store.cc:276] 0811 18:57:29.583723 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.583806 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:34.129862 (+4546056us) ts_tablet_manager.cc:1145] Deleting tablet f3670f8908d54ebb82f698cdd207e48f
0811 18:57:34.137143 (+ 7281us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.137303 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4537805}
I20260811 18:57:34.138247 4163 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.139971 4163 tablet_bootstrap.cc:654] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.138049 4162 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.142944 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277a245243d14de3be26ce3962e1a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.144315 4162 ts_tablet_manager.cc:1397] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.000s
I20260811 18:57:34.145123 4162 raft_consensus.cc:348] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.145339 4162 raft_consensus.cc:374] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.145395 4162 raft_consensus.cc:729] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.145602 4162 consensus_queue.cc:260] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.146224 4167 tablet_replica.cc:330] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.146219 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dd80633f926942e88f54b32d9a03ea3a
I20260811 18:57:34.146294 4162 ts_tablet_manager.cc:1428] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:34.146441 4167 raft_consensus.cc:2229] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.146566 4162 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.146580 4167 raft_consensus.cc:2258] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.149001 4167 ts_tablet_manager.cc:1905] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.149277 5417 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.149427 5417 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.150054 5417 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.152066 4162 tablet_bootstrap.cc:654] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.153363 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
W20260811 18:57:34.158098 3678 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.159353 3678 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.160220 5407 raft_consensus.cc:2726] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.161211 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.161279 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 084b60eeb6b64e1da7336203d75770f4 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.161610 3719 tablet_service.cc:1467] Processing CreateTablet for tablet eb43127dda814a718b338906fc2a3c47 (DEFAULT_TABLE table=default.Table_91bea743ec0b491299f45f3ac23cb734 [id=630cc439da6f48acb379cff1ba6d003a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.161744 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b60eeb6b64e1da7336203d75770f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.161996 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb43127dda814a718b338906fc2a3c47. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.162602 3678 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.162719 3678 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.162848 3678 raft_consensus.cc:2670] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.163519 3678 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.163615 3678 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.166424 5407 raft_consensus.cc:2726] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.166900 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 20da7ea64e3d49e08b3886693625acad (DEFAULT_TABLE table=default.Table_2e691dcf4a764665aa9275bfa441105a [id=fda127e467d449368d65942e5e5b98f7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:34.167124 3678 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.167229 3678 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.167322 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20da7ea64e3d49e08b3886693625acad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.167371 3678 raft_consensus.cc:2670] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.168826 4163 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.168977 4163 ts_tablet_manager.cc:1397] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.003s
I20260811 18:57:34.170255 4163 raft_consensus.cc:348] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.170531 4163 raft_consensus.cc:374] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.170639 4163 raft_consensus.cc:729] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.170680 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 9390e8e7967643ab8decce41dde094b2 (DEFAULT_TABLE table=default.Table_9a13d9e49bc748e6b820a614e55ba4c0 [id=d1b8f780910b4d5bb2eeb196d7ff9b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.171047 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9390e8e7967643ab8decce41dde094b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.170924 4163 consensus_queue.cc:260] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.170909 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7e288f51b09f4c79bb78a9c52a33e16f (DEFAULT_TABLE table=default.Table_79f5b2394f094ecc85733e272dcfe369 [id=f35c0335cfe241b7971283ad3b09f2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.171571 4163 ts_tablet_manager.cc:1428] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:34.172441 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e288f51b09f4c79bb78a9c52a33e16f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.174674 3678 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.174865 3678 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.175063 3678 raft_consensus.cc:2670] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.175676 3678 leader_election.cc:334] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.175846 3678 leader_election.cc:302] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.176033 3678 raft_consensus.cc:2670] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.178174 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5 (DEFAULT_TABLE table=default.Table_8f1d0ca0ad934be687840ccd25db713a [id=c7d39b2c17b54f489fc99c48e6554637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178150 3715 tablet_service.cc:1467] Processing CreateTablet for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2 (DEFAULT_TABLE table=default.Table_4e35d031a62148d582b7bdb0240f63da [id=b48dc34f86424368ab050aa368f8759b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178295 3705 tablet_service.cc:1467] Processing CreateTablet for tablet e861f6e4e3f645de84a67ff87124f057 (DEFAULT_TABLE table=default.Table_4dc767eb645648aebd0aacf2d46f9c3b [id=56302f9edb93485cb3dc030323072103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178591 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.178563 5417 raft_consensus.cc:482] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.178764 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.178704 5417 raft_consensus.cc:504] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.179401 5417 leader_election.cc:288] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.179818 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e861f6e4e3f645de84a67ff87124f057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.193686 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 3b540ae350254346a675b54118c5cfcb (DEFAULT_TABLE table=default.Table_1bfaa8ab4fcd40efb60e1718e424bccf [id=bbe3afd3838c4c2ebf633f308d3de7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.194257 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b540ae350254346a675b54118c5cfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.196224 4167 ts_tablet_manager.cc:1918] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.196375 4167 log.cc:1196] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:34.196707 4167 ts_tablet_manager.cc:1939] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.196823 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:34.197490 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 817) took 4613 ms. Trace:
I20260811 18:57:34.197602 4167 rpcz_store.cc:276] 0811 18:57:29.584153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.584218 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:34.146173 (+4561955us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:34.146185 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.157083 (+ 10898us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.196121 (+ 39038us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.197470 (+ 1349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4527748}
I20260811 18:57:34.197948 4167 tablet_replica.cc:330] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.198192 4167 raft_consensus.cc:2229] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.201167 4163 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.203199 4163 tablet_bootstrap.cc:654] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.210745 4167 raft_consensus.cc:2258] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.210644 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7009400051004543ade835c61177ac98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.211337 4167 ts_tablet_manager.cc:1905] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:34.211944 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7009400051004543ade835c61177ac98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.203364 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 7216c095cc03424bbbe97a52a9600ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.209270 3678 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.214601 3678 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.214993 3678 raft_consensus.cc:2670] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.215710 3678 leader_election.cc:334] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.216162 3678 leader_election.cc:302] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.216504 3678 raft_consensus.cc:2670] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.217228 3678 leader_election.cc:334] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.218101 3678 leader_election.cc:302] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.218879 5407 raft_consensus.cc:2726] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.218914 3678 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.219432 3678 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.219877 5407 raft_consensus.cc:2726] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.219924 3678 leader_election.cc:334] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.220389 3678 leader_election.cc:302] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.220896 5407 raft_consensus.cc:2726] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.220942 3678 leader_election.cc:334] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.221421 3678 leader_election.cc:302] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.221762 5407 raft_consensus.cc:2726] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.221819 3678 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.222975 3678 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.222656 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.222795 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.222525 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.223448 5407 raft_consensus.cc:2726] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.223526 3678 leader_election.cc:334] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.224437 3678 leader_election.cc:302] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.224556 4169 ts_tablet_manager.cc:1918] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.224709 4169 log.cc:1196] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f335e50ae57547ed80b144129faaffbb
I20260811 18:57:34.225025 5407 raft_consensus.cc:2726] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.225131 3678 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.225250 3678 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.225561 4169 ts_tablet_manager.cc:1939] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.226051 5407 raft_consensus.cc:2726] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.226100 3678 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.226199 3678 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.231174 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.231483 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.230708 3678 raft_consensus.cc:2670] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.232854 3678 consensus_peers.cc:487] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.233359 3678 consensus_peers.cc:487] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.233729 3678 consensus_peers.cc:487] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234043 3678 consensus_peers.cc:487] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234485 3678 consensus_peers.cc:487] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234782 3678 consensus_peers.cc:487] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.235232 3678 consensus_peers.cc:487] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.235831 3678 consensus_peers.cc:487] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.235998 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 43ad6993b63e4bbd98c8cc52f1a53451 (table default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]) successfully deleted
W20260811 18:57:34.236367 3678 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.236723 3678 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.237165 3678 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.237619 3678 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.237803 5407 raft_consensus.cc:2726] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.238058 5443 raft_consensus.cc:2726] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.238421 3678 leader_election.cc:334] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.238783 3678 leader_election.cc:302] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.239161 5407 raft_consensus.cc:2726] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.239416 3678 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.239722 3678 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.240097 5407 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.240372 3678 leader_election.cc:334] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.240676 3678 leader_election.cc:302] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.242643 5407 raft_consensus.cc:2726] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.243021 3678 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.243386 3678 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.243835 5407 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.244251 3678 consensus_peers.cc:487] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.227365 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be6d9578e7847c2822278486119390c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.245126 3678 consensus_peers.cc:487] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.252080 3678 leader_election.cc:334] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.252650 3678 leader_election.cc:302] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.253189 5407 raft_consensus.cc:2726] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.254556 5407 raft_consensus.cc:482] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.254747 5407 raft_consensus.cc:504] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.255360 5407 leader_election.cc:288] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.258857 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0db19ab70f764e00b17ee2e0f4742b8b" } new_table_name: "default.Table_d85b163d70284446be67f85eddf05383" modify_external_catalogs: true
I20260811 18:57:34.259342 4162 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.259485 4162 ts_tablet_manager.cc:1397] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.113s user 0.003s sys 0.010s
I20260811 18:57:34.260342 4162 raft_consensus.cc:348] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.260550 4162 raft_consensus.cc:374] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.260601 4162 raft_consensus.cc:729] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.260787 4162 consensus_queue.cc:260] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.261137 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a79b27c0bd524af985c2e6167624551a" } new_table_name: "default.Table_5fe41c188c7b4726bbc18f5bcf86b3c2" modify_external_catalogs: true
I20260811 18:57:34.261482 4162 ts_tablet_manager.cc:1428] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:57:34.237653 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f3670f8908d54ebb82f698cdd207e48f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3670f8908d54ebb82f698cdd207e48f already in progress: creating tablet
I20260811 18:57:34.262353 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.262753 4162 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.262703 5417 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.262851 5417 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.263465 5417 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.264813 3853 tablet_service.cc:1467] Processing CreateTablet for tablet eb43127dda814a718b338906fc2a3c47 (DEFAULT_TABLE table=default.Table_91bea743ec0b491299f45f3ac23cb734 [id=630cc439da6f48acb379cff1ba6d003a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.265031 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 122b7898fea44ef78f269b1505d8ddc4 (table default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b [id=1bcc678c540844b38d761cec1618c852]) successfully deleted
W20260811 18:57:34.262521 3678 leader_election.cc:334] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.265748 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 20da7ea64e3d49e08b3886693625acad (DEFAULT_TABLE table=default.Table_2e691dcf4a764665aa9275bfa441105a [id=fda127e467d449368d65942e5e5b98f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.265859 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 277a245243d14de3be26ce3962e1a9b1 (DEFAULT_TABLE table=default.Table_1dc240fd92f941489de07a83cdafb797 [id=3921e9add8ac4b0caa95a18563adb2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.266330 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 7e288f51b09f4c79bb78a9c52a33e16f (DEFAULT_TABLE table=default.Table_79f5b2394f094ecc85733e272dcfe369 [id=f35c0335cfe241b7971283ad3b09f2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.266754 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 084b60eeb6b64e1da7336203d75770f4 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.267899 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aee6beb2e70421083841520733087f8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.268357 4162 tablet_bootstrap.cc:654] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.268774 3676 leader_election.cc:341] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.268940 3676 leader_election.cc:302] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.269420 5407 raft_consensus.cc:2726] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.235378 4167 ts_tablet_manager.cc:1918] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.269675 4167 log.cc:1196] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:34.269644 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 839) took 2906 ms. Trace:
I20260811 18:57:34.270046 4167 ts_tablet_manager.cc:1939] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.269831 4169 rpcz_store.cc:276] 0811 18:57:31.363218 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.363296 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:33.950472 (+2587176us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:33.950482 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.059351 (+108869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.224422 (+165071us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.269525 (+ 45103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2586915}
I20260811 18:57:34.270434 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.270634 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.270598 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 838) took 2907 ms. Trace:
I20260811 18:57:34.270752 4169 rpcz_store.cc:276] 0811 18:57:31.362814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.364000 (+ 1186us) service_pool.cc:227] Handling call
0811 18:57:34.270404 (+2906404us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:34.270418 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.270552 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2906100}
I20260811 18:57:34.271445 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 841) took 2884 ms. Trace:
I20260811 18:57:34.271641 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 9390e8e7967643ab8decce41dde094b2 (DEFAULT_TABLE table=default.Table_9a13d9e49bc748e6b820a614e55ba4c0 [id=d1b8f780910b4d5bb2eeb196d7ff9b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.271857 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 826) took 4338 ms. Trace:
I20260811 18:57:34.271634 4169 rpcz_store.cc:276] 0811 18:57:31.387004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.387071 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:34.271212 (+2884141us) ts_tablet_manager.cc:1145] Deleting tablet 99e9474ef8814514832f0e48c2b3495f
0811 18:57:34.271226 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.271424 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2883791}
I20260811 18:57:34.272595 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb43127dda814a718b338906fc2a3c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.271925 4167 rpcz_store.cc:276] 0811 18:57:29.933454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.070108 (+136654us) service_pool.cc:227] Handling call
0811 18:57:34.197906 (+4127798us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:34.197917 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.225852 (+ 27935us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.235266 (+ 9414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.271836 (+ 36570us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4125317}
I20260811 18:57:34.273139 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dae19af473234626ac121fc618e92a4b" } new_table_name: "default.Table_c48fbb6941794b85becdf643c4d21775" modify_external_catalogs: true
I20260811 18:57:34.274636 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7216c095cc03424bbbe97a52a9600ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.274910 5443 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.275024 5443 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.275480 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.275574 5443 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.275839 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.276177 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.276722 3676 leader_election.cc:341] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.277145 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20da7ea64e3d49e08b3886693625acad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.277681 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 828) took 4120 ms. Trace:
I20260811 18:57:34.277769 4167 rpcz_store.cc:276] 0811 18:57:30.157532 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.161642 (+ 4110us) service_pool.cc:227] Handling call
0811 18:57:34.277571 (+4115929us) ts_tablet_manager.cc:1145] Deleting tablet 98ad733726594d2a8087f857f155dbe2
0811 18:57:34.277583 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.277665 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4115662}
I20260811 18:57:34.277823 3864 tablet_service.cc:1467] Processing CreateTablet for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5 (DEFAULT_TABLE table=default.Table_8f1d0ca0ad934be687840ccd25db713a [id=c7d39b2c17b54f489fc99c48e6554637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.278173 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278249 4163 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.278411 4163 ts_tablet_manager.cc:1397] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.001s sys 0.003s
I20260811 18:57:34.278765 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277a245243d14de3be26ce3962e1a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278872 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 829) took 4119 ms. Trace:
I20260811 18:57:34.279035 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b60eeb6b64e1da7336203d75770f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278954 4167 rpcz_store.cc:276] 0811 18:57:30.159306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.163887 (+ 4581us) service_pool.cc:227] Handling call
0811 18:57:34.278214 (+4114327us) ts_tablet_manager.cc:1145] Deleting tablet 85140a3800f34a23b716703fd4e73d17
0811 18:57:34.278753 (+ 539us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.278853 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4113992}
W20260811 18:57:34.279958 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet f335e50ae57547ed80b144129faaffbb: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.281030 4167 tablet_replica.cc:330] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.281558 4167 raft_consensus.cc:2229] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.283038 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e288f51b09f4c79bb78a9c52a33e16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.285298 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2 (DEFAULT_TABLE table=default.Table_4e35d031a62148d582b7bdb0240f63da [id=b48dc34f86424368ab050aa368f8759b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.285769 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.287113 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9390e8e7967643ab8decce41dde094b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.287235 4167 raft_consensus.cc:2258] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.289011 4167 ts_tablet_manager.cc:1905] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.290911 3855 tablet_service.cc:1467] Processing CreateTablet for tablet e861f6e4e3f645de84a67ff87124f057 (DEFAULT_TABLE table=default.Table_4dc767eb645648aebd0aacf2d46f9c3b [id=56302f9edb93485cb3dc030323072103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.291586 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:34.291801 3743 raft_consensus.cc:2454] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:34.292894 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 3b540ae350254346a675b54118c5cfcb (DEFAULT_TABLE table=default.Table_1bfaa8ab4fcd40efb60e1718e424bccf [id=bbe3afd3838c4c2ebf633f308d3de7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.295517 4162 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.295746 4162 ts_tablet_manager.cc:1397] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.002s
I20260811 18:57:34.296669 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.296612 4162 raft_consensus.cc:348] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.296833 4162 raft_consensus.cc:374] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.296897 4162 raft_consensus.cc:729] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.297113 4162 consensus_queue.cc:260] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.297261 4163 raft_consensus.cc:348] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.297538 4163 raft_consensus.cc:374] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.297724 4162 ts_tablet_manager.cc:1428] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:34.297637 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 840) took 2910 ms. Trace:
I20260811 18:57:34.297741 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be6d9578e7847c2822278486119390c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.297984 4162 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.297837 4169 rpcz_store.cc:276] 0811 18:57:31.386765 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.387457 (+ 692us) service_pool.cc:227] Handling call
0811 18:57:34.272095 (+2884638us) ts_tablet_manager.cc:1145] Deleting tablet 761902f7366d43ffbde226a0ade2e188
0811 18:57:34.297362 (+ 25267us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x618000005118. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d090f80fd 00007f4d090f916e 00007f4d1c04dcfd 00007f4d1bfcb883 00007f4d1826b6e6 00007f4d1826ca37 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c
0811 18:57:34.297375 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.297589 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":56460032,"tablet-delete.queue_time_us":2884316}
I20260811 18:57:34.299283 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 849) took 2560 ms. Trace:
I20260811 18:57:34.299423 4169 rpcz_store.cc:276] 0811 18:57:31.738802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.738859 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:34.298486 (+2559627us) ts_tablet_manager.cc:1145] Deleting tablet 35ebc32eb5d5443eaaeba245b566f98a
0811 18:57:34.298528 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.299237 (+ 709us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2559397}
I20260811 18:57:34.299825 4162 tablet_bootstrap.cc:654] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.300091 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 847) took 2564 ms. Trace:
I20260811 18:57:34.300318 4169 rpcz_store.cc:276] 0811 18:57:31.735570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.739154 (+ 3584us) service_pool.cc:227] Handling call
0811 18:57:34.299892 (+2560738us) ts_tablet_manager.cc:1145] Deleting tablet 1eb30e64e86040098dcee5fc2e9b7266
0811 18:57:34.299905 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.300048 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2560413}
I20260811 18:57:34.301016 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.297623 4163 raft_consensus.cc:729] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.301558 4163 consensus_queue.cc:260] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.303179 4163 ts_tablet_manager.cc:1428] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260811 18:57:34.304968 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e861f6e4e3f645de84a67ff87124f057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.307531 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.308581 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet dd80633f926942e88f54b32d9a03ea3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.300989 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 850) took 2544 ms. Trace:
I20260811 18:57:34.310722 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b540ae350254346a675b54118c5cfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.309706 4169 rpcz_store.cc:276] 0811 18:57:31.756579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.785593 (+ 29014us) service_pool.cc:227] Handling call
0811 18:57:34.300793 (+2515200us) ts_tablet_manager.cc:1145] Deleting tablet 3dd4ec30acde4585a1c3451927217cb3
0811 18:57:34.300805 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.300942 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2514936}
I20260811 18:57:34.317293 5443 raft_consensus.cc:482] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.317451 5443 raft_consensus.cc:504] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.318033 5443 leader_election.cc:288] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.321688 4167 ts_tablet_manager.cc:1918] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.323261 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45aff3e8cb0c4f7191e4522c6ef7d156" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.326774 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 859) took 2511 ms. Trace:
I20260811 18:57:34.326905 4169 rpcz_store.cc:276] 0811 18:57:31.815117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.815186 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:34.311743 (+2496557us) ts_tablet_manager.cc:1145] Deleting tablet f2f31331974f42c3a22b3ac21b690f82
0811 18:57:34.326601 (+ 14858us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.326753 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2496216}
I20260811 18:57:34.327245 4169 tablet_replica.cc:330] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.327521 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0bfb54175c834517b7af291c942333cd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.327525 4169 raft_consensus.cc:2229] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.327709 4169 raft_consensus.cc:2258] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.328406 4169 ts_tablet_manager.cc:1905] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.328455 4167 log.cc:1196] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:34.328827 4167 ts_tablet_manager.cc:1939] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.327462 3676 leader_election.cc:341] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.328320 4162 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.330801 4162 ts_tablet_manager.cc:1397] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20260811 18:57:34.332428 4163 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.332319 4162 raft_consensus.cc:348] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.332618 4162 raft_consensus.cc:374] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.332705 4162 raft_consensus.cc:729] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.332945 4162 consensus_queue.cc:260] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.333839 4162 ts_tablet_manager.cc:1428] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:34.334574 4163 tablet_bootstrap.cc:654] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.336319 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 40898ceb18cc4f2d9104818798049ae2
I20260811 18:57:34.336835 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 40898ceb18cc4f2d9104818798049ae2 deleted and replaced by tablet a62cdd333ad8494897ae8ec5afd5af74
I20260811 18:57:34.338276 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 832) took 3646 ms. Trace:
I20260811 18:57:34.338490 5443 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.338610 5443 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.339187 5443 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.338410 4167 rpcz_store.cc:276] 0811 18:57:30.691456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.691515 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:34.280978 (+3589463us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:34.280992 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.292069 (+ 11077us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.321584 (+ 29515us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.338252 (+ 16668us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3586057}
I20260811 18:57:34.343889 4162 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.346022 4162 tablet_bootstrap.cc:654] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.336017 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336203 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336364 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet dd80633f926942e88f54b32d9a03ea3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336436 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336620 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336773 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.346884 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 856) took 3276 ms. Trace:
W20260811 18:57:34.349355 3678 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.354120 3678 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.354023 4167 rpcz_store.cc:276] 0811 18:57:31.070670 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.070817 (+ 147us) service_pool.cc:227] Handling call
0811 18:57:34.346706 (+3275889us) ts_tablet_manager.cc:1145] Deleting tablet 22abc6f8e9ef4c32ab69af31bdaad872
0811 18:57:34.346717 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.346866 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3263483}
I20260811 18:57:34.355110 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 858) took 3276 ms. Trace:
I20260811 18:57:34.355333 5443 raft_consensus.cc:2726] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.356094 4167 rpcz_store.cc:276] 0811 18:57:31.078794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.083842 (+ 5048us) service_pool.cc:227] Handling call
0811 18:57:34.354974 (+3271132us) ts_tablet_manager.cc:1145] Deleting tablet 3030c31bc2b24ee296f29b5d5aa402ff
0811 18:57:34.354986 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.355092 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3270763}
I20260811 18:57:34.357604 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 859) took 3278 ms. Trace:
I20260811 18:57:34.357771 4167 rpcz_store.cc:276] 0811 18:57:31.079020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.084228 (+ 5208us) service_pool.cc:227] Handling call
0811 18:57:34.357476 (+3273248us) ts_tablet_manager.cc:1145] Deleting tablet 6009c9f6bbb0493ebe125120a1a47f3f
0811 18:57:34.357488 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.357585 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3272912}
W20260811 18:57:34.337523 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.343202 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a7051bc539468aafba30fbaf0669d4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.358493 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.343807 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:34.359127 3676 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.359606 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.359596 3678 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.359721 3678 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.360078 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 861) took 3280 ms. Trace:
I20260811 18:57:34.360574 4167 rpcz_store.cc:276] 0811 18:57:31.079550 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.095154 (+ 15604us) service_pool.cc:227] Handling call
0811 18:57:34.359963 (+3264809us) ts_tablet_manager.cc:1145] Deleting tablet 7d747f03ad234805a4ea4ef60840d426
0811 18:57:34.359974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.360060 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3264598}
W20260811 18:57:34.360998 3824 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.360939 5443 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.361292 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.361434 3824 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.361672 3824 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.361232 3678 leader_election.cc:334] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.362283 3824 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.362294 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a25b1eddd4a64847b51ad3cdc0238ff0
I20260811 18:57:34.362624 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 860) took 3283 ms. Trace:
I20260811 18:57:34.362799 3550 catalog_manager.cc:6565] ReplaceTablet: tablet a25b1eddd4a64847b51ad3cdc0238ff0 deleted and replaced by tablet 8ed141f91ec548db87ac1e719256931b
I20260811 18:57:34.362778 4167 rpcz_store.cc:276] 0811 18:57:31.079280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.089414 (+ 10134us) service_pool.cc:227] Handling call
0811 18:57:34.362509 (+3273095us) ts_tablet_manager.cc:1145] Deleting tablet 90fde17971ea49aaad212a5836661b08
0811 18:57:34.362522 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.362606 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3224511}
W20260811 18:57:34.362881 3824 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.363262 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 865) took 3283 ms. Trace:
I20260811 18:57:34.363343 4167 rpcz_store.cc:276] 0811 18:57:31.080281 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.167681 (+ 87400us) service_pool.cc:227] Handling call
0811 18:57:34.363148 (+3195467us) ts_tablet_manager.cc:1145] Deleting tablet c0f68d33e1b14144b34c17e3c7201408
0811 18:57:34.363160 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.363245 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3195196}
W20260811 18:57:34.363415 3824 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.363643 3824 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.364203 3824 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.364418 3824 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.369009 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 866) took 3127 ms. Trace:
I20260811 18:57:34.368952 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0d7c25599688441595b4b8f6cfea4584"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.369201 4167 rpcz_store.cc:276] 0811 18:57:31.241338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.241428 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:34.368850 (+3127422us) ts_tablet_manager.cc:1145] Deleting tablet 01a5fdab68694761b7493b42a0bf7c5b
0811 18:57:34.368865 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.368985 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3108715}
W20260811 18:57:34.369668 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d7c25599688441595b4b8f6cfea4584 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.369843 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 867) took 3109 ms. Trace:
I20260811 18:57:34.369935 4167 rpcz_store.cc:276] 0811 18:57:31.260726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.261417 (+ 691us) service_pool.cc:227] Handling call
0811 18:57:34.369641 (+3108224us) ts_tablet_manager.cc:1145] Deleting tablet c944d5cfd9b3405b8151abfc9f2fd998
0811 18:57:34.369667 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.369757 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3097869}
I20260811 18:57:34.370560 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 868) took 3099 ms. Trace:
I20260811 18:57:34.370673 4167 rpcz_store.cc:276] 0811 18:57:31.271161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.271802 (+ 641us) service_pool.cc:227] Handling call
0811 18:57:34.370444 (+3098642us) ts_tablet_manager.cc:1145] Deleting tablet 4e0747ef88574c68ae074732cb7095b9
0811 18:57:34.370456 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.370543 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3050892}
I20260811 18:57:34.375027 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 871) took 2998 ms. Trace:
I20260811 18:57:34.375590 4167 rpcz_store.cc:276] 0811 18:57:31.376414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.376470 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:34.374867 (+2998397us) ts_tablet_manager.cc:1145] Deleting tablet 99e9474ef8814514832f0e48c2b3495f
0811 18:57:34.374881 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.375004 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2998173}
I20260811 18:57:34.376781 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 870) took 3000 ms. Trace:
I20260811 18:57:34.377156 4167 rpcz_store.cc:276] 0811 18:57:31.376288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.382547 (+ 6259us) service_pool.cc:227] Handling call
0811 18:57:34.376482 (+2993935us) ts_tablet_manager.cc:1145] Deleting tablet 761902f7366d43ffbde226a0ade2e188
0811 18:57:34.376495 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.376760 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2993604}
I20260811 18:57:34.378016 4167 tablet_replica.cc:330] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.378762 4167 raft_consensus.cc:2229] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.379140 4167 raft_consensus.cc:2258] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.380491 4167 ts_tablet_manager.cc:1905] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.381772 5443 raft_consensus.cc:482] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.381906 5443 raft_consensus.cc:504] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.382256 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495f858f5eec496680f982bccd44415d" } new_table_name: "default.Table_8c4206dfbf6d4214980816f41a2c6db8" modify_external_catalogs: true
I20260811 18:57:34.382582 5443 leader_election.cc:288] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.383499 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7057b8a7bae249caaed304b5c66e6871" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.384275 3676 leader_election.cc:341] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.391660 4162 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.391912 4162 ts_tablet_manager.cc:1397] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.001s
I20260811 18:57:34.392750 4162 raft_consensus.cc:348] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.393060 4162 raft_consensus.cc:374] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.393172 4162 raft_consensus.cc:729] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.393479 4162 consensus_queue.cc:260] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.394448 4162 ts_tablet_manager.cc:1428] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:34.396966 4162 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.401391 4169 ts_tablet_manager.cc:1918] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.402714 4169 log.cc:1196] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c08436f33db043b1bc6395cd92102e59
I20260811 18:57:34.402802 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_53f67a53b6684eaeb56341d3d428f565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.403487 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53f67a53b6684eaeb56341d3d428f565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.405352 4169 ts_tablet_manager.cc:1939] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:34.407186 3824 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.407621 3824 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.408221 5443 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.408413 5443 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.409273 5443 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.409097 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 861) took 2587 ms. Trace:
W20260811 18:57:34.409698 3824 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.409830 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.409981 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.410117 3824 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.410410 3676 leader_election.cc:341] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.410530 3824 leader_election.cc:334] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.410662 3824 leader_election.cc:302] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.411140 3824 raft_consensus.cc:2670] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.411852 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.410410 4162 tablet_bootstrap.cc:654] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.409683 4169 rpcz_store.cc:276] 0811 18:57:31.821554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.822979 (+ 1425us) service_pool.cc:227] Handling call
0811 18:57:34.327202 (+2504223us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:34.327213 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.350872 (+ 23659us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.400773 (+ 49901us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.408915 (+ 8142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2504031}
I20260811 18:57:34.415861 4169 tablet_replica.cc:330] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.417552 4169 raft_consensus.cc:2229] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:34.417840 3824 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.423262 3824 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.423723 4169 raft_consensus.cc:2258] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:34.425438 3824 consensus_peers.cc:487] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.426586 3824 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.428589 3824 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.430476 3824 consensus_peers.cc:487] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.431437 3824 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.433507 3824 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.435842 3824 consensus_peers.cc:487] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.436697 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_58f2aeab56c6446184cfeddcb1134314"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.436929 3824 leader_election.cc:334] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.439080 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.440045 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.440685 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 40898ceb18cc4f2d9104818798049ae2 with delete_type TABLET_DATA_DELETED (replaced by tablet a62cdd333ad8494897ae8ec5afd5af74) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.440824 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ed141f91ec548db87ac1e719256931b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.444033 4162 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.444262 4162 ts_tablet_manager.cc:1397] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.001s
W20260811 18:57:34.448830 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10e441b5df3d40fd93cf2dafe33bc2f4 consensus state has no leader
I20260811 18:57:34.449596 4162 raft_consensus.cc:348] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.449839 4162 raft_consensus.cc:374] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.449888 4162 raft_consensus.cc:729] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:34.446403 3824 leader_election.cc:334] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.450052 4162 consensus_queue.cc:260] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.450596 4162 ts_tablet_manager.cc:1428] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
W20260811 18:57:34.450872 3824 consensus_peers.cc:487] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.452574 4167 ts_tablet_manager.cc:1918] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:34.452637 3824 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.453904 3824 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.454900 3824 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.455395 4167 log.cc:1196] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:34.455878 4167 ts_tablet_manager.cc:1939] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.456147 3824 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.456807 4162 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.457216 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" } modify_external_catalogs: true
I20260811 18:57:34.457479 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" } modify_external_catalogs: true
W20260811 18:57:34.457296 3824 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.458379 3824 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.458902 4162 tablet_bootstrap.cc:654] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.459300 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 869) took 3088 ms. Trace:
W20260811 18:57:34.459661 3824 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.461787 4167 rpcz_store.cc:276] 0811 18:57:31.370640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.392165 (+ 21525us) service_pool.cc:227] Handling call
0811 18:57:34.377963 (+2985798us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:34.377974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.390179 (+ 12205us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.452471 (+ 62292us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.459271 (+ 6800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2970142}
W20260811 18:57:34.461575 3824 consensus_peers.cc:487] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.462424 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 879) took 2712 ms. Trace:
W20260811 18:57:34.463096 3824 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.466526 4167 rpcz_store.cc:276] 0811 18:57:31.749510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.749550 (+ 40us) service_pool.cc:227] Handling call
0811 18:57:34.462246 (+2712696us) ts_tablet_manager.cc:1145] Deleting tablet 3dd4ec30acde4585a1c3451927217cb3
0811 18:57:34.462258 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.462401 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2712491}
W20260811 18:57:34.466800 3824 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.468807 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a3466b2364d64b0cad93b68732e0dae2 consensus state has no leader
I20260811 18:57:34.469206 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 876) took 2720 ms. Trace:
I20260811 18:57:34.469733 4167 rpcz_store.cc:276] 0811 18:57:31.749039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.757994 (+ 8955us) service_pool.cc:227] Handling call
0811 18:57:34.469006 (+2711012us) ts_tablet_manager.cc:1145] Deleting tablet 1eb30e64e86040098dcee5fc2e9b7266
0811 18:57:34.469026 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.469181 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2710685}
W20260811 18:57:34.469547 3824 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.470955 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 878) took 2721 ms. Trace:
I20260811 18:57:34.471062 4167 rpcz_store.cc:276] 0811 18:57:31.749382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.749477 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:34.470821 (+2721344us) ts_tablet_manager.cc:1145] Deleting tablet 35ebc32eb5d5443eaaeba245b566f98a
0811 18:57:34.470833 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.470936 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2711157}
I20260811 18:57:34.471382 4167 tablet_replica.cc:330] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.471623 4167 raft_consensus.cc:2229] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.471825 4167 raft_consensus.cc:2258] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.472313 4169 ts_tablet_manager.cc:1905] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.472421 4167 ts_tablet_manager.cc:1905] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.474647 4162 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.474845 4162 ts_tablet_manager.cc:1397] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260811 18:57:34.475790 4162 raft_consensus.cc:348] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.476261 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58f2aeab56c6446184cfeddcb1134314 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.476859 4162 raft_consensus.cc:374] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.477017 4162 raft_consensus.cc:729] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.477442 4162 consensus_queue.cc:260] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.482391 3824 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.485913 3678 leader_election.cc:302] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.485417 3824 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.487759 3824 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.489174 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" } modify_external_catalogs: true
I20260811 18:57:34.485733 4162 ts_tablet_manager.cc:1428] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260811 18:57:34.489475 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" } modify_external_catalogs: true
W20260811 18:57:34.489954 3824 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.492249 4162 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:34.491828 3824 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.493041 5443 raft_consensus.cc:2726] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.493080 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 131844us
W20260811 18:57:34.494077 3824 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.495708 3824 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.498972 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ebe3fed5aaf45918dbadfde06b2c24e consensus state has no leader
I20260811 18:57:34.499264 4162 tablet_bootstrap.cc:654] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.498728 3824 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.502767 4167 ts_tablet_manager.cc:1918] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.505183 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_061ae2e2037348cab0a2499a98d273a6" } modify_external_catalogs: true
I20260811 18:57:34.505563 4167 log.cc:1196] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:34.505496 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_061ae2e2037348cab0a2499a98d273a6" } modify_external_catalogs: true
I20260811 18:57:34.506029 4167 ts_tablet_manager.cc:1939] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.507001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 889) took 2687 ms. Trace:
I20260811 18:57:34.510268 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c26851245cab4a2d80e94d4fe278fd50 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
W20260811 18:57:34.510720 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 98ad733726594d2a8087f857f155dbe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98ad733726594d2a8087f857f155dbe2 already in progress: creating tablet
W20260811 18:57:34.511142 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 85140a3800f34a23b716703fd4e73d17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85140a3800f34a23b716703fd4e73d17 already in progress: creating tablet
W20260811 18:57:34.511560 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 22abc6f8e9ef4c32ab69af31bdaad872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22abc6f8e9ef4c32ab69af31bdaad872 already in progress: creating tablet
W20260811 18:57:34.511830 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3030c31bc2b24ee296f29b5d5aa402ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3030c31bc2b24ee296f29b5d5aa402ff already in progress: creating tablet
W20260811 18:57:34.512115 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6009c9f6bbb0493ebe125120a1a47f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6009c9f6bbb0493ebe125120a1a47f3f already in progress: creating tablet
W20260811 18:57:34.512362 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7d747f03ad234805a4ea4ef60840d426 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d747f03ad234805a4ea4ef60840d426 already in progress: creating tablet
I20260811 18:57:34.516027 4163 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.515317 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf22cbb415c485ebc1407d9104b715d" } modify_external_catalogs: true
I20260811 18:57:34.516245 4163 ts_tablet_manager.cc:1397] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.184s user 0.020s sys 0.032s
I20260811 18:57:34.516309 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf22cbb415c485ebc1407d9104b715d" } modify_external_catalogs: true
W20260811 18:57:34.517040 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 90fde17971ea49aaad212a5836661b08 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90fde17971ea49aaad212a5836661b08 already in progress: creating tablet
W20260811 18:57:34.517333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c0f68d33e1b14144b34c17e3c7201408 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f68d33e1b14144b34c17e3c7201408 already in progress: creating tablet
W20260811 18:57:34.517607 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 01a5fdab68694761b7493b42a0bf7c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01a5fdab68694761b7493b42a0bf7c5b already in progress: creating tablet
W20260811 18:57:34.517887 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c944d5cfd9b3405b8151abfc9f2fd998 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c944d5cfd9b3405b8151abfc9f2fd998 already in progress: creating tablet
W20260811 18:57:34.518169 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4e0747ef88574c68ae074732cb7095b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e0747ef88574c68ae074732cb7095b9 already in progress: creating tablet
W20260811 18:57:34.518401 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 531ba05901734d6b93bc2be506e8ec5b consensus state has no leader
W20260811 18:57:34.518590 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 99e9474ef8814514832f0e48c2b3495f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99e9474ef8814514832f0e48c2b3495f already in progress: creating tablet
W20260811 18:57:34.519114 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 761902f7366d43ffbde226a0ade2e188 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761902f7366d43ffbde226a0ade2e188 already in progress: creating tablet
I20260811 18:57:34.519024 4163 raft_consensus.cc:348] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.519522 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b3f0cdcee681460084662e01152ad9dd (table default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]) successfully deleted
W20260811 18:57:34.519848 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3dd4ec30acde4585a1c3451927217cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd4ec30acde4585a1c3451927217cb3 already in progress: creating tablet
W20260811 18:57:34.520323 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1eb30e64e86040098dcee5fc2e9b7266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1eb30e64e86040098dcee5fc2e9b7266 already in progress: creating tablet
W20260811 18:57:34.520617 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 35ebc32eb5d5443eaaeba245b566f98a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35ebc32eb5d5443eaaeba245b566f98a already in progress: creating tablet
I20260811 18:57:34.522102 4163 raft_consensus.cc:374] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.522254 4163 raft_consensus.cc:729] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.522558 4163 consensus_queue.cc:260] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.525287 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67" } modify_external_catalogs: true
I20260811 18:57:34.525521 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67" } modify_external_catalogs: true
I20260811 18:57:34.526262 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_acd269829eb743178d173c6cc3d7301e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.526970 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acd269829eb743178d173c6cc3d7301e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.508095 3678 leader_election.cc:334] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.529417 3678 leader_election.cc:302] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.529376 4167 rpcz_store.cc:276] 0811 18:57:31.819978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823490 (+ 3512us) service_pool.cc:227] Handling call
0811 18:57:34.471330 (+2647840us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:34.471344 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.488050 (+ 16706us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.502653 (+ 14603us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.506976 (+ 4323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2647646}
W20260811 18:57:34.529803 3678 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.529882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 887) took 2710 ms. Trace:
I20260811 18:57:34.529912 3678 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.529958 4167 rpcz_store.cc:276] 0811 18:57:31.819714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823753 (+ 4039us) service_pool.cc:227] Handling call
0811 18:57:34.529737 (+2705984us) ts_tablet_manager.cc:1145] Deleting tablet f1800e9e2b064a6fb546b9d33cefa68b
0811 18:57:34.529751 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.529864 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705831}
I20260811 18:57:34.530309 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 888) took 2710 ms. Trace:
I20260811 18:57:34.530436 5407 raft_consensus.cc:2726] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.530414 4167 rpcz_store.cc:276] 0811 18:57:31.819819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824166 (+ 4347us) service_pool.cc:227] Handling call
0811 18:57:34.530191 (+2706025us) ts_tablet_manager.cc:1145] Deleting tablet f2f31331974f42c3a22b3ac21b690f82
0811 18:57:34.530203 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.530292 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705834}
I20260811 18:57:34.530603 5407 raft_consensus.cc:2726] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.530740 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 886) took 2711 ms. Trace:
I20260811 18:57:34.530827 4167 rpcz_store.cc:276] 0811 18:57:31.819628 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824570 (+ 4942us) service_pool.cc:227] Handling call
0811 18:57:34.530632 (+2706062us) ts_tablet_manager.cc:1145] Deleting tablet b702aef88e124443888002d8ccd47b2f
0811 18:57:34.530643 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.530723 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705895}
I20260811 18:57:34.534202 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c4f7db8f684b436caef8f1315eadb0ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.534934 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4f7db8f684b436caef8f1315eadb0ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.533785 5443 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.539669 5443 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.540400 5443 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.541050 4163 ts_tablet_manager.cc:1428] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.001s sys 0.001s
I20260811 18:57:34.539463 4162 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.541473 4162 ts_tablet_manager.cc:1397] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
W20260811 18:57:34.542228 3678 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.542470 4162 raft_consensus.cc:348] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.542698 4162 raft_consensus.cc:374] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.542774 4162 raft_consensus.cc:729] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.543053 4162 consensus_queue.cc:260] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.541431 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00d943b778741b4aeec076ba46a9c40" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.545157 3676 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.545284 3676 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.546043 5407 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.549479 4162 ts_tablet_manager.cc:1428] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260811 18:57:34.555362 4162 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.559032 4167 tablet_replica.cc:330] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.559584 4167 raft_consensus.cc:2229] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.559864 4167 raft_consensus.cc:2258] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.560561 4167 ts_tablet_manager.cc:1905] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.563480 4163 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.563735 4162 tablet_bootstrap.cc:654] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.565179 4163 tablet_bootstrap.cc:654] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.571216 5443 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.571386 5443 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.572101 5443 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.573276 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b325df1db4fde9d4b26b559121316" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.574254 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_50769590dc5544e0a551369ba2c9768c" } new_table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" modify_external_catalogs: true
W20260811 18:57:34.574797 3676 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.575471 4167 ts_tablet_manager.cc:1918] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.575610 4167 log.cc:1196] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:34.576011 4167 ts_tablet_manager.cc:1939] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.574249 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e75355b4dffb4b4992e47c68f868d172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.576890 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 904) took 2460 ms. Trace:
W20260811 18:57:34.577181 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e75355b4dffb4b4992e47c68f868d172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.586040 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 374c94a12e374d0e82180421c80d92d0 consensus state has no leader
I20260811 18:57:34.586515 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet afa57ed239ac4afebe517cc48d879645 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.582391 4162 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:34.586704 3678 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.586789 4162 ts_tablet_manager.cc:1397] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20260811 18:57:34.586843 3678 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.587477 5443 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.587605 4162 raft_consensus.cc:348] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.587930 4162 raft_consensus.cc:374] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.588006 4162 raft_consensus.cc:729] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.588230 4162 consensus_queue.cc:260] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.588847 4162 ts_tablet_manager.cc:1428] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:34.588944 4167 rpcz_store.cc:276] 0811 18:57:32.116369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.116816 (+ 447us) service_pool.cc:227] Handling call
0811 18:57:34.558966 (+2442150us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:34.558982 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.572072 (+ 13090us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.575363 (+ 3291us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.576866 (+ 1503us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2441915}
I20260811 18:57:34.594655 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 968fa8b8912b40a8bdba77db1d39c5c0 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.596254 4162 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.598510 4169 ts_tablet_manager.cc:1918] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.598773 4169 log.cc:1196] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fbba1923ce0049448fe4e28f2dd258dc
I20260811 18:57:34.599339 4169 ts_tablet_manager.cc:1939] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.599807 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 860) took 2778 ms. Trace:
I20260811 18:57:34.603310 5443 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.603472 5443 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.604125 5443 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.606609 5443 raft_consensus.cc:482] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.606758 5443 raft_consensus.cc:504] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.607322 5443 leader_election.cc:288] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:34.607967 3678 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.609200 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aa3debf8d8774103a45a754a6f08465a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.601878 4167 tablet_replica.cc:330] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.610028 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa3debf8d8774103a45a754a6f08465a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.610139 4167 raft_consensus.cc:2229] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.610582 4167 raft_consensus.cc:2258] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:34.610373 3678 leader_election.cc:334] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.611495 4167 ts_tablet_manager.cc:1905] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.615950 4162 tablet_bootstrap.cc:654] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.622766 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8c07150e31994bdf9a00ac2e7a954691 (table default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]) successfully deleted
W20260811 18:57:34.623220 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1800e9e2b064a6fb546b9d33cefa68b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1800e9e2b064a6fb546b9d33cefa68b already in progress: creating tablet
W20260811 18:57:34.623593 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f2f31331974f42c3a22b3ac21b690f82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f31331974f42c3a22b3ac21b690f82 already in progress: creating tablet
I20260811 18:57:34.623884 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:34.624074 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b702aef88e124443888002d8ccd47b2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b702aef88e124443888002d8ccd47b2f already in progress: creating tablet
I20260811 18:57:34.624485 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f7d4c26da86b49b8a37f2289f4ef6614 (table default.Table_523277d801974e708b0f7e9c401bb36a [id=5600dcaaf83c4f978c3d47f79e01568c]) successfully deleted
I20260811 18:57:34.627728 4167 ts_tablet_manager.cc:1918] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.636221 4167 log.cc:1196] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:34.636984 4167 ts_tablet_manager.cc:1939] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.638154 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_97e3f5df19fc4b04a37f62286b42f7e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.638976 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97e3f5df19fc4b04a37f62286b42f7e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.644762 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 903) took 2530 ms. Trace:
I20260811 18:57:34.644954 4167 rpcz_store.cc:276] 0811 18:57:32.114402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.121504 (+ 7102us) service_pool.cc:227] Handling call
0811 18:57:34.589222 (+2467718us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:34.601819 (+ 12597us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.623870 (+ 22051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.627623 (+ 3753us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.644733 (+ 17110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2463676}
I20260811 18:57:34.645664 4162 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.645823 4162 ts_tablet_manager.cc:1397] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.003s
I20260811 18:57:34.645943 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 905) took 2464 ms. Trace:
I20260811 18:57:34.646414 4167 rpcz_store.cc:276] 0811 18:57:32.181526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181613 (+ 87us) service_pool.cc:227] Handling call
0811 18:57:34.645606 (+2463993us) ts_tablet_manager.cc:1145] Deleting tablet ccc63b46a4e346f79d1e03a82490794c
0811 18:57:34.645617 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.645924 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2463737}
I20260811 18:57:34.646695 4162 raft_consensus.cc:348] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.646987 4162 raft_consensus.cc:374] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.647089 4162 raft_consensus.cc:729] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.647312 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 906) took 2465 ms. Trace:
I20260811 18:57:34.648411 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a5361d65b50b445f9d85f5ddda046f99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.648900 4167 rpcz_store.cc:276] 0811 18:57:32.181678 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181922 (+ 244us) service_pool.cc:227] Handling call
0811 18:57:34.646934 (+2465012us) ts_tablet_manager.cc:1145] Deleting tablet cdb92902c2584751af77d3117f52cb4e
0811 18:57:34.646947 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.647295 (+ 348us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2464835}
W20260811 18:57:34.649176 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5361d65b50b445f9d85f5ddda046f99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.647341 4162 consensus_queue.cc:260] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.652415 4162 ts_tablet_manager.cc:1428] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:34.652783 4162 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.654590 4162 tablet_bootstrap.cc:654] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.604847 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.608057 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cccebfc54484a9b9abffcce4e459fb5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.658990 5417 raft_consensus.cc:482] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.661927 5417 raft_consensus.cc:504] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:34.662132 3676 leader_election.cc:341] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.662271 3676 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.662722 5417 leader_election.cc:288] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:34.662717 3676 leader_election.cc:341] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.662834 3676 leader_election.cc:302] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.662907 4169 rpcz_store.cc:276] 0811 18:57:31.821447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823296 (+ 1849us) service_pool.cc:227] Handling call
0811 18:57:34.415573 (+2592277us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:34.415589 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.523072 (+107483us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.598333 (+ 75261us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.599784 (+ 1451us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2591888}
I20260811 18:57:34.663160 5443 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.663370 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 857) took 2861 ms. Trace:
I20260811 18:57:34.663282 5443 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.663457 4169 rpcz_store.cc:276] 0811 18:57:31.801530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.821842 (+ 20312us) service_pool.cc:227] Handling call
0811 18:57:34.663207 (+2841365us) ts_tablet_manager.cc:1145] Deleting tablet b702aef88e124443888002d8ccd47b2f
0811 18:57:34.663225 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.663349 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839188}
I20260811 18:57:34.663868 5443 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.663908 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 858) took 2849 ms. Trace:
I20260811 18:57:34.664012 5443 raft_consensus.cc:2726] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.663988 4169 rpcz_store.cc:276] 0811 18:57:31.814960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824460 (+ 9500us) service_pool.cc:227] Handling call
0811 18:57:34.663769 (+2839309us) ts_tablet_manager.cc:1145] Deleting tablet f1800e9e2b064a6fb546b9d33cefa68b
0811 18:57:34.663782 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.663892 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839148}
I20260811 18:57:34.664162 5443 raft_consensus.cc:2726] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.664309 4169 tablet_replica.cc:330] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.664513 4169 raft_consensus.cc:2229] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.665253 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 907) took 2468 ms. Trace:
W20260811 18:57:34.667845 3678 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.669201 4167 rpcz_store.cc:276] 0811 18:57:32.197315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.197413 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:34.665113 (+2467700us) ts_tablet_manager.cc:1145] Deleting tablet 99de8ce45e514ff291cb9e5c0a4c00d1
0811 18:57:34.665125 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.665232 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2467432}
I20260811 18:57:34.670622 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f335e50ae57547ed80b144129faaffbb (table default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]) successfully deleted
W20260811 18:57:34.671031 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because the tablet was not found. No further retry: Not found: Tablet not found: 67e63ee9b9fd4b12ae791b9d5bc2e648
W20260811 18:57:34.671773 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 99e9474ef8814514832f0e48c2b3495f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99e9474ef8814514832f0e48c2b3495f already in progress: creating tablet
I20260811 18:57:34.672556 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_485cdcb897e6430ba77e994afbd44a64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.669940 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:34.673319 3743 raft_consensus.cc:2454] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:34.673461 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_485cdcb897e6430ba77e994afbd44a64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.676028 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 761902f7366d43ffbde226a0ade2e188 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761902f7366d43ffbde226a0ade2e188 already in progress: creating tablet
W20260811 18:57:34.676690 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 35ebc32eb5d5443eaaeba245b566f98a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35ebc32eb5d5443eaaeba245b566f98a already in progress: creating tablet
W20260811 18:57:34.677286 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1eb30e64e86040098dcee5fc2e9b7266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1eb30e64e86040098dcee5fc2e9b7266 already in progress: creating tablet
W20260811 18:57:34.677759 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3dd4ec30acde4585a1c3451927217cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd4ec30acde4585a1c3451927217cb3 already in progress: creating tablet
I20260811 18:57:34.671820 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 908) took 2474 ms. Trace:
W20260811 18:57:34.678928 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f2f31331974f42c3a22b3ac21b690f82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f31331974f42c3a22b3ac21b690f82 already in progress: creating tablet
I20260811 18:57:34.679379 4167 rpcz_store.cc:276] 0811 18:57:32.197483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.197732 (+ 249us) service_pool.cc:227] Handling call
0811 18:57:34.670630 (+2472898us) ts_tablet_manager.cc:1145] Deleting tablet 14799a6f61a946008acee8a09d050c1e
0811 18:57:34.670643 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.671797 (+ 1154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2472750}
I20260811 18:57:34.680956 4167 tablet_replica.cc:330] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.682240 4167 raft_consensus.cc:2229] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.684160 4167 raft_consensus.cc:2258] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.686313 4167 ts_tablet_manager.cc:1905] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.690299 5490 raft_consensus.cc:482] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.692790 5490 raft_consensus.cc:504] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.695304 3678 leader_election.cc:334] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.695366 5490 leader_election.cc:288] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.696635 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bfb54175c834517b7af291c942333cd
I20260811 18:57:34.697198 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0bfb54175c834517b7af291c942333cd deleted and replaced by tablet 13af4d419ae542cc961c75f28efd0204
I20260811 18:57:34.698865 4169 pending_rounds.cc:62] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.698949 4169 pending_rounds.cc:68] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400173454458880 op_type: NO_OP noop_request { }
I20260811 18:57:34.699209 4169 raft_consensus.cc:2866] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:34.699370 4169 raft_consensus.cc:2258] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.680259 4163 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.683394 5417 consensus_queue.cc:1035] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:34.683650 3824 leader_election.cc:341] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:34.685091 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3f120758c6405dafd2ea0bf54c81d8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.700596 3824 consensus_peers.cc:487] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 122b7898fea44ef78f269b1505d8ddc4. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.695492 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce94d37efd2245bf8cc71fdd90bd9735" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.701005 3676 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.701125 3676 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.701030 3824 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:34.701352 5490 raft_consensus.cc:2726] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.701864 3676 leader_election.cc:341] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.701973 3676 leader_election.cc:302] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.702311 5490 raft_consensus.cc:2726] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.704167 4162 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.705318 4162 ts_tablet_manager.cc:1397] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.015s
I20260811 18:57:34.706037 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e03268eb26a144ba8b53cbd960628036" } new_table_name: "default.Table_5cf0c22ada454c098b23f5915efe0380" modify_external_catalogs: true
I20260811 18:57:34.712174 4162 raft_consensus.cc:348] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.714032 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1c1896700e724204a37439f134a4d988
I20260811 18:57:34.714243 4162 raft_consensus.cc:374] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.715088 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7de254c88a674097b5df7ea987172a9e" } new_table_name: "default.Table_7f65859fb42941e3a8375f65182f9ec4" modify_external_catalogs: true
I20260811 18:57:34.714994 4162 raft_consensus.cc:729] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.716747 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4dd203b7eb8643f493b4642c0396e258
I20260811 18:57:34.717469 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6953ae59bab4944a8db87e863735f79" } new_table_name: "default.Table_76b86d4f079c45b1a7f34a195fc97dfb" modify_external_catalogs: true
I20260811 18:57:34.717042 4162 consensus_queue.cc:260] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.724790 4162 ts_tablet_manager.cc:1428] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.002s sys 0.001s
I20260811 18:57:34.726403 5417 raft_consensus.cc:2781] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:34.726526 5417 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:34.726595 5417 raft_consensus.cc:3037] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.729628 5417 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.730939 5417 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.733464 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet afa57ed239ac4afebe517cc48d879645
I20260811 18:57:34.734031 3551 catalog_manager.cc:6565] ReplaceTablet: tablet afa57ed239ac4afebe517cc48d879645 deleted and replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5
I20260811 18:57:34.735067 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b27f9dbb52f34687a33c2ebf20e8f780
I20260811 18:57:34.738492 4167 ts_tablet_manager.cc:1918] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.738652 4167 log.cc:1196] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:34.738991 4167 ts_tablet_manager.cc:1939] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.740938 3824 leader_election.cc:341] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 43ad6993b63e4bbd98c8cc52f1a53451
W20260811 18:57:34.741910 3824 consensus_peers.cc:487] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c26851245cab4a2d80e94d4fe278fd50. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.741353 4163 ts_tablet_manager.cc:1397] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.178s user 0.002s sys 0.001s
I20260811 18:57:34.743551 5376 consensus_queue.cc:1035] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:34.744426 4163 raft_consensus.cc:348] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.746709 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 909) took 2548 ms. Trace:
I20260811 18:57:34.746795 4163 raft_consensus.cc:374] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.746906 4167 rpcz_store.cc:276] 0811 18:57:32.198466 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.198523 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:34.680901 (+2482378us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:34.680914 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.735089 (+ 54175us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.738372 (+ 3283us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.746681 (+ 8309us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":226,"tablet-delete.queue_time_us":2482168}
I20260811 18:57:34.747690 4167 tablet_replica.cc:330] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.747639 4163 raft_consensus.cc:729] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.748039 4167 raft_consensus.cc:2229] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.747936 4163 consensus_queue.cc:260] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.700099 4169 ts_tablet_manager.cc:1905] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.748364 4167 raft_consensus.cc:2258] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.749012 4167 ts_tablet_manager.cc:1905] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.745266 3824 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:34.752547 4162 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.754719 5490 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.754878 5490 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.755577 5490 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.756821 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.757468 3676 leader_election.cc:341] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.757691 3678 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.757786 3678 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.757988 5376 raft_consensus.cc:2781] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:34.758095 5376 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:34.758172 5376 raft_consensus.cc:3037] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.758419 5490 raft_consensus.cc:2726] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.761323 5376 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.762319 5376 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.763672 4167 ts_tablet_manager.cc:1918] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.763873 4167 log.cc:1196] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/58c515c153df4317b3fb427983ee15e0
I20260811 18:57:34.764267 4167 ts_tablet_manager.cc:1939] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.765214 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 910) took 2487 ms. Trace:
I20260811 18:57:34.765362 4167 rpcz_store.cc:276] 0811 18:57:32.277478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.278560 (+ 1082us) service_pool.cc:227] Handling call
0811 18:57:34.747633 (+2469073us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:34.747648 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.752462 (+ 4814us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.763550 (+ 11088us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.765186 (+ 1636us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2457641}
I20260811 18:57:34.766060 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 911) took 2488 ms. Trace:
I20260811 18:57:34.766314 4167 rpcz_store.cc:276] 0811 18:57:32.277633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.278678 (+ 1045us) service_pool.cc:227] Handling call
0811 18:57:34.765920 (+2487242us) ts_tablet_manager.cc:1145] Deleting tablet e1ad3860d03f46baadd5ede854f1a909
0811 18:57:34.765933 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.766043 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2475833}
I20260811 18:57:34.766819 4167 tablet_replica.cc:330] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.767089 4167 raft_consensus.cc:2229] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.767459 4167 raft_consensus.cc:2258] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.768321 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c08436f33db043b1bc6395cd92102e59 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
I20260811 18:57:34.768797 4167 ts_tablet_manager.cc:1905] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.776235 4167 ts_tablet_manager.cc:1918] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.776419 4167 log.cc:1196] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:34.776760 4167 ts_tablet_manager.cc:1939] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.777633 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 912) took 2215 ms. Trace:
I20260811 18:57:34.777766 4167 rpcz_store.cc:276] 0811 18:57:32.562219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.562289 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:34.766769 (+2204480us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:34.766781 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.772572 (+ 5791us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.776126 (+ 3554us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.777610 (+ 1484us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2204204}
I20260811 18:57:34.778586 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 921) took 2115 ms. Trace:
I20260811 18:57:34.778702 4167 rpcz_store.cc:276] 0811 18:57:32.662850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.666422 (+ 3572us) service_pool.cc:227] Handling call
0811 18:57:34.778295 (+2111873us) ts_tablet_manager.cc:1145] Deleting tablet 0a5b06fa572045f3a5fc5a0b123d77cc
0811 18:57:34.778308 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.778567 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2111567}
I20260811 18:57:34.779222 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 918) took 2121 ms. Trace:
I20260811 18:57:34.779373 4167 rpcz_store.cc:276] 0811 18:57:32.658006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.668878 (+ 10872us) service_pool.cc:227] Handling call
0811 18:57:34.779105 (+2110227us) ts_tablet_manager.cc:1145] Deleting tablet 844bcd16f06c4788af9c0d7d320464aa
0811 18:57:34.779116 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.779206 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2109944}
I20260811 18:57:34.779844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 932) took 1909 ms. Trace:
I20260811 18:57:34.779945 4167 rpcz_store.cc:276] 0811 18:57:32.870869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.874715 (+ 3846us) service_pool.cc:227] Handling call
0811 18:57:34.779746 (+1905031us) ts_tablet_manager.cc:1145] Deleting tablet e7d75d8468fc4c938fde0b2951d84562
0811 18:57:34.779757 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.779830 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1904668}
I20260811 18:57:34.780367 4167 tablet_replica.cc:330] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.780628 4167 raft_consensus.cc:2229] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.780815 4167 raft_consensus.cc:2258] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.781430 4167 ts_tablet_manager.cc:1905] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.782358 4163 ts_tablet_manager.cc:1428] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
W20260811 18:57:34.783121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b702aef88e124443888002d8ccd47b2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b702aef88e124443888002d8ccd47b2f already in progress: creating tablet
W20260811 18:57:34.783495 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1800e9e2b064a6fb546b9d33cefa68b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1800e9e2b064a6fb546b9d33cefa68b already in progress: creating tablet
W20260811 18:57:34.784664 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 91994.7us
I20260811 18:57:34.792405 4167 ts_tablet_manager.cc:1918] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.792577 4167 log.cc:1196] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7251204325dd424192a6f58e18e71c34
I20260811 18:57:34.792924 4167 ts_tablet_manager.cc:1939] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.793624 4163 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.793912 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 941) took 1676 ms. Trace:
I20260811 18:57:34.794032 4167 rpcz_store.cc:276] 0811 18:57:33.117658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.117733 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:34.780322 (+1662589us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:34.780332 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.785812 (+ 5480us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.792275 (+ 6463us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.793889 (+ 1614us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1662309}
I20260811 18:57:34.794579 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 942) took 1647 ms. Trace:
I20260811 18:57:34.794723 4167 rpcz_store.cc:276] 0811 18:57:33.146654 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.146722 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:34.794434 (+1647712us) ts_tablet_manager.cc:1145] Deleting tablet 6df5545532e346f7abcdc4cd228b84da
0811 18:57:34.794449 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.794558 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1647470}
I20260811 18:57:34.795033 4167 tablet_replica.cc:330] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.795255 4167 raft_consensus.cc:2229] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.795423 4167 raft_consensus.cc:2258] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.796036 4167 ts_tablet_manager.cc:1905] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:34.795830 3824 leader_election.cc:334] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.798460 3824 leader_election.cc:302] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:34.802798 4167 ts_tablet_manager.cc:1918] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.802989 4167 log.cc:1196] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:34.803040 4162 tablet_bootstrap.cc:654] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.804008 4163 tablet_bootstrap.cc:654] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.806311 5490 raft_consensus.cc:482] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.806495 5490 raft_consensus.cc:504] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.806668 4169 ts_tablet_manager.cc:1918] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.806784 4169 log.cc:1196] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:34.807186 4169 ts_tablet_manager.cc:1939] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.807232 5490 leader_election.cc:288] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.809407 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d32bb4eca047e4ba786c04b8bebf8b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.812016 3676 leader_election.cc:341] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.803333 4167 ts_tablet_manager.cc:1939] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.816901 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 943) took 1614 ms. Trace:
I20260811 18:57:34.822567 4167 rpcz_store.cc:276] 0811 18:57:33.202791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.202864 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:34.794984 (+1592120us) ts_tablet_manager.cc:1145] Deleting tablet b2718cd4a98742dfa8014a8a38c69c45
0811 18:57:34.794994 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.799653 (+ 4659us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.802702 (+ 3049us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.816876 (+ 14174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591905}
W20260811 18:57:34.814810 3678 leader_election.cc:334] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.823192 3678 leader_election.cc:302] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.823314 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 944) took 1572 ms. Trace:
I20260811 18:57:34.823414 4167 rpcz_store.cc:276] 0811 18:57:33.251387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.251467 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:34.823160 (+1571693us) ts_tablet_manager.cc:1145] Deleting tablet 4af28c051e8243eaa73d4a5ed1abcea6
0811 18:57:34.823175 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.823295 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1571384}
I20260811 18:57:34.823702 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 945) took 1449 ms. Trace:
I20260811 18:57:34.823776 4167 rpcz_store.cc:276] 0811 18:57:33.374541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.374626 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:34.823618 (+1448992us) ts_tablet_manager.cc:1145] Deleting tablet 9608d6ddcad749cd9e72dd6f265db0da
0811 18:57:34.823627 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.823690 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1448755}
I20260811 18:57:34.823933 5490 raft_consensus.cc:2726] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.823993 4167 tablet_replica.cc:330] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.815939 4162 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.824397 4162 ts_tablet_manager.cc:1397] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.073s user 0.006s sys 0.000s
I20260811 18:57:34.825287 4162 raft_consensus.cc:348] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.825727 4162 raft_consensus.cc:374] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.825991 4162 raft_consensus.cc:729] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.826259 4162 consensus_queue.cc:260] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.828876 4167 raft_consensus.cc:2229] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.829051 4167 raft_consensus.cc:2258] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.829595 4167 ts_tablet_manager.cc:1905] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.831198 4162 ts_tablet_manager.cc:1428] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:34.831491 5490 raft_consensus.cc:482] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.831653 5490 raft_consensus.cc:504] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.832238 5490 leader_election.cc:288] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.833318 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c8d542ae2489ba466ffadd518db12" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.833981 3676 leader_election.cc:341] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.848726 5376 raft_consensus.cc:2726] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.849236 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 862) took 3027 ms. Trace:
I20260811 18:57:34.849349 4169 rpcz_store.cc:276] 0811 18:57:31.821632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.825444 (+ 3812us) service_pool.cc:227] Handling call
0811 18:57:34.664265 (+2838821us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:34.664275 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.803163 (+138888us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.806569 (+ 3406us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.849209 (+ 42640us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41070,"tablet-delete.queue_time_us":2838648}
I20260811 18:57:34.849840 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 875) took 2668 ms. Trace:
I20260811 18:57:34.849918 4169 rpcz_store.cc:276] 0811 18:57:32.181388 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181581 (+ 193us) service_pool.cc:227] Handling call
0811 18:57:34.849684 (+2668103us) ts_tablet_manager.cc:1145] Deleting tablet cdb92902c2584751af77d3117f52cb4e
0811 18:57:34.849697 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.849824 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2640023}
I20260811 18:57:34.850267 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 876) took 2636 ms. Trace:
I20260811 18:57:34.850411 4169 rpcz_store.cc:276] 0811 18:57:32.213886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.215511 (+ 1625us) service_pool.cc:227] Handling call
0811 18:57:34.850134 (+2634623us) ts_tablet_manager.cc:1145] Deleting tablet ccc63b46a4e346f79d1e03a82490794c
0811 18:57:34.850157 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.850252 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2634335}
I20260811 18:57:34.850828 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 878) took 2633 ms. Trace:
I20260811 18:57:34.850912 4169 rpcz_store.cc:276] 0811 18:57:32.216998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.217086 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:34.850687 (+2633601us) ts_tablet_manager.cc:1145] Deleting tablet 14799a6f61a946008acee8a09d050c1e
0811 18:57:34.850700 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.850809 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633338}
I20260811 18:57:34.851256 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 877) took 2634 ms. Trace:
I20260811 18:57:34.851334 4169 rpcz_store.cc:276] 0811 18:57:32.216870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.220032 (+ 3162us) service_pool.cc:227] Handling call
0811 18:57:34.851128 (+2631096us) ts_tablet_manager.cc:1145] Deleting tablet 99de8ce45e514ff291cb9e5c0a4c00d1
0811 18:57:34.851139 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.851239 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2630876}
I20260811 18:57:34.851648 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 879) took 2616 ms. Trace:
I20260811 18:57:34.851724 4169 rpcz_store.cc:276] 0811 18:57:32.235277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.236166 (+ 889us) service_pool.cc:227] Handling call
0811 18:57:34.851538 (+2615372us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:34.851549 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.851632 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2615174}
I20260811 18:57:34.852083 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 880) took 2578 ms. Trace:
I20260811 18:57:34.852164 4169 rpcz_store.cc:276] 0811 18:57:32.274128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.274201 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:34.851954 (+2577753us) ts_tablet_manager.cc:1145] Deleting tablet e1ad3860d03f46baadd5ede854f1a909
0811 18:57:34.851965 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.852067 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2577468}
I20260811 18:57:34.852416 4169 tablet_replica.cc:330] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.852674 4169 raft_consensus.cc:2229] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.852943 4169 pending_rounds.cc:62] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.852994 4169 pending_rounds.cc:68] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400178813452288 op_type: NO_OP noop_request { }
I20260811 18:57:34.853134 4169 raft_consensus.cc:2866] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:34.853276 4169 raft_consensus.cc:2258] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.854584 4169 ts_tablet_manager.cc:1905] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.861030 5376 raft_consensus.cc:482] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.861207 5376 raft_consensus.cc:504] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.861845 5376 leader_election.cc:288] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.862911 4169 ts_tablet_manager.cc:1918] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.864864 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d37e81d376e14907b13746cca831a96b
I20260811 18:57:34.866585 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f86bfa65a8774b51bc7e7fe59442bf2f
W20260811 18:57:34.835907 3678 leader_election.cc:334] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.845440 4167 ts_tablet_manager.cc:1918] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.872424 4167 log.cc:1196] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:34.872817 4167 ts_tablet_manager.cc:1939] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.873189 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 946) took 1130 ms. Trace:
I20260811 18:57:34.873281 4167 rpcz_store.cc:276] 0811 18:57:33.742841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.743076 (+ 235us) service_pool.cc:227] Handling call
0811 18:57:34.823954 (+1080878us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:34.823962 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.833054 (+ 9092us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.845311 (+ 12257us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.873172 (+ 27861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1080605}
I20260811 18:57:34.873340 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d9a2caa47da74f95bd2276d3b1547505"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.874195 4167 tablet_replica.cc:330] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.874353 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9a2caa47da74f95bd2276d3b1547505 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.874581 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_5627fd661e4441d8a677d35bc0f16cf0 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 3 got 0
I20260811 18:57:34.874836 4167 raft_consensus.cc:2229] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.875005 4167 raft_consensus.cc:2258] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.875546 4167 ts_tablet_manager.cc:1905] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.875555 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b2be35bff506498281ab88f9cbb55de8" } new_table_name: "default.Table_0cde3a66762e4c0ea53da6ce0999bf3c" modify_external_catalogs: true
I20260811 18:57:34.835273 4162 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.877939 4162 tablet_bootstrap.cc:654] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.878720 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cf1083177b984dbc8f987dc5575ae11a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.879323 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf1083177b984dbc8f987dc5575ae11a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.880829 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.881424 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f4984467f654126ae2d4a7fe64c94f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.872399 3678 leader_election.cc:302] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.882436 5490 raft_consensus.cc:2726] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.882704 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46803.8us
I20260811 18:57:34.882840 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ae1b2ddf92b411f84f56e6a92130fa8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.883468 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.888016 4167 ts_tablet_manager.cc:1918] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.888175 4167 log.cc:1196] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:34.888530 4167 ts_tablet_manager.cc:1939] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.889868 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_5627fd661e4441d8a677d35bc0f16cf0 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 3 got 0
I20260811 18:57:34.891690 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fab5512bddad41239e285f5b77ef1958"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.892555 4167 tablet_replica.cc:330] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.892628 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fab5512bddad41239e285f5b77ef1958 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.884410 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_567110d2149d4cbfbfcf11fbbaddd0fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.894239 4167 raft_consensus.cc:2229] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:34.894590 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_567110d2149d4cbfbfcf11fbbaddd0fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.894616 4167 raft_consensus.cc:2258] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.895227 4167 ts_tablet_manager.cc:1905] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.896243 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8518b7d355d14d14983f2c89aad83f5e" } new_table_name: "default.Table_b493ffc53fa84d9983273e1dabc3fa40" modify_external_catalogs: true
W20260811 18:57:34.899528 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
W20260811 18:57:34.901069 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:34.910450 4169 log.cc:1196] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:34.911156 4169 ts_tablet_manager.cc:1939] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.921674 5490 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.921876 5490 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.922571 5490 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:34.924713 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.925510 4167 ts_tablet_manager.cc:1918] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.932961 4167 log.cc:1196] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:34.933573 4167 ts_tablet_manager.cc:1939] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.933372 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd16e9b8092642c896a5be846347ad41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.934149 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd16e9b8092642c896a5be846347ad41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.935873 4167 tablet_replica.cc:330] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.936486 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 46505ba544104458888dbc2a37115dcc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.936729 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 881) took 2378 ms. Trace:
I20260811 18:57:34.936849 4169 rpcz_store.cc:276] 0811 18:57:32.558479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.558559 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:34.852371 (+2293812us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:34.852381 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.858296 (+ 5915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.862816 (+ 4520us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.936704 (+ 73888us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2259814}
I20260811 18:57:34.937408 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 887) took 2287 ms. Trace:
I20260811 18:57:34.937498 4169 rpcz_store.cc:276] 0811 18:57:32.650161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.651393 (+ 1232us) service_pool.cc:227] Handling call
0811 18:57:34.937252 (+2285859us) ts_tablet_manager.cc:1145] Deleting tablet 844bcd16f06c4788af9c0d7d320464aa
0811 18:57:34.937267 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.937383 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2285674}
I20260811 18:57:34.937870 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 890) took 2287 ms. Trace:
I20260811 18:57:34.937956 4169 rpcz_store.cc:276] 0811 18:57:32.650535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.650733 (+ 198us) service_pool.cc:227] Handling call
0811 18:57:34.937763 (+2287030us) ts_tablet_manager.cc:1145] Deleting tablet 0a5b06fa572045f3a5fc5a0b123d77cc
0811 18:57:34.937775 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.937852 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2286130}
I20260811 18:57:34.938284 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 902) took 2042 ms. Trace:
I20260811 18:57:34.938331 4167 raft_consensus.cc:2229] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.938388 4169 rpcz_store.cc:276] 0811 18:57:32.896031 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.918456 (+ 22425us) service_pool.cc:227] Handling call
0811 18:57:34.938177 (+2019721us) ts_tablet_manager.cc:1145] Deleting tablet e7d75d8468fc4c938fde0b2951d84562
0811 18:57:34.938188 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.938265 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2019460}
I20260811 18:57:34.938540 4167 raft_consensus.cc:2258] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.938671 4169 tablet_replica.cc:330] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.938895 4169 raft_consensus.cc:2229] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.939122 4167 ts_tablet_manager.cc:1905] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.906885 4163 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.923656 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.850251 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 55173.3us
W20260811 18:57:34.942520 3676 leader_election.cc:341] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.942665 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.942942 3824 leader_election.cc:334] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.943063 3824 leader_election.cc:302] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:34.943066 5490 raft_consensus.cc:2726] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.943337 5417 raft_consensus.cc:2726] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.944016 3824 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.938830 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae02c9b0b066474894546829aed304e3 (table default.Table_76b86d4f079c45b1a7f34a195fc97dfb [id=137cef1d67054dc6be05fcef9b22c375], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:34.946537 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:34.946764 3741 raft_consensus.cc:3037] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.947350 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:34.947530 3744 raft_consensus.cc:3037] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.949812 3741 raft_consensus.cc:2454] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:34.950126 3744 raft_consensus.cc:2454] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:34.950629 4163 ts_tablet_manager.cc:1397] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.157s user 0.006s sys 0.001s
I20260811 18:57:34.955235 4163 raft_consensus.cc:348] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.957461 4163 raft_consensus.cc:374] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.958292 4163 raft_consensus.cc:729] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.962972 4162 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.961867 4163 consensus_queue.cc:260] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.964939 4162 ts_tablet_manager.cc:1397] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.129s user 0.005s sys 0.000s
I20260811 18:57:34.967805 5417 raft_consensus.cc:482] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.967965 5417 raft_consensus.cc:504] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.968619 5417 leader_election.cc:288] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.969379 5417 raft_consensus.cc:482] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.969509 5417 raft_consensus.cc:504] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.970078 5417 leader_election.cc:288] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.970414 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 40898ceb18cc4f2d9104818798049ae2 with delete_type TABLET_DATA_DELETED (replaced by tablet a62cdd333ad8494897ae8ec5afd5af74) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.970842 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ed141f91ec548db87ac1e719256931b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.969846 4169 pending_rounds.cc:62] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.970062 4163 ts_tablet_manager.cc:1428] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260811 18:57:34.971737 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.971539 4169 pending_rounds.cc:68] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400180473872384 op_type: NO_OP noop_request { }
I20260811 18:57:34.974300 4163 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.973603 4169 raft_consensus.cc:2866] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260811 18:57:34.976393 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:34.975629 4169 raft_consensus.cc:2258] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.976583 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.976796 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 302cf69ea7c14fdaac72f6a9a9aff113 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.977417 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.978956 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8eaafc7677344cac8d9616996c5f416c (DEFAULT_TABLE table=default.Table_7e50906e16414e8b99c43763a538d6be [id=dab5f1b103eb489fb1bbb15659b424ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.979668 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eaafc7677344cac8d9616996c5f416c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.979809 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 38186aed20134de1bc06e0eb13806d34 (DEFAULT_TABLE table=default.Table_4464846117e74da2b263ff545f9937b3 [id=5c41b99d64b54cdbb7c2424ce5e9483b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.980195 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38186aed20134de1bc06e0eb13806d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.980492 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f1e69da8480425682a8348cad2ffa51" } modify_external_catalogs: true
I20260811 18:57:34.980779 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f1e69da8480425682a8348cad2ffa51" } modify_external_catalogs: true
I20260811 18:57:34.980510 4162 raft_consensus.cc:348] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.985038 4162 raft_consensus.cc:374] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.986029 4169 ts_tablet_manager.cc:1905] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.987449 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 (DEFAULT_TABLE table=default.Table_3ee627ac9abf47889b5fed972cecd97e [id=17c62f2fe48546309ce0f54050e43299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.987941 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccd56a8b7cc454ea3f2f08b59374521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.988091 3856 tablet_service.cc:1467] Processing CreateTablet for tablet fae3750e5870411e8fe7dedae2fe3ec0 (DEFAULT_TABLE table=default.Table_be8ef75ca590418486730a869708b637 [id=07e28abb2e494245999c05a44c9107fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.988510 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae3750e5870411e8fe7dedae2fe3ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.988883 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.991767 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aa50e46e061b41f79d5d6d048ea15132" } modify_external_catalogs: true
I20260811 18:57:34.991998 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aa50e46e061b41f79d5d6d048ea15132" } modify_external_catalogs: true
I20260811 18:57:34.993060 5417 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.993201 5417 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.993788 5417 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.994947 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1896700e724204a37439f134a4d988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.995570 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dd203b7eb8643f493b4642c0396e258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.996107 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet afa57ed239ac4afebe517cc48d879645 with delete_type TABLET_DATA_DELETED (replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.996876 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:34.939157 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 94193.6us
I20260811 18:57:34.997524 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 36861ae786c549d687e687fe853d1801 (DEFAULT_TABLE table=default.Table_aa50e46e061b41f79d5d6d048ea15132 [id=5ef75d1e27a8460e83162dd6e254ab9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.997972 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36861ae786c549d687e687fe853d1801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.003962 4163 tablet_bootstrap.cc:654] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.004680 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a" } modify_external_catalogs: true
I20260811 18:57:35.004985 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a" } modify_external_catalogs: true
I20260811 18:57:34.986721 4162 raft_consensus.cc:729] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.009186 4162 consensus_queue.cc:260] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.009852 4162 ts_tablet_manager.cc:1428] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:35.015811 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 09913224068f4a71bab606dea187daee (DEFAULT_TABLE table=default.Table_1afc7507935345b2a081a64ed01106da [id=9b590a32f284495ca2293a9e3b7e9e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.015936 3852 tablet_service.cc:1467] Processing CreateTablet for tablet ad8f24cdcf1c4973b388303fce0a5c08 (DEFAULT_TABLE table=default.Table_2033e26987554c839ae47bd350d82580 [id=4a215557a1474b22bffc1f4ba8a16025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.015940 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f15b0581b33b4796b527e87caf606eb2 (DEFAULT_TABLE table=default.Table_fae8f41d600f4fbfaed39f695cf6672a [id=a98ff2e2e2c14d4fa18822452154a31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.016847 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 6a70ecf0db7641419513558768d4c882 (DEFAULT_TABLE table=default.Table_58b2359b8659428190fe3cd7688c1d4d [id=df01d48865104be1ba7895f5bed2f809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017025 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 61f4112fcb7445b3803456e59a24ad0c (DEFAULT_TABLE table=default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b [id=6ec509fc5d6449dc8493d9d72c95f279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017060 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa4822d4c1149c29187c9ac880766e3 (DEFAULT_TABLE table=default.Table_2fc380eb763b488e815a72be20c8f563 [id=611c2ca37d194e2a886789d9291ad3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.016801 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 15855cd92875485c96b2eb2082e88a0b (DEFAULT_TABLE table=default.Table_6080085ba7594b18b5594e8868f0b80e [id=6e4857cbdcab4ce59110f567698f8f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017525 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 762db87ede7a490da344412e0ff2ff98 (DEFAULT_TABLE table=default.Table_f414c31b8cff4e889368e4207afb1b53 [id=0822821331174d16b7f07c61263ae948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.019268 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 727ad79a25364ca2ae9a5246ce8bf641 (DEFAULT_TABLE table=default.Table_1e71d261fb704c1296dfc437453d6f38 [id=0497d50325384d099124f631f89e536a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.019912 4162 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.020295 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8323c8e9cc114d2a99b346afe179a86b (DEFAULT_TABLE table=default.Table_f868e00d47d34ad995459fb265d9157a [id=4ebe1460d84e46e2aede53d51d79a6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.020578 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8488352995394b02b14172ea5eed8290"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.021195 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8488352995394b02b14172ea5eed8290 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.023422 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1f988d05c9984be98e5f1f83939bc620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.023663 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09913224068f4a71bab606dea187daee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.023838 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762db87ede7a490da344412e0ff2ff98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.024032 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8323c8e9cc114d2a99b346afe179a86b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.024093 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f988d05c9984be98e5f1f83939bc620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.024309 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8f24cdcf1c4973b388303fce0a5c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.025157 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f4112fcb7445b3803456e59a24ad0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.028751 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a70ecf0db7641419513558768d4c882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.030459 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa4822d4c1149c29187c9ac880766e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.033208 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727ad79a25364ca2ae9a5246ce8bf641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.034456 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15b0581b33b4796b527e87caf606eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.040326 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.041693 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15855cd92875485c96b2eb2082e88a0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.041736 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114d22f22d3d4dc09563aa1108e1f509 consensus state has no leader
I20260811 18:57:35.042788 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.043184 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4e1fc5686944dfa3eb57702c810eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.042940 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2 (DEFAULT_TABLE table=default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 [id=c321bcbff9b04df6ad72fbe9efc2d0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.043708 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.049816 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.117s user 0.002s sys 0.000s
I20260811 18:57:35.056667 3860 tablet_service.cc:1467] Processing CreateTablet for tablet c1239b67f2784a61b56fc0596121f2e0 (DEFAULT_TABLE table=default.Table_b0759818f6cc4cd8b2747ae532c19ccc [id=afc22fa155de4ba8b573ca5f20cafa0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.057287 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1239b67f2784a61b56fc0596121f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.065750 4162 tablet_bootstrap.cc:654] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.068358 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.068763 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.073802 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc" } modify_external_catalogs: true
I20260811 18:57:35.074100 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc" } modify_external_catalogs: true
I20260811 18:57:35.075853 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.076293 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b79839172cff41c3a8235675d44c21fa" } modify_external_catalogs: true
I20260811 18:57:35.076687 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b79839172cff41c3a8235675d44c21fa" } modify_external_catalogs: true
I20260811 18:57:35.078915 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_84be3c943435450c938c21da022a54f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.079490 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84be3c943435450c938c21da022a54f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.081743 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.082193 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.083135 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1342b8cd65b4c31a47b27a79c678d2b
I20260811 18:57:35.083319 4163 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.083451 4163 ts_tablet_manager.cc:1397] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.110s user 0.004s sys 0.000s
I20260811 18:57:35.083428 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a905052284004a77adf260546090e666" } new_table_name: "default.Table_43e653688f9047b2a322519e3f4de13d" modify_external_catalogs: true
I20260811 18:57:35.084229 4163 raft_consensus.cc:348] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.084438 4163 raft_consensus.cc:374] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.084497 4163 raft_consensus.cc:729] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.084703 4163 consensus_queue.cc:260] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.085459 4163 ts_tablet_manager.cc:1428] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:35.086706 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e5f47ccc193d48efa15d60a06b148e00
I20260811 18:57:35.089620 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.089924 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.090010 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.090225 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091485 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 8eaafc7677344cac8d9616996c5f416c (DEFAULT_TABLE table=default.Table_7e50906e16414e8b99c43763a538d6be [id=dab5f1b103eb489fb1bbb15659b424ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.091686 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1896700e724204a37439f134a4d988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091912 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091967 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet afa57ed239ac4afebe517cc48d879645 with delete_type TABLET_DATA_DELETED (replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092132 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dd203b7eb8643f493b4642c0396e258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092342 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092584 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092149 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 (DEFAULT_TABLE table=default.Table_3ee627ac9abf47889b5fed972cecd97e [id=17c62f2fe48546309ce0f54050e43299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.093165 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 38186aed20134de1bc06e0eb13806d34 (DEFAULT_TABLE table=default.Table_4464846117e74da2b263ff545f9937b3 [id=5c41b99d64b54cdbb7c2424ce5e9483b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.091908 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eaafc7677344cac8d9616996c5f416c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.095093 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38186aed20134de1bc06e0eb13806d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.099941 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 36861ae786c549d687e687fe853d1801 (DEFAULT_TABLE table=default.Table_aa50e46e061b41f79d5d6d048ea15132 [id=5ef75d1e27a8460e83162dd6e254ab9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100137 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 15855cd92875485c96b2eb2082e88a0b (DEFAULT_TABLE table=default.Table_6080085ba7594b18b5594e8868f0b80e [id=6e4857cbdcab4ce59110f567698f8f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100330 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36861ae786c549d687e687fe853d1801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100492 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15855cd92875485c96b2eb2082e88a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100416 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f15b0581b33b4796b527e87caf606eb2 (DEFAULT_TABLE table=default.Table_fae8f41d600f4fbfaed39f695cf6672a [id=a98ff2e2e2c14d4fa18822452154a31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100608 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fae3750e5870411e8fe7dedae2fe3ec0 (DEFAULT_TABLE table=default.Table_be8ef75ca590418486730a869708b637 [id=07e28abb2e494245999c05a44c9107fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100737 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15b0581b33b4796b527e87caf606eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100896 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae3750e5870411e8fe7dedae2fe3ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.101138 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 8323c8e9cc114d2a99b346afe179a86b (DEFAULT_TABLE table=default.Table_f868e00d47d34ad995459fb265d9157a [id=4ebe1460d84e46e2aede53d51d79a6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.101423 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8323c8e9cc114d2a99b346afe179a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.102553 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 727ad79a25364ca2ae9a5246ce8bf641 (DEFAULT_TABLE table=default.Table_1e71d261fb704c1296dfc437453d6f38 [id=0497d50325384d099124f631f89e536a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.102841 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727ad79a25364ca2ae9a5246ce8bf641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.106549 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccd56a8b7cc454ea3f2f08b59374521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.106663 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 6a70ecf0db7641419513558768d4c882 (DEFAULT_TABLE table=default.Table_58b2359b8659428190fe3cd7688c1d4d [id=df01d48865104be1ba7895f5bed2f809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.106791 3717 tablet_service.cc:1467] Processing CreateTablet for tablet ad8f24cdcf1c4973b388303fce0a5c08 (DEFAULT_TABLE table=default.Table_2033e26987554c839ae47bd350d82580 [id=4a215557a1474b22bffc1f4ba8a16025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.106997 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a70ecf0db7641419513558768d4c882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.107139 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8f24cdcf1c4973b388303fce0a5c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100394 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 09913224068f4a71bab606dea187daee (DEFAULT_TABLE table=default.Table_1afc7507935345b2a081a64ed01106da [id=9b590a32f284495ca2293a9e3b7e9e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.110694 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09913224068f4a71bab606dea187daee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.111121 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 61f4112fcb7445b3803456e59a24ad0c (DEFAULT_TABLE table=default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b [id=6ec509fc5d6449dc8493d9d72c95f279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.111433 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f4112fcb7445b3803456e59a24ad0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.115135 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa4822d4c1149c29187c9ac880766e3 (DEFAULT_TABLE table=default.Table_2fc380eb763b488e815a72be20c8f563 [id=611c2ca37d194e2a886789d9291ad3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.115440 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa4822d4c1149c29187c9ac880766e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.118867 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 762db87ede7a490da344412e0ff2ff98 (DEFAULT_TABLE table=default.Table_f414c31b8cff4e889368e4207afb1b53 [id=0822821331174d16b7f07c61263ae948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.119200 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762db87ede7a490da344412e0ff2ff98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.123011 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.123342 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4e1fc5686944dfa3eb57702c810eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.124886 3705 tablet_service.cc:1467] Processing CreateTablet for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2 (DEFAULT_TABLE table=default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 [id=c321bcbff9b04df6ad72fbe9efc2d0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.125892 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.126966 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.127372 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c1239b67f2784a61b56fc0596121f2e0 (DEFAULT_TABLE table=default.Table_b0759818f6cc4cd8b2747ae532c19ccc [id=afc22fa155de4ba8b573ca5f20cafa0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.127235 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.127710 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1239b67f2784a61b56fc0596121f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.128165 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_98051d104e9e4d9b976577088d6126a2" } new_table_name: "default.Table_59c9cdf2f7424fa7980a53799ff004d3" modify_external_catalogs: true
I20260811 18:57:35.128214 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_315c3cbf08354dae86745d91bbc2cd11" } new_table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" modify_external_catalogs: true
I20260811 18:57:35.130462 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.134187 4162 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.134356 4162 ts_tablet_manager.cc:1397] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.115s user 0.003s sys 0.003s
I20260811 18:57:35.135110 4162 raft_consensus.cc:348] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.135330 4162 raft_consensus.cc:374] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.135375 4162 raft_consensus.cc:729] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.135672 4162 consensus_queue.cc:260] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.140739 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a902ad70a25546be926ff8f7bf2ed64f (table default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]) successfully deleted
W20260811 18:57:35.141081 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cdb92902c2584751af77d3117f52cb4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdb92902c2584751af77d3117f52cb4e already in progress: creating tablet
W20260811 18:57:35.141352 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ccc63b46a4e346f79d1e03a82490794c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccc63b46a4e346f79d1e03a82490794c already in progress: creating tablet
W20260811 18:57:35.141815 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 14799a6f61a946008acee8a09d050c1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14799a6f61a946008acee8a09d050c1e already in progress: creating tablet
W20260811 18:57:35.141984 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99de8ce45e514ff291cb9e5c0a4c00d1 already in progress: creating tablet
W20260811 18:57:35.142163 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 55f97f37fa7c483cacaeed54f93e7a24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55f97f37fa7c483cacaeed54f93e7a24 already in progress: creating tablet
W20260811 18:57:35.142400 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e1ad3860d03f46baadd5ede854f1a909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1ad3860d03f46baadd5ede854f1a909 already in progress: creating tablet
I20260811 18:57:35.143620 4162 ts_tablet_manager.cc:1428] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260811 18:57:35.147202 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fcb1c9787d3f4fd181f86deca28ad160
I20260811 18:57:35.149772 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 122b7898fea44ef78f269b1505d8ddc4 (table default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b [id=1bcc678c540844b38d761cec1618c852]) successfully deleted
W20260811 18:57:35.150051 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 844bcd16f06c4788af9c0d7d320464aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 844bcd16f06c4788af9c0d7d320464aa already in progress: creating tablet
W20260811 18:57:35.150377 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0a5b06fa572045f3a5fc5a0b123d77cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5b06fa572045f3a5fc5a0b123d77cc already in progress: creating tablet
W20260811 18:57:35.150609 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e7d75d8468fc4c938fde0b2951d84562 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7d75d8468fc4c938fde0b2951d84562 already in progress: creating tablet
W20260811 18:57:35.151011 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f24c8db84f04484584ad1ba5d97f96d4 consensus state has no leader
W20260811 18:57:35.151983 3824 consensus_peers.cc:487] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.152113 3824 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.152338 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.156924 5417 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.157085 5417 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.157825 5417 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.146665 4162 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:35.161530 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5bbd3274980470b9fe71457b49bdd62 consensus state has no leader
I20260811 18:57:35.161914 4162 tablet_bootstrap.cc:654] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.162545 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c96a264e017471ea114ca0c6a9500ef consensus state has no leader
W20260811 18:57:35.162948 3824 leader_election.cc:334] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.153110 4167 ts_tablet_manager.cc:1918] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.163221 4167 log.cc:1196] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/45fde25bcb40493aafb083e161aa6932
I20260811 18:57:35.163846 4167 ts_tablet_manager.cc:1939] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.165692 4167 tablet_replica.cc:330] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.166103 4167 raft_consensus.cc:2229] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.166445 4167 raft_consensus.cc:2258] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.167590 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:35.167824 3740 raft_consensus.cc:2454] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:35.168919 5417 raft_consensus.cc:482] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.169072 5417 raft_consensus.cc:504] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.169631 5417 leader_election.cc:288] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:35.170262 3824 leader_election.cc:341] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:35.170476 3824 leader_election.cc:302] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.170749 5417 raft_consensus.cc:2726] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.174428 4169 ts_tablet_manager.cc:1918] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.174638 4169 log.cc:1196] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:35.175108 4169 ts_tablet_manager.cc:1939] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.175504 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 913) took 2199 ms. Trace:
I20260811 18:57:35.175644 4169 rpcz_store.cc:276] 0811 18:57:32.975983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.976595 (+ 612us) service_pool.cc:227] Handling call
0811 18:57:34.938626 (+1962031us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:34.938640 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.040944 (+102304us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.146236 (+105292us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.175488 (+ 29252us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":144,"tablet-delete.queue_time_us":1961695}
I20260811 18:57:35.176162 4169 tablet_replica.cc:330] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.176771 4169 raft_consensus.cc:2229] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.176875 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8e24f37301a145fe9b5029eaa654e2af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.177280 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (table default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]) successfully deleted
W20260811 18:57:35.177561 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e24f37301a145fe9b5029eaa654e2af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:35.177537 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ccc63b46a4e346f79d1e03a82490794c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccc63b46a4e346f79d1e03a82490794c already in progress: creating tablet
W20260811 18:57:35.180197 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cdb92902c2584751af77d3117f52cb4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdb92902c2584751af77d3117f52cb4e already in progress: creating tablet
W20260811 18:57:35.181669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99de8ce45e514ff291cb9e5c0a4c00d1 already in progress: creating tablet
I20260811 18:57:35.180629 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d7c25599688441595b4b8f6cfea4584" } new_table_name: "default.Table_41871fb0db9c472689c3dd40888d4c77" modify_external_catalogs: true
W20260811 18:57:35.182677 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 14799a6f61a946008acee8a09d050c1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14799a6f61a946008acee8a09d050c1e already in progress: creating tablet
I20260811 18:57:35.182906 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 55f97f37fa7c483cacaeed54f93e7a24 (table default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]) successfully deleted
I20260811 18:57:35.183094 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 58c515c153df4317b3fb427983ee15e0 (table default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]) successfully deleted
W20260811 18:57:35.183334 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e1ad3860d03f46baadd5ede854f1a909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1ad3860d03f46baadd5ede854f1a909 already in progress: creating tablet
I20260811 18:57:35.183568 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6432abd4fc5c4f9ca95971ca4795e1cf (table default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
W20260811 18:57:35.183779 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0a5b06fa572045f3a5fc5a0b123d77cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5b06fa572045f3a5fc5a0b123d77cc already in progress: creating tablet
W20260811 18:57:35.183992 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 844bcd16f06c4788af9c0d7d320464aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 844bcd16f06c4788af9c0d7d320464aa already in progress: creating tablet
W20260811 18:57:35.184194 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e7d75d8468fc4c938fde0b2951d84562 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7d75d8468fc4c938fde0b2951d84562 already in progress: creating tablet
I20260811 18:57:35.184389 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7251204325dd424192a6f58e18e71c34 (table default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]) successfully deleted
W20260811 18:57:35.184585 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6df5545532e346f7abcdc4cd228b84da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df5545532e346f7abcdc4cd228b84da already in progress: creating tablet
I20260811 18:57:35.184868 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:35.185089 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4af28c051e8243eaa73d4a5ed1abcea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af28c051e8243eaa73d4a5ed1abcea6 already in progress: creating tablet
W20260811 18:57:35.185375 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9608d6ddcad749cd9e72dd6f265db0da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9608d6ddcad749cd9e72dd6f265db0da already in progress: creating tablet
W20260811 18:57:35.185676 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 05cba6b392b642e59321570aac3541d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05cba6b392b642e59321570aac3541d0 already in progress: creating tablet
W20260811 18:57:35.185951 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 101994cc708a4fadb46fc8c5915a5587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 101994cc708a4fadb46fc8c5915a5587 already in progress: creating tablet
W20260811 18:57:35.186151 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 already in progress: creating tablet
W20260811 18:57:35.186393 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7216c095cc03424bbbe97a52a9600ebb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7216c095cc03424bbbe97a52a9600ebb already in progress: creating tablet
I20260811 18:57:35.187420 4163 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.190160 5510 tablet.cc:1722] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260811 18:57:35.194725 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a25a758955e43598be8e8d9c99b750e has no consensus state
I20260811 18:57:35.194726 4169 raft_consensus.cc:2258] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.195731 4169 ts_tablet_manager.cc:1905] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.196126 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1fed68aa37dc45c4bd9492de217dd73c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.196789 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fed68aa37dc45c4bd9492de217dd73c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.198694 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d6113c39441d47a8a1d07df0fbd2371e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.198997 3824 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:35.199296 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6113c39441d47a8a1d07df0fbd2371e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.199301 5417 raft_consensus.cc:2781] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.199432 5417 raft_consensus.cc:686] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.199714 5417 consensus_queue.cc:237] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.201915 4163 tablet_bootstrap.cc:654] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.206830 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_44d5558ecc5b4b7cba0c0a120270f3eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.207436 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44d5558ecc5b4b7cba0c0a120270f3eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.211647 4167 ts_tablet_manager.cc:1905] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.213649 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b7a67e56b3f641139862409e95439c6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.214287 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7a67e56b3f641139862409e95439c6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.214728 3824 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.215121 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:35.215086 3824 consensus_peers.cc:487] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f3060822021431584a494e6941be756. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.221007 5376 raft_consensus.cc:2781] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.222381 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:35.227742 3824 consensus_peers.cc:487] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2718cd4a98742dfa8014a8a38c69c45. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.227954 3824 consensus_peers.cc:487] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3263da75f0d84528b7c8cc55dcaeaa46. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.230018 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 43689.9us
I20260811 18:57:35.230270 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 968fa8b8912b40a8bdba77db1d39c5c0 (table default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]) successfully deleted
I20260811 18:57:35.230492 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 68a7051bc539468aafba30fbaf0669d4 (table default.Table_821a790ed161401bb9758d71a0760916 [id=4a5d04ba5f3942a3942909ebd2bf5e20]) successfully deleted
I20260811 18:57:35.231968 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 45fde25bcb40493aafb083e161aa6932 (table default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c [id=4bc5d2d6894e40ebab164e5ffd6ac600]) successfully deleted
W20260811 18:57:35.239728 3824 leader_election.cc:341] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0ac9bdc606be40558dfa33e1e7a8ebc7
W20260811 18:57:35.240202 3824 consensus_peers.cc:487] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 221d0d8d69f142a29ec7065a82b3f769. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.240453 3824 leader_election.cc:341] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: f7d4c26da86b49b8a37f2289f4ef6614
W20260811 18:57:35.240697 3824 leader_election.cc:341] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:35.240933 3824 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.241362 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.241978 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f47ccc193d48efa15d60a06b148e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.242216 3824 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.243093 5511 raft_consensus.cc:2781] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:35.242991 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_45ec6e6d5001405faabf015cd3bd77e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.243245 5511 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:35.243466 5511 raft_consensus.cc:3037] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:35.243722 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45ec6e6d5001405faabf015cd3bd77e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.245424 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.224295 4162 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.246114 4162 ts_tablet_manager.cc:1397] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.100s user 0.002s sys 0.002s
I20260811 18:57:35.247558 4162 raft_consensus.cc:348] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.248041 4162 raft_consensus.cc:374] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.248170 4162 raft_consensus.cc:729] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.248528 4162 consensus_queue.cc:260] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.249461 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e71d261fb704c1296dfc437453d6f38" } new_table_name: "default.Table_adb037985442440c80c98862194343d9" modify_external_catalogs: true
I20260811 18:57:35.250748 4163 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.250875 4163 ts_tablet_manager.cc:1397] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.000s
I20260811 18:57:35.251652 5490 raft_consensus.cc:482] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.251782 5490 raft_consensus.cc:504] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.252360 5490 leader_election.cc:288] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:35.254638 3678 leader_election.cc:334] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.256886 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.260021 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 727ad79a25364ca2ae9a5246ce8bf641 consensus state has no leader
I20260811 18:57:35.249507 4162 ts_tablet_manager.cc:1428] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:35.252573 4163 raft_consensus.cc:348] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.256026 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cce97bfdb1d41e685e091c8c5cc6946" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.267155 3676 leader_election.cc:341] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.267709 3676 leader_election.cc:302] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.268085 5490 raft_consensus.cc:2726] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.262710 5376 raft_consensus.cc:686] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.268929 5376 consensus_queue.cc:237] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.267604 4163 raft_consensus.cc:374] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.269557 4163 raft_consensus.cc:729] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.271557 4163 consensus_queue.cc:260] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.272912 4163 ts_tablet_manager.cc:1428] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.002s sys 0.001s
I20260811 18:57:35.273604 4163 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.277872 4163 tablet_bootstrap.cc:654] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.279557 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 302cf69ea7c14fdaac72f6a9a9aff113
I20260811 18:57:35.280014 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 302cf69ea7c14fdaac72f6a9a9aff113 deleted and replaced by tablet 9708871030cc4ea6b620fb647403c42a
I20260811 18:57:35.280689 5490 raft_consensus.cc:482] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.280850 5490 raft_consensus.cc:504] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.281555 5490 leader_election.cc:288] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.283948 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847581412494e34b2c76e605ff11772" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.284672 3676 leader_election.cc:341] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.285956 3678 leader_election.cc:334] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.286065 3678 leader_election.cc:302] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.286758 5490 raft_consensus.cc:2726] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.287014 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_21556170a88148319a2e03775ac1e02f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.287694 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21556170a88148319a2e03775ac1e02f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.290567 4162 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.292500 4162 tablet_bootstrap.cc:654] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.292585 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 with delete_type TABLET_DATA_DELETED (replaced by tablet 9708871030cc4ea6b620fb647403c42a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.293351 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c" } modify_external_catalogs: true
I20260811 18:57:35.293570 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c" } modify_external_catalogs: true
I20260811 18:57:35.296373 4163 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.296679 4163 ts_tablet_manager.cc:1397] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.006s
I20260811 18:57:35.297240 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e50906e16414e8b99c43763a538d6be" } modify_external_catalogs: true
I20260811 18:57:35.297459 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e50906e16414e8b99c43763a538d6be" } modify_external_catalogs: true
I20260811 18:57:35.298369 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_36c481dceea34aacbb4ad6aabe5e403c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.298453 4163 raft_consensus.cc:348] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:35.298987 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36c481dceea34aacbb4ad6aabe5e403c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.299050 4163 raft_consensus.cc:374] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.299309 4163 raft_consensus.cc:729] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.299801 4163 consensus_queue.cc:260] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.301036 4163 ts_tablet_manager.cc:1428] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:35.301528 4163 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.301563 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef967930b08e4f7492ffa9993b43f445"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.302179 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef967930b08e4f7492ffa9993b43f445 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.304737 4163 tablet_bootstrap.cc:654] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.305907 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 36861ae786c549d687e687fe853d1801
I20260811 18:57:35.306491 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 46505ba544104458888dbc2a37115dcc
I20260811 18:57:35.306875 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 46505ba544104458888dbc2a37115dcc deleted and replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e
I20260811 18:57:35.307381 4167 ts_tablet_manager.cc:1918] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.307596 4167 log.cc:1196] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:35.308079 4167 ts_tablet_manager.cc:1939] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.311328 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 979) took 1080 ms. Trace:
W20260811 18:57:35.322548 3824 consensus_peers.cc:487] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.322793 3824 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.323185 3824 leader_election.cc:334] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.323339 3824 leader_election.cc:302] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:35.323664 3824 consensus_peers.cc:487] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.326653 4163 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.327955 4163 ts_tablet_manager.cc:1397] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.003s
I20260811 18:57:35.328087 4169 ts_tablet_manager.cc:1918] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.329530 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c26851245cab4a2d80e94d4fe278fd50 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
I20260811 18:57:35.331485 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 18c599d945044e0088bc66c8472d2495
W20260811 18:57:35.340058 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2f31331974f42c3a22b3ac21b690f82: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.322836 5407 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.341651 4167 rpcz_store.cc:276] 0811 18:57:34.231069 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.231120 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:35.165587 (+934467us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:35.165627 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.302575 (+136948us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.307242 (+ 4667us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.311297 (+ 4055us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":934263}
I20260811 18:57:35.336782 4162 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.342193 4162 ts_tablet_manager.cc:1397] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.001s
I20260811 18:57:35.343137 4162 raft_consensus.cc:348] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.343372 4162 raft_consensus.cc:374] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.343426 4162 raft_consensus.cc:729] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.343600 4162 consensus_queue.cc:260] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.344108 5407 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.344640 5407 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.346472 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 36861ae786c549d687e687fe853d1801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.346514 4169 log.cc:1196] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:35.346710 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 46505ba544104458888dbc2a37115dcc with delete_type TABLET_DATA_DELETED (replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.347054 4169 ts_tablet_manager.cc:1939] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.347469 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 912) took 2376 ms. Trace:
I20260811 18:57:35.347586 4169 rpcz_store.cc:276] 0811 18:57:32.971263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.979785 (+ 8522us) service_pool.cc:227] Handling call
0811 18:57:35.176115 (+2196330us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:35.176127 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.270940 (+ 94813us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.327958 (+ 57018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.347452 (+ 19494us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17477,"tablet-delete.queue_time_us":2192760}
I20260811 18:57:35.343598 5417 raft_consensus.cc:2726] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.343753 5376 raft_consensus.cc:482] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.344528 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345019 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345065 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 with delete_type TABLET_DATA_DELETED (replaced by tablet 9708871030cc4ea6b620fb647403c42a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345739 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 18c599d945044e0088bc66c8472d2495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.341192 5526 tablet.cc:1722] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.345741 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:35.346086 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.346220 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f47ccc193d48efa15d60a06b148e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.348363 5376 raft_consensus.cc:504] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.348801 4162 ts_tablet_manager.cc:1428] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20260811 18:57:35.348861 3676 leader_election.cc:341] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.348970 5376 leader_election.cc:288] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.349086 4162 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:35.349306 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.051s user 0.004s sys 0.000s
W20260811 18:57:35.351804 3678 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.351926 3678 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.352353 5407 raft_consensus.cc:2726] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.353606 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 978) took 1122 ms. Trace:
I20260811 18:57:35.354599 4169 tablet_replica.cc:330] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.354992 4169 raft_consensus.cc:2229] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.355511 4169 pending_rounds.cc:62] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:35.355651 4169 pending_rounds.cc:68] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400176040996864 op_type: NO_OP noop_request { }
I20260811 18:57:35.355888 4169 raft_consensus.cc:2866] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:35.355851 4163 raft_consensus.cc:348] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.246291 5511 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.359305 5511 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.360001 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.360100 4163 raft_consensus.cc:374] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.360294 4163 raft_consensus.cc:729] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.360847 4167 rpcz_store.cc:276] 0811 18:57:34.230925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.231435 (+ 510us) service_pool.cc:227] Handling call
0811 18:57:35.353484 (+1122049us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:35.353498 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.353585 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1121794}
I20260811 18:57:35.360741 4163 consensus_queue.cc:260] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.361435 3546 catalog_manager.cc:5495] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.361928 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
I20260811 18:57:35.356094 4169 raft_consensus.cc:2258] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.363319 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 974) took 1169 ms. Trace:
I20260811 18:57:35.363426 4167 rpcz_store.cc:276] 0811 18:57:34.193699 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.227298 (+ 33599us) service_pool.cc:227] Handling call
0811 18:57:35.363151 (+1135853us) ts_tablet_manager.cc:1145] Deleting tablet 7be6d9578e7847c2822278486119390c
0811 18:57:35.363169 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.363293 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1118132}
I20260811 18:57:35.364429 4162 tablet_bootstrap.cc:654] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.346310 3824 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.366194 4167 tablet_replica.cc:330] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.367228 4167 raft_consensus.cc:2229] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.369361 4167 raft_consensus.cc:2258] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:35.366981 3824 leader_election.cc:334] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.369601 3824 leader_election.cc:302] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.370513 5511 raft_consensus.cc:2726] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.371346 3824 leader_election.cc:334] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.372121 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" } modify_external_catalogs: true
I20260811 18:57:35.372334 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" } modify_external_catalogs: true
I20260811 18:57:35.373944 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_df08bd98f2ef430eaf0e10876b9e265e" } new_table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" modify_external_catalogs: true
I20260811 18:57:35.371824 3824 leader_election.cc:302] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.374747 4167 ts_tablet_manager.cc:1905] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.375571 5511 raft_consensus.cc:2726] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.375427 4163 ts_tablet_manager.cc:1428] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260811 18:57:35.378203 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc1e05602be443528f758fba1ac0195f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.378847 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc1e05602be443528f758fba1ac0195f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.380167 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e863da40303e45bda8d9464f8ec9b8a0" } new_table_name: "default.Table_0184bf33514f4ff7bdb551628147f29d" modify_external_catalogs: true
I20260811 18:57:35.381621 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2635923a5784be1839e861c2277e900" } new_table_name: "default.Table_2afc6c2e3a254c57840c4e891d796bf5" modify_external_catalogs: true
I20260811 18:57:35.382778 4163 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.384897 4169 ts_tablet_manager.cc:1905] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:35.385588 3824 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.389057 3824 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:35.390179 3824 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.391324 5407 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.391481 5407 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.392156 5407 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.393035 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0f6df7a291456d80538e2b76417873" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.393648 3676 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.394526 3678 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.394649 3678 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.394883 5407 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.395177 5511 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.398118 3824 leader_election.cc:341] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:35.404587 4163 tablet_bootstrap.cc:654] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.407037 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 36861ae786c549d687e687fe853d1801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.407696 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9bc5448a555944dfba8ccae09e25cbe9 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:35.409843 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8eaafc7677344cac8d9616996c5f416c
I20260811 18:57:35.410558 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91eff2208ee74ec7b44c4908dcdbe335"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.410920 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 46505ba544104458888dbc2a37115dcc with delete_type TABLET_DATA_DELETED (replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.411008 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 18c599d945044e0088bc66c8472d2495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.411247 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91eff2208ee74ec7b44c4908dcdbe335 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.416239 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da (DEFAULT_TABLE table=default.Table_28350d4a6baf4232bef35569f82e0d4c [id=27d6b1acb551467eb6ca07fff162c310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.416697 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a15e7a0c0f4283ac5e575d3c15c9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.417049 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:35.417268 3745 raft_consensus.cc:3037] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.419896 3745 raft_consensus.cc:2454] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:35.420984 4167 ts_tablet_manager.cc:1918] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.422657 4167 log.cc:1196] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:35.423031 4167 ts_tablet_manager.cc:1939] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.423475 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 982) took 1153 ms. Trace:
I20260811 18:57:35.423576 4167 rpcz_store.cc:276] 0811 18:57:34.270025 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.270376 (+ 351us) service_pool.cc:227] Handling call
0811 18:57:35.366139 (+1095763us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:35.366152 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.384566 (+ 18414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.420871 (+ 36305us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.423454 (+ 2583us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1095585}
I20260811 18:57:35.424340 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.425364 4167 tablet_replica.cc:330] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.425571 4167 raft_consensus.cc:2229] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.425741 4167 raft_consensus.cc:2258] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.426295 4167 ts_tablet_manager.cc:1905] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.428783 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 4e83123680c74cd187604eb237d588d1 (DEFAULT_TABLE table=default.Table_33ef706171fc47669bf54e9a71d90905 [id=dc8bc36ec82541a787edd3efac919baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.429503 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e83123680c74cd187604eb237d588d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.432983 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b0a6f7cded4c4fffbc45114fdb116623 (DEFAULT_TABLE table=default.Table_97e3f5df19fc4b04a37f62286b42f7e9 [id=e72c856e06ea481ba854aad4d2740689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.433383 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a6f7cded4c4fffbc45114fdb116623. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.440640 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa5a3d1ada554d778d507382c8836133 consensus state has no leader
I20260811 18:57:35.441079 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_99de6d48691743d798f56442d85d5be8" } new_table_name: "default.Table_b3c46b3c07b54286b190d12bff8f1965" modify_external_catalogs: true
I20260811 18:57:35.441633 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1cb848dc1c1438aa868e39197e05009"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.442255 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1cb848dc1c1438aa868e39197e05009 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.442454 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eaafc7677344cac8d9616996c5f416c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.442783 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e2fb15b72c1d49d2ac4961bc04655a29
I20260811 18:57:35.443475 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1f8487f0094407ba50ccdaecfebe7ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.443751 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91d19b48908740128914fa5c231dfc04 consensus state has no leader
W20260811 18:57:35.444087 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f8487f0094407ba50ccdaecfebe7ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.444240 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9df97a2d74d040ef8f3dd56e41b8c28b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.444792 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9df97a2d74d040ef8f3dd56e41b8c28b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.444929 3867 tablet_service.cc:1467] Processing CreateTablet for tablet fc8df2ed36764928ab40376ebf0838a7 (DEFAULT_TABLE table=default.Table_53f67a53b6684eaeb56341d3d428f565 [id=8fb2161513954b7880919ffbf7505a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.445418 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc8df2ed36764928ab40376ebf0838a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.446616 3553 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_667271a3af6a4ab489e6ef9a2a2769a0 [id=e98d6386b7fe47a9afba14f90faab197] alter complete (version 1)
I20260811 18:57:35.426420 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da (DEFAULT_TABLE table=default.Table_28350d4a6baf4232bef35569f82e0d4c [id=27d6b1acb551467eb6ca07fff162c310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.447350 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
I20260811 18:57:35.447635 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a15e7a0c0f4283ac5e575d3c15c9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.447750 3551 catalog_manager.cc:6565] ReplaceTablet: tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae deleted and replaced by tablet dfb6ac49f1d4403dad38a3125264088a
W20260811 18:57:35.448393 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f2f5ce2b1d14a9a988cff644e196088 consensus state has no leader
I20260811 18:57:35.448787 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:35.427325 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 4e83123680c74cd187604eb237d588d1 (DEFAULT_TABLE table=default.Table_33ef706171fc47669bf54e9a71d90905 [id=dc8bc36ec82541a787edd3efac919baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.450677 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e83123680c74cd187604eb237d588d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.453658 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7811427611da4cd1a27acc552c72c374"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.454267 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7811427611da4cd1a27acc552c72c374 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.454852 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b9e645f8bb1848ce8f2cd0feb6ea191c (DEFAULT_TABLE table=default.Table_7009400051004543ade835c61177ac98 [id=9c96db8126534db6aaeea68052a454cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.455315 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e645f8bb1848ce8f2cd0feb6ea191c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.456996 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.434461 4162 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.461331 4162 ts_tablet_manager.cc:1397] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.112s user 0.007s sys 0.000s
I20260811 18:57:35.462503 4162 raft_consensus.cc:348] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.462878 4162 raft_consensus.cc:374] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.462976 4162 raft_consensus.cc:729] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.463223 4162 consensus_queue.cc:260] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.463841 4162 ts_tablet_manager.cc:1428] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:35.471657 4162 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.471750 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.472406 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae with delete_type TABLET_DATA_DELETED (replaced by tablet dfb6ac49f1d4403dad38a3125264088a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.472965 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.474066 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (table default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]) successfully deleted
W20260811 18:57:35.474565 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.056s user 0.001s sys 0.003s
I20260811 18:57:35.474701 4162 tablet_bootstrap.cc:654] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.476770 3824 leader_election.cc:334] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.477267 3824 leader_election.cc:302] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.477689 5511 raft_consensus.cc:2726] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.478183 3824 consensus_peers.cc:487] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.481630 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 52727.1us
I20260811 18:57:35.486878 5407 raft_consensus.cc:482] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.487044 5407 raft_consensus.cc:504] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.487737 5407 leader_election.cc:288] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.488595 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b404e99e9414c67b3cd5aff12203a53" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.489185 3676 leader_election.cc:341] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.490872 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because the tablet was not found. No further retry: Not found: Tablet not found: fe203387bc9c45ba82afe6b058aa5e5f
W20260811 18:57:35.491185 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7be6d9578e7847c2822278486119390c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7be6d9578e7847c2822278486119390c already in progress: creating tablet
I20260811 18:57:35.491462 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 48d32bb4eca047e4ba786c04b8bebf8b (table default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]) successfully deleted
W20260811 18:57:35.492498 3678 leader_election.cc:334] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.492610 3678 leader_election.cc:302] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.493076 5407 raft_consensus.cc:2726] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.493319 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eaafc7677344cac8d9616996c5f416c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.495417 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ec98c10d285144029e63c99243d4d9fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.496577 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec98c10d285144029e63c99243d4d9fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.498090 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba1e7c81d1bf4209b7089639928742f1" } new_table_name: "default.Table_14587bf2389240a1b10454cad65b748a" modify_external_catalogs: true
I20260811 18:57:35.500945 4163 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.501312 4163 ts_tablet_manager.cc:1397] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.002s sys 0.004s
I20260811 18:57:35.501982 3551 catalog_manager.cc:5495] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.503438 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:35.503822 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.504069 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:35.505438 4163 raft_consensus.cc:348] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.505816 4163 raft_consensus.cc:374] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.505955 4163 raft_consensus.cc:729] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.506283 4163 consensus_queue.cc:260] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.508292 4162 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.508486 4162 ts_tablet_manager.cc:1397] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.000s
I20260811 18:57:35.510278 4163 ts_tablet_manager.cc:1428] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:35.510697 4163 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:35.516179 3824 leader_election.cc:341] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 45fde25bcb40493aafb083e161aa6932
I20260811 18:57:35.517300 3824 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.518620 5511 raft_consensus.cc:2781] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.518795 5511 raft_consensus.cc:686] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.519061 5511 consensus_queue.cc:237] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.521121 5510 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:35.523518 4162 raft_consensus.cc:348] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.526859 4162 raft_consensus.cc:374] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.526950 4162 raft_consensus.cc:729] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.527180 4162 consensus_queue.cc:260] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.527869 4162 ts_tablet_manager.cc:1428] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
W20260811 18:57:35.530963 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45d6d76629d440129e4509fad38dde43 consensus state has no leader
I20260811 18:57:35.537204 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28350d4a6baf4232bef35569f82e0d4c" } modify_external_catalogs: true
I20260811 18:57:35.537510 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28350d4a6baf4232bef35569f82e0d4c" } modify_external_catalogs: true
I20260811 18:57:35.537958 4169 ts_tablet_manager.cc:1918] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.538630 4169 log.cc:1196] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0b3ec38f9b55489ba00ed76f9e4456f5
W20260811 18:57:35.538317 3824 leader_election.cc:334] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.539425 3824 leader_election.cc:302] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.541270 4169 ts_tablet_manager.cc:1939] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.542454 5376 raft_consensus.cc:2726] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.541939 4163 tablet_bootstrap.cc:654] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.543885 3824 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.545624 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fc8df2ed36764928ab40376ebf0838a7 (DEFAULT_TABLE table=default.Table_53f67a53b6684eaeb56341d3d428f565 [id=8fb2161513954b7880919ffbf7505a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.545715 3705 tablet_service.cc:1467] Processing CreateTablet for tablet b0a6f7cded4c4fffbc45114fdb116623 (DEFAULT_TABLE table=default.Table_97e3f5df19fc4b04a37f62286b42f7e9 [id=e72c856e06ea481ba854aad4d2740689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.548939 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.550478 3853 tablet_service.cc:1467] Processing CreateTablet for tablet fa6509e905694f979191469d20e1de41 (DEFAULT_TABLE table=default.Table_c031c1fe18014450aa4cffbc9cd0f573 [id=b9b4076a00c247e38bbe4223c548d8d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.550590 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5533c71460874591bb7380ba652e299d (DEFAULT_TABLE table=default.Table_a5361d65b50b445f9d85f5ddda046f99 [id=a5a8b9a494464ac280c90a3cbf8b4839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.550927 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6509e905694f979191469d20e1de41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.551810 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5533c71460874591bb7380ba652e299d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.551821 4162 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.558940 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc8df2ed36764928ab40376ebf0838a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.560608 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae with delete_type TABLET_DATA_DELETED (replaced by tablet dfb6ac49f1d4403dad38a3125264088a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.560901 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a6f7cded4c4fffbc45114fdb116623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.563098 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a62cdd333ad8494897ae8ec5afd5af74 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.563184 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.564270 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62cdd333ad8494897ae8ec5afd5af74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.566581 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b9e645f8bb1848ce8f2cd0feb6ea191c (DEFAULT_TABLE table=default.Table_7009400051004543ade835c61177ac98 [id=9c96db8126534db6aaeea68052a454cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.567003 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e645f8bb1848ce8f2cd0feb6ea191c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.574127 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 827de91945274026a2c8fd17224876d4 (DEFAULT_TABLE table=default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a [id=982c198aba3c4e94bfd3421d3478bb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.574754 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827de91945274026a2c8fd17224876d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.575685 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7407d7434a3b4290889506a933ae2f05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.576474 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7407d7434a3b4290889506a933ae2f05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.577160 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" } modify_external_catalogs: true
I20260811 18:57:35.579293 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 6a25a758955e43598be8e8d9c99b750e (DEFAULT_TABLE table=default.Table_41871fb0db9c472689c3dd40888d4c77 [id=af21ca7213cb485dbef755cf5fa54fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.579770 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a25a758955e43598be8e8d9c99b750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.580650 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d8bd22de86c94d138ea871d6e957b877 (DEFAULT_TABLE table=default.Table_aa3debf8d8774103a45a754a6f08465a [id=e3f3e088f9c04932895af3dd8eae54f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.580993 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bd22de86c94d138ea871d6e957b877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.583345 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed141f91ec548db87ac1e719256931b (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.583380 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6aacb785fd444621820be42007d55668"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.583693 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed141f91ec548db87ac1e719256931b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.584088 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6aacb785fd444621820be42007d55668 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.585711 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 09f786a1b5064367ae6d89fc1262a0b5 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.586104 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f786a1b5064367ae6d89fc1262a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.588135 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8752a36a67ad4fb380132326a813d172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.588769 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8752a36a67ad4fb380132326a813d172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.600639 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5533c71460874591bb7380ba652e299d (DEFAULT_TABLE table=default.Table_a5361d65b50b445f9d85f5ddda046f99 [id=a5a8b9a494464ac280c90a3cbf8b4839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.601222 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5533c71460874591bb7380ba652e299d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.603819 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.606586 3715 tablet_service.cc:1467] Processing CreateTablet for tablet fa6509e905694f979191469d20e1de41 (DEFAULT_TABLE table=default.Table_c031c1fe18014450aa4cffbc9cd0f573 [id=b9b4076a00c247e38bbe4223c548d8d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.607265 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6509e905694f979191469d20e1de41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.619416 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fe20c54096804a88ba68b22d0cbea93c (DEFAULT_TABLE table=default.Table_6b28d478364f4057a8c2619e5d90b850 [id=a7df2477039d47e3aa9a641f0c30e4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.620440 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe20c54096804a88ba68b22d0cbea93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.626652 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 (DEFAULT_TABLE table=default.Table_485cdcb897e6430ba77e994afbd44a64 [id=2a7d0a1f579a41639b10989738a9c8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.625985 4162 tablet_bootstrap.cc:654] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.627292 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1700ee40ac0d40198c5b8a161a90b6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.628903 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" } modify_external_catalogs: true
I20260811 18:57:35.630381 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 46a15e7a0c0f4283ac5e575d3c15c9da
I20260811 18:57:35.630736 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet c0f68d33e1b14144b34c17e3c7201408 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.631135 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2" } modify_external_catalogs: true
W20260811 18:57:35.631824 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114397bea09444479df113552c22e154 consensus state has no leader
I20260811 18:57:35.632316 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2" } modify_external_catalogs: true
I20260811 18:57:35.633280 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ec27862124b74126a62f8588ec1237fa (DEFAULT_TABLE table=default.Table_acd269829eb743178d173c6cc3d7301e [id=9602f52d737341158fe431549212c038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.633420 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 (DEFAULT_TABLE table=default.Table_58f2aeab56c6446184cfeddcb1134314 [id=e4ed6236205a46d78af24a1e46371aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.633711 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec27862124b74126a62f8588ec1237fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.633934 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e71af5ee8b463eb158ac9ae76ff252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.637728 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f4848d8f28744cc49234e7ccfd8d158c (DEFAULT_TABLE table=default.Table_489a043a9e89488ca2fbcd69855c4e5d [id=05d2ef429d684b01a2e8e8d31eeeccf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.637938 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 13af4d419ae542cc961c75f28efd0204 (DEFAULT_TABLE table=default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.638176 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4848d8f28744cc49234e7ccfd8d158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.638358 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13af4d419ae542cc961c75f28efd0204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.638875 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.641970 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 6a25a758955e43598be8e8d9c99b750e (DEFAULT_TABLE table=default.Table_41871fb0db9c472689c3dd40888d4c77 [id=af21ca7213cb485dbef755cf5fa54fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.642446 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a25a758955e43598be8e8d9c99b750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.637245 3718 tablet_service.cc:1467] Processing CreateTablet for tablet a62cdd333ad8494897ae8ec5afd5af74 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.638971 5526 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:35.643602 3705 tablet_service.cc:1467] Processing CreateTablet for tablet d8bd22de86c94d138ea871d6e957b877 (DEFAULT_TABLE table=default.Table_aa3debf8d8774103a45a754a6f08465a [id=e3f3e088f9c04932895af3dd8eae54f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.643966 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bd22de86c94d138ea871d6e957b877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.639706 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 827de91945274026a2c8fd17224876d4 (DEFAULT_TABLE table=default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a [id=982c198aba3c4e94bfd3421d3478bb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.639916 5407 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.647384 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827de91945274026a2c8fd17224876d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.647238 5407 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.648042 5407 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:35.653178 3678 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.653528 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62cdd333ad8494897ae8ec5afd5af74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.658506 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 09f786a1b5064367ae6d89fc1262a0b5 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.658900 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.660475 3676 leader_election.cc:341] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.660586 3676 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.661114 5407 raft_consensus.cc:2726] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.661612 3713 tablet_service.cc:1467] Processing CreateTablet for tablet fe20c54096804a88ba68b22d0cbea93c (DEFAULT_TABLE table=default.Table_6b28d478364f4057a8c2619e5d90b850 [id=a7df2477039d47e3aa9a641f0c30e4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.661680 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.661906 4163 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.662032 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 (DEFAULT_TABLE table=default.Table_485cdcb897e6430ba77e994afbd44a64 [id=2a7d0a1f579a41639b10989738a9c8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.658506 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed141f91ec548db87ac1e719256931b (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.668287 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:35.669082 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.668699 3546 catalog_manager.cc:5495] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.670051 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac558a0e8c943649378539ae2c99654 (DEFAULT_TABLE table=default.Table_e75355b4dffb4b4992e47c68f868d172 [id=c703521c919843518a92cb6008f79401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.670854 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 545610361d354707b143b15d22811911 (DEFAULT_TABLE table=default.Table_c4f7db8f684b436caef8f1315eadb0ee [id=442be0a799fd4e73bfa778f2e6e358be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.671339 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f786a1b5064367ae6d89fc1262a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.674415 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 (DEFAULT_TABLE table=default.Table_58f2aeab56c6446184cfeddcb1134314 [id=e4ed6236205a46d78af24a1e46371aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.674852 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe20c54096804a88ba68b22d0cbea93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675022 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac558a0e8c943649378539ae2c99654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675187 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e71af5ee8b463eb158ac9ae76ff252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675448 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1700ee40ac0d40198c5b8a161a90b6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675420 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ec27862124b74126a62f8588ec1237fa (DEFAULT_TABLE table=default.Table_acd269829eb743178d173c6cc3d7301e [id=9602f52d737341158fe431549212c038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.675734 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac558a0e8c943649378539ae2c99654 (DEFAULT_TABLE table=default.Table_e75355b4dffb4b4992e47c68f868d172 [id=c703521c919843518a92cb6008f79401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.676175 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec27862124b74126a62f8588ec1237fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.686182 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ef47211aa8864d13a55551da47804c19
I20260811 18:57:35.687121 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 22e1a33348e34e35bab786f8966c408b
I20260811 18:57:35.688453 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f" } modify_external_catalogs: true
I20260811 18:57:35.688716 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f" } modify_external_catalogs: true
I20260811 18:57:35.691546 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed141f91ec548db87ac1e719256931b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.691556 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a4f7f1249bb248359c032a4e20f7c9b4" } new_table_name: "default.Table_7b7a6563095e481982b84ec228a04f29" modify_external_catalogs: true
I20260811 18:57:35.693097 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f167c64b15d4d18a642c45b00f75658"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.693413 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ebca8dd587b144d7909761c052084855" } modify_external_catalogs: true
I20260811 18:57:35.693652 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ebca8dd587b144d7909761c052084855" } modify_external_catalogs: true
W20260811 18:57:35.693737 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f167c64b15d4d18a642c45b00f75658 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.694516 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545610361d354707b143b15d22811911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.695147 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c031c1fe18014450aa4cffbc9cd0f573" } new_table_name: "default.Table_4ecec4051b794418b73f208e526c9bbb" modify_external_catalogs: true
I20260811 18:57:35.696663 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c94c9b20d784df0811302a8381a7301" } new_table_name: "default.Table_9049c4cdc3654b26a7b3d0cad713ec01" modify_external_catalogs: true
I20260811 18:57:35.699156 4163 ts_tablet_manager.cc:1397] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.151s user 0.002s sys 0.004s
I20260811 18:57:35.700631 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac558a0e8c943649378539ae2c99654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.641893 4167 ts_tablet_manager.cc:1918] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.702934 4167 log.cc:1196] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:35.703305 4167 ts_tablet_manager.cc:1939] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.704106 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 983) took 1433 ms. Trace:
I20260811 18:57:35.704205 4167 rpcz_store.cc:276] 0811 18:57:34.270140 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.270584 (+ 444us) service_pool.cc:227] Handling call
0811 18:57:35.425313 (+1154729us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:35.425325 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.550562 (+125237us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.641782 (+ 91220us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.704081 (+ 62299us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154583}
I20260811 18:57:35.704567 4167 tablet_replica.cc:330] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.704807 4167 raft_consensus.cc:2229] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.704945 4167 raft_consensus.cc:2258] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.705446 4167 ts_tablet_manager.cc:1905] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.711683 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8edcf621b1064a75906a7ffe63e1a9d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.712379 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8edcf621b1064a75906a7ffe63e1a9d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.712697 4163 raft_consensus.cc:348] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.712930 4163 raft_consensus.cc:374] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.712986 4163 raft_consensus.cc:729] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.718016 4163 consensus_queue.cc:260] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.719458 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_5cf0c22ada454c098b23f5915efe0380 [id=64009634513f46eeae6442cfadb7b0f5] alter complete (version 4)
I20260811 18:57:35.723748 5407 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.723968 5407 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:35.726689 3678 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.731956 4163 ts_tablet_manager.cc:1428] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
W20260811 18:57:35.733458 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 838a9f3025a1466296844c218cedf4fa consensus state has no leader
I20260811 18:57:35.734277 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6f804703e20c4214b8b2a31efeaef59e
W20260811 18:57:35.735441 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 419824ebd14e415f90072b8c335f1ad1 consensus state has no leader
I20260811 18:57:35.735697 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet ef47211aa8864d13a55551da47804c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.735953 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9b931eca7ba04c11af6571cb418f882e
W20260811 18:57:35.737073 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa6509e905694f979191469d20e1de41 consensus state has no leader
I20260811 18:57:35.737017 5511 raft_consensus.cc:482] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.737178 5511 raft_consensus.cc:504] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.737814 5511 leader_election.cc:288] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.739403 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e1a33348e34e35bab786f8966c408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.744307 4163 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.747031 4163 tablet_bootstrap.cc:654] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.748555 5407 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.749262 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d495354c0994fb4a74c376078e41a26" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.749899 3676 leader_election.cc:341] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.750041 3676 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.752214 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 545610361d354707b143b15d22811911 (DEFAULT_TABLE table=default.Table_c4f7db8f684b436caef8f1315eadb0ee [id=442be0a799fd4e73bfa778f2e6e358be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.752369 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e80925854f7a430c93376f6448db3632" } new_table_name: "default.Table_59b6f528f1f44b1996e11ae2e4763f93" modify_external_catalogs: true
I20260811 18:57:35.752709 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545610361d354707b143b15d22811911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.753537 5407 raft_consensus.cc:2726] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.753764 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f4848d8f28744cc49234e7ccfd8d158c (DEFAULT_TABLE table=default.Table_489a043a9e89488ca2fbcd69855c4e5d [id=05d2ef429d684b01a2e8e8d31eeeccf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.754206 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4848d8f28744cc49234e7ccfd8d158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.755192 4162 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.755366 4162 ts_tablet_manager.cc:1397] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.204s user 0.005s sys 0.002s
I20260811 18:57:35.755743 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 13af4d419ae542cc961c75f28efd0204 (DEFAULT_TABLE table=default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.755893 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_458dd09db5894e0fae4cc31deb113bdd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.756502 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_458dd09db5894e0fae4cc31deb113bdd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.756893 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13af4d419ae542cc961c75f28efd0204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.759356 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.759928 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0084f6fe30164d4d9dcc8ad20699380e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.760462 4162 raft_consensus.cc:348] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.760673 4162 raft_consensus.cc:374] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.760730 4162 raft_consensus.cc:729] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.760959 4162 consensus_queue.cc:260] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.764948 4162 ts_tablet_manager.cc:1428] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:35.767051 4162 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.780081 3740 raft_consensus.cc:1261] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:35.798623 4162 tablet_bootstrap.cc:654] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.799743 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f804703e20c4214b8b2a31efeaef59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.799880 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b931eca7ba04c11af6571cb418f882e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:35.800091 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 053d923618464b0c919563b4be9bc118 consensus state has no leader
I20260811 18:57:35.801237 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.809252 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 60091bd891a94cc4aac6ef47673a4ee5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.810775 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet fc3f120758c6405dafd2ea0bf54c81d8 (table default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]) successfully deleted
I20260811 18:57:35.811172 4163 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.811326 4163 ts_tablet_manager.cc:1397] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.006s
I20260811 18:57:35.812124 4163 raft_consensus.cc:348] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.812367 4163 raft_consensus.cc:374] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.812417 4163 raft_consensus.cc:729] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.812824 4163 consensus_queue.cc:260] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.817435 4167 ts_tablet_manager.cc:1918] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.817580 4167 log.cc:1196] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:35.818002 4167 ts_tablet_manager.cc:1939] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.819664 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.820446 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet ef47211aa8864d13a55551da47804c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.820981 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e1a33348e34e35bab786f8966c408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.821451 4163 ts_tablet_manager.cc:1428] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:35.821514 5511 consensus_queue.cc:1035] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:35.823685 3824 leader_election.cc:341] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:35.826105 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f804703e20c4214b8b2a31efeaef59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.827364 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:35.827895 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:35.828248 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.828758 3824 leader_election.cc:334] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.829286 3824 leader_election.cc:302] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.829761 4163 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.830351 5511 raft_consensus.cc:2726] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.832787 4162 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.832934 4162 ts_tablet_manager.cc:1397] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.003s
I20260811 18:57:35.833552 4163 tablet_bootstrap.cc:654] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.839800 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b931eca7ba04c11af6571cb418f882e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.844568 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3f83f2ec5f2847c9820675f2f29f0674"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.845299 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f83f2ec5f2847c9820675f2f29f0674 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.845496 4162 raft_consensus.cc:348] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.845767 4162 raft_consensus.cc:374] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.845829 4162 raft_consensus.cc:729] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.846032 4162 consensus_queue.cc:260] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.846692 4162 ts_tablet_manager.cc:1428] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260811 18:57:35.846997 4162 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.847074 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26f305827594444882a9b3b84edba403" } new_table_name: "default.Table_c33b8642f3da4f97830feaf6f6a67ea8" modify_external_catalogs: true
I20260811 18:57:35.847641 5511 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.847771 5511 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.848366 5511 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.848462 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e394e4379aae4ac8b877d52273a6bf37" } modify_external_catalogs: true
I20260811 18:57:35.848649 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e394e4379aae4ac8b877d52273a6bf37" } modify_external_catalogs: true
I20260811 18:57:35.848999 4162 tablet_bootstrap.cc:654] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.849866 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5437383c0bbb4384825f578e83c10acc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.850440 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5437383c0bbb4384825f578e83c10acc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.851936 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c421939f1354150bddc1156b40cf5fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.852564 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c421939f1354150bddc1156b40cf5fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.857321 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 984) took 1587 ms. Trace:
I20260811 18:57:35.857445 4167 rpcz_store.cc:276] 0811 18:57:34.270301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.275421 (+ 5120us) service_pool.cc:227] Handling call
0811 18:57:35.704518 (+1429097us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:35.704531 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.799227 (+ 94696us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.817316 (+ 18089us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.857294 (+ 39978us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1428906}
I20260811 18:57:35.860569 4163 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.861995 4162 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.862273 4162 ts_tablet_manager.cc:1397] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260811 18:57:35.863178 4162 raft_consensus.cc:348] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.863468 4162 raft_consensus.cc:374] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.863571 4162 raft_consensus.cc:729] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.863871 4162 consensus_queue.cc:260] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.865943 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.865974 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e3021c51525846e199f279f988248af0
W20260811 18:57:35.867012 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:35.870071 4162 ts_tablet_manager.cc:1428] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:35.871220 4162 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.877878 4163 ts_tablet_manager.cc:1397] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.007s
I20260811 18:57:35.881117 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet e3021c51525846e199f279f988248af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.881279 4163 raft_consensus.cc:348] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.871752 4167 tablet_replica.cc:330] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.882280 4167 raft_consensus.cc:2229] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.882513 4163 raft_consensus.cc:374] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.882670 4163 raft_consensus.cc:729] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.883170 4167 raft_consensus.cc:2258] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.883172 4163 consensus_queue.cc:260] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.883978 4163 ts_tablet_manager.cc:1428] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:35.884593 4163 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:35.882654 3824 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7251204325dd424192a6f58e18e71c34
I20260811 18:57:35.887473 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e3021c51525846e199f279f988248af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.889703 4162 tablet_bootstrap.cc:654] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.894498 4167 ts_tablet_manager.cc:1905] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.902302 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3410fcb02604cdabfae4dfa57a0bac0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.903053 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3410fcb02604cdabfae4dfa57a0bac0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:35.903858 3824 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.904062 3824 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.904472 5511 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.909616 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" } modify_external_catalogs: true
I20260811 18:57:35.909794 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" } modify_external_catalogs: true
I20260811 18:57:35.909924 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" } modify_external_catalogs: true
I20260811 18:57:35.910005 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" } modify_external_catalogs: true
I20260811 18:57:35.910684 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1ec52777b5484eb18531a70c114a32f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.911378 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ec52777b5484eb18531a70c114a32f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.911552 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d00d943b778741b4aeec076ba46a9c40 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:35.913017 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
I20260811 18:57:35.914156 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_def64923a6b24aa0a7b9ac4a66dcace5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.914886 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_def64923a6b24aa0a7b9ac4a66dcace5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.917487 5511 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.917665 5511 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.918406 5511 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.922636 4163 tablet_bootstrap.cc:654] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.925982 4162 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.926235 4162 ts_tablet_manager.cc:1397] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.002s
I20260811 18:57:35.927392 4162 raft_consensus.cc:348] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.927656 4162 raft_consensus.cc:374] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.927786 4162 raft_consensus.cc:729] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.928066 4162 consensus_queue.cc:260] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.928678 4162 ts_tablet_manager.cc:1428] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:35.929436 4162 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.939085 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.940296 5407 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.940454 5407 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.941097 5407 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.941696 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ceca57cb245f4a3a6cb4b36f1306d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.942596 3676 leader_election.cc:341] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.943069 4163 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.943341 4163 ts_tablet_manager.cc:1397] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.002s
I20260811 18:57:35.945559 5510 tablet.cc:1722] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.947212 4163 raft_consensus.cc:348] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:35.949533 3678 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.949698 3678 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.949966 5407 raft_consensus.cc:2726] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.950402 4163 raft_consensus.cc:374] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.950263 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:35.950498 4163 raft_consensus.cc:729] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.950559 3740 raft_consensus.cc:2454] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:35.950700 4163 consensus_queue.cc:260] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.953441 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0" } modify_external_catalogs: true
I20260811 18:57:35.953573 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_56b89b2b2c4b45768b2866c74d9f1390" } new_table_name: "default.Table_22bd9bbe66b64e209048c8164f116ba7" modify_external_catalogs: true
I20260811 18:57:35.953753 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0" } modify_external_catalogs: true
I20260811 18:57:35.953838 4163 ts_tablet_manager.cc:1428] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:35.955129 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_485cdcb897e6430ba77e994afbd44a64" } modify_external_catalogs: true
I20260811 18:57:35.955314 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_485cdcb897e6430ba77e994afbd44a64" } modify_external_catalogs: true
I20260811 18:57:35.956218 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cce74eefcdbb44369f1915d3f40f54e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.956825 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cce74eefcdbb44369f1915d3f40f54e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.958177 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3" } new_table_name: "default.Table_6c8484897dd046ab8499d1069356cdcd" modify_external_catalogs: true
I20260811 18:57:35.957834 4163 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.962946 4163 tablet_bootstrap.cc:654] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.941800 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2f2f5ce2b1d14a9a988cff644e196088 (table default.Table_b46915474a4146eebe6a8da8b75a4160 [id=ec725d0625094fc994ee36547333988d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2f2f5ce2b1d14a9a988cff644e196088 consensus state has no leader
I20260811 18:57:35.965509 5526 tablet.cc:1722] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.966248 3824 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.967226 5376 raft_consensus.cc:2781] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:35.967307 5376 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:35.967365 5376 raft_consensus.cc:3037] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.969887 5376 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.970577 5376 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.972887 4162 tablet_bootstrap.cc:654] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.972462 3824 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.975183 4167 ts_tablet_manager.cc:1918] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.975343 4167 log.cc:1196] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0aee6beb2e70421083841520733087f8
I20260811 18:57:35.975802 4167 ts_tablet_manager.cc:1939] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.979072 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 981) took 1709 ms. Trace:
I20260811 18:57:35.979197 4167 rpcz_store.cc:276] 0811 18:57:34.269930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.275792 (+ 5862us) service_pool.cc:227] Handling call
0811 18:57:35.870936 (+1595144us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:35.871195 (+ 259us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.924446 (+ 53251us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.975060 (+ 50614us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.979048 (+ 3988us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1594082}
I20260811 18:57:35.984977 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 911) took 3013 ms. Trace:
I20260811 18:57:35.986578 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:35.986804 3740 raft_consensus.cc:3037] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.986501 5376 raft_consensus.cc:482] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.988350 5376 raft_consensus.cc:504] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.989045 5376 leader_election.cc:288] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:35.989856 3740 raft_consensus.cc:2454] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:35.989199 4169 rpcz_store.cc:276] 0811 18:57:32.971100 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.024937 (+ 53837us) service_pool.cc:227] Handling call
0811 18:57:35.354528 (+2329591us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:35.354547 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.451779 (+ 97232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.527313 (+ 75534us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.984954 (+457641us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2283934}
I20260811 18:57:35.992334 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 980) took 1722 ms. Trace:
I20260811 18:57:35.993530 4162 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.993738 4162 ts_tablet_manager.cc:1397] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.002s
I20260811 18:57:35.996906 4162 raft_consensus.cc:348] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.997169 4162 raft_consensus.cc:374] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.997247 4162 raft_consensus.cc:729] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.997529 4162 consensus_queue.cc:260] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:35.998654 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1af33fd98c6486f975368ec90493db7 consensus state has no leader
I20260811 18:57:35.994230 4169 tablet_replica.cc:330] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.998930 4169 raft_consensus.cc:2229] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.999035 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed17e76c67444161b978d62bf0bef64f" } new_table_name: "default.Table_d1bad93a76ef4643bf354383513e18b8" modify_external_catalogs: true
I20260811 18:57:35.999306 4169 raft_consensus.cc:2258] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.999899 4169 ts_tablet_manager.cc:1905] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.998684 3824 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.000986 5376 raft_consensus.cc:2781] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:36.001258 5376 raft_consensus.cc:686] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:36.001524 5376 consensus_queue.cc:237] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.002123 4162 ts_tablet_manager.cc:1428] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:36.002564 4162 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.992449 4167 rpcz_store.cc:276] 0811 18:57:34.269815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.308290 (+ 38475us) service_pool.cc:227] Handling call
0811 18:57:35.992195 (+1683905us) ts_tablet_manager.cc:1145] Deleting tablet dd80633f926942e88f54b32d9a03ea3a
0811 18:57:35.992207 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.992313 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1682961}
I20260811 18:57:36.002981 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.003525 4167 tablet_replica.cc:330] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:36.003764 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.003998 4167 raft_consensus.cc:2229] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.004189 4167 raft_consensus.cc:2258] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.006261 5407 raft_consensus.cc:482] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.006456 5407 raft_consensus.cc:504] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.006877 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7249f865dd9d4e0fa204c278006d4cee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.007124 5407 leader_election.cc:288] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.007635 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7249f865dd9d4e0fa204c278006d4cee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.008078 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19243ab3d6744aaa87a133f1462cd2fa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.008770 3676 leader_election.cc:341] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.009044 4167 ts_tablet_manager.cc:1905] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.010483 4162 tablet_bootstrap.cc:654] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:36.010885 3678 leader_election.cc:334] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.011021 3678 leader_election.cc:302] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.012356 5490 raft_consensus.cc:2726] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.013491 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:36.013931 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:36.014258 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
W20260811 18:57:36.015842 3824 leader_election.cc:341] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0aee6beb2e70421083841520733087f8
I20260811 18:57:36.016419 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
I20260811 18:57:36.018225 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.018884 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.026127 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.028000 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 1 got 0
I20260811 18:57:36.028470 4162 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.028851 4162 ts_tablet_manager.cc:1397] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.004s
W20260811 18:57:36.029569 3824 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.029927 4162 raft_consensus.cc:348] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.030470 4162 raft_consensus.cc:374] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.030740 4162 raft_consensus.cc:729] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.038594 4162 consensus_queue.cc:260] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.043979 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 0db48155ac754cd3bbac5ac2e21141e6 (DEFAULT_TABLE table=default.Table_91eff2208ee74ec7b44c4908dcdbe335 [id=017537968f9146dd9eff5b3d180287ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.044499 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db48155ac754cd3bbac5ac2e21141e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.044865 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f6b2ca385374abeb709fccbdf6fa0ba consensus state has no leader
I20260811 18:57:36.047047 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
W20260811 18:57:36.049953 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
I20260811 18:57:36.050426 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_767c174a258144229f562c9dda12392b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.050863 4162 ts_tablet_manager.cc:1428] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
W20260811 18:57:36.051069 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_767c174a258144229f562c9dda12392b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.052433 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd (DEFAULT_TABLE table=default.Table_dc1e05602be443528f758fba1ac0195f [id=4986af2a7d044e87ae7468ac0e6abfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.052650 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2053e3676584227a3ea13c4f912e315" } new_table_name: "default.Table_7e6c466c96e8465c9677dcaddee8819a" modify_external_catalogs: true
I20260811 18:57:36.052861 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76506ad179a9408c9e4cf8e4275ebddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.054677 4162 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.058579 4162 tablet_bootstrap.cc:654] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.059325 4167 ts_tablet_manager.cc:1918] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.060405 4167 log.cc:1196] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:36.060873 4167 ts_tablet_manager.cc:1939] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.061775 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 985) took 1650 ms. Trace:
I20260811 18:57:36.061866 4167 rpcz_store.cc:276] 0811 18:57:34.411674 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.411771 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:36.003471 (+1591700us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:36.003484 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.015543 (+ 12059us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.059230 (+ 43687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.061756 (+ 2526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591472}
I20260811 18:57:36.062181 4167 tablet_replica.cc:330] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.062417 4167 raft_consensus.cc:2229] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.062574 4167 raft_consensus.cc:2258] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.063062 4167 ts_tablet_manager.cc:1905] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.063468 5376 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.063591 5376 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.064132 5376 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.072245 4163 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.072450 4163 ts_tablet_manager.cc:1397] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.116s user 0.004s sys 0.000s
I20260811 18:57:36.073302 4162 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.073479 4162 ts_tablet_manager.cc:1397] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:36.074107 4163 raft_consensus.cc:348] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.074400 4163 raft_consensus.cc:374] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.074506 4163 raft_consensus.cc:729] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.074251 4162 raft_consensus.cc:348] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.074623 4162 raft_consensus.cc:374] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.074692 4162 raft_consensus.cc:729] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.074817 4163 consensus_queue.cc:260] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.075105 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90ce2b983a431383edb7a9296520a4 (DEFAULT_TABLE table=default.Table_36c481dceea34aacbb4ad6aabe5e403c [id=eba0b555e52141f1b62443826556b6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.075258 3856 tablet_service.cc:1467] Processing CreateTablet for tablet ff7d3c7a5fcf4c2faec0d41371788b2f (DEFAULT_TABLE table=default.Table_45ec6e6d5001405faabf015cd3bd77e5 [id=89ba458c14f14a2bbe6fe1982c0043c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.075394 4163 ts_tablet_manager.cc:1428] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:36.075567 4162 consensus_queue.cc:260] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.075867 3851 tablet_service.cc:1467] Processing CreateTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af (DEFAULT_TABLE table=default.Table_b7a67e56b3f641139862409e95439c6a [id=05cdc8978385423994bd961054aac04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.076248 4162 ts_tablet_manager.cc:1428] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:57:36.076579 4162 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.076565 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 28aaee10703b4993a701a6eecf52d284 (DEFAULT_TABLE table=default.Table_1fed68aa37dc45c4bd9492de217dd73c [id=5f07f3afc20146048d4e068138204639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.082382 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1700ee40ac0d40198c5b8a161a90b6b6
I20260811 18:57:36.083123 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e89523faca1046669de08be2f6b963dd" } modify_external_catalogs: true
I20260811 18:57:36.083366 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e89523faca1046669de08be2f6b963dd" } modify_external_catalogs: true
I20260811 18:57:36.085134 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5627fd661e4441d8a677d35bc0f16cf0" } new_table_name: "default.Table_2e89aca0564646a5bcc2c6ce3f851910" modify_external_catalogs: true
I20260811 18:57:36.086663 4162 tablet_bootstrap.cc:654] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.095325 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ac48afc7eec4410e8d6055ecdb794a4f (DEFAULT_TABLE table=default.Table_d6113c39441d47a8a1d07df0fbd2371e [id=a14d06dea10b47699e87570920020fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.097420 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 41e4ec7cfec94a1988fc6c950421b48a (DEFAULT_TABLE table=default.Table_8e24f37301a145fe9b5029eaa654e2af [id=9a96027cb9cd4b6c9ec6c42c9d429f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.097779 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90ce2b983a431383edb7a9296520a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.097810 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 56937c88026541e4b70d4f507ce03bc3 (DEFAULT_TABLE table=default.Table_1f988d05c9984be98e5f1f83939bc620 [id=b544982375764fafa02b3af75fec3b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.098008 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ffd3c95b744242ba9dee25d95d8e2b68 (DEFAULT_TABLE table=default.Table_84be3c943435450c938c21da022a54f0 [id=f038b486cea94c9482f0bb57ba9e2784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.098104 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56937c88026541e4b70d4f507ce03bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.098315 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7d3c7a5fcf4c2faec0d41371788b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.098502 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e4ec7cfec94a1988fc6c950421b48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.106372 4167 ts_tablet_manager.cc:1918] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.106508 4167 log.cc:1196] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/059b325df1db4fde9d4b26b559121316
I20260811 18:57:36.106864 4167 ts_tablet_manager.cc:1939] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.108050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 986) took 1668 ms. Trace:
I20260811 18:57:36.108161 4167 rpcz_store.cc:276] 0811 18:57:34.439848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.439948 (+ 100us) service_pool.cc:227] Handling call
0811 18:57:36.062138 (+1622190us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:36.062149 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.066190 (+ 4041us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.106241 (+ 40051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.108029 (+ 1788us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1621912}
I20260811 18:57:36.109841 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65cb8d901c04d8ba2b7a40efc7fa3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.110006 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 987) took 1669 ms. Trace:
I20260811 18:57:36.110080 4167 rpcz_store.cc:276] 0811 18:57:34.440538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.440622 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:36.109883 (+1669261us) ts_tablet_manager.cc:1145] Deleting tablet 40898ceb18cc4f2d9104818798049ae2
0811 18:57:36.109892 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.109992 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1669063}
I20260811 18:57:36.110414 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 988) took 1669 ms. Trace:
I20260811 18:57:36.110484 4167 rpcz_store.cc:276] 0811 18:57:34.440720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.440788 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:36.110296 (+1669508us) ts_tablet_manager.cc:1145] Deleting tablet a25b1eddd4a64847b51ad3cdc0238ff0
0811 18:57:36.110305 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.110399 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1669364}
I20260811 18:57:36.110698 4167 tablet_replica.cc:330] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.109546 4162 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.111438 4162 ts_tablet_manager.cc:1397] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.002s
I20260811 18:57:36.112145 4163 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.112445 4162 raft_consensus.cc:348] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.112689 4162 raft_consensus.cc:374] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.112813 4162 raft_consensus.cc:729] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.113062 4162 consensus_queue.cc:260] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.113834 4162 ts_tablet_manager.cc:1428] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:36.114131 4162 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.114259 4163 tablet_bootstrap.cc:654] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.114625 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28aaee10703b4993a701a6eecf52d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.114995 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd3c95b744242ba9dee25d95d8e2b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.117450 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 38a4783168554053aa953dba5e893b6b
I20260811 18:57:36.118211 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_153744b340b44d8c9e939deedadaced9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.118458 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac48afc7eec4410e8d6055ecdb794a4f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.118887 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153744b340b44d8c9e939deedadaced9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.120101 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 374c94a12e374d0e82180421c80d92d0
I20260811 18:57:36.120690 5490 raft_consensus.cc:482] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.120819 5490 raft_consensus.cc:504] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.121462 5490 leader_election.cc:288] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.122062 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c365f79d4a4201ac323f16bddf3e62" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.122866 3676 leader_election.cc:341] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.123994 3678 leader_election.cc:334] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.124112 3678 leader_election.cc:302] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.124540 5407 raft_consensus.cc:2726] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.127409 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 1811e5f62a714f8388460dfcb1717e81 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:36.129528 4162 tablet_bootstrap.cc:654] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.129793 4167 raft_consensus.cc:2229] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.130007 4167 raft_consensus.cc:2258] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.132752 5490 raft_consensus.cc:482] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.132889 5490 raft_consensus.cc:504] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.133524 5490 leader_election.cc:288] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.133631 4167 ts_tablet_manager.cc:1905] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.134233 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6e0d1eeb63493bbea4cd78dd085a3c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:36.134521 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa57a509ea14c5e935b73663b159ef7 (DEFAULT_TABLE table=default.Table_bd16e9b8092642c896a5be846347ad41 [id=297c5c3b5878432493957c51b96ea17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.134742 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.134951 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a4783168554053aa953dba5e893b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:36.134984 3676 leader_election.cc:341] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.136030 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa57a509ea14c5e935b73663b159ef7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.136124 3678 leader_election.cc:334] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.136240 3678 leader_election.cc:302] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.136629 5490 raft_consensus.cc:2726] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.144506 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b71b6efc42de4200bee281e7e70c6bc1 (DEFAULT_TABLE table=default.Table_567110d2149d4cbfbfcf11fbbaddd0fa [id=9816fe72f79d446588b0ea0b81a79786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.144966 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71b6efc42de4200bee281e7e70c6bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.148311 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 60091bd891a94cc4aac6ef47673a4ee5
I20260811 18:57:36.148849 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 60091bd891a94cc4aac6ef47673a4ee5 deleted and replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7
I20260811 18:57:36.149399 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b807102dd88842e2b6e466a147c5b675"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.149981 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b807102dd88842e2b6e466a147c5b675 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.145313 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5d8c1424a045ee927b79351d236624 (DEFAULT_TABLE table=default.Table_fab5512bddad41239e285f5b77ef1958 [id=dab6b2d17a584bce88a0d6318c810076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.150785 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5d8c1424a045ee927b79351d236624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.150930 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_15068805ee264778ac398274e74d630a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.151441 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15068805ee264778ac398274e74d630a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.144622 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fe080f2071964da6b54f1ee5af98d918 (DEFAULT_TABLE table=default.Table_4f4984467f654126ae2d4a7fe64c94f0 [id=afc91ffd37634c85b9909fcff10ca65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.154985 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe080f2071964da6b54f1ee5af98d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.157119 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fe080f2071964da6b54f1ee5af98d918
I20260811 18:57:36.157869 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fa66e20bff3745ca961cfec0dc3a65c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.158466 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa66e20bff3745ca961cfec0dc3a65c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.162214 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737] alter complete (version 1)
I20260811 18:57:36.163089 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6c18f70552714e0f9a63822c3b6fb0a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.163669 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c18f70552714e0f9a63822c3b6fb0a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.169165 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 18a44d23ec2441b784693e1cbbd62ad8 (DEFAULT_TABLE table=default.Table_a1cb848dc1c1438aa868e39197e05009 [id=a42d5dcf61f14b7caecfbcfe5b654ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.169654 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a44d23ec2441b784693e1cbbd62ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.173388 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd (DEFAULT_TABLE table=default.Table_dc1e05602be443528f758fba1ac0195f [id=4986af2a7d044e87ae7468ac0e6abfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.173799 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76506ad179a9408c9e4cf8e4275ebddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.174597 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90ce2b983a431383edb7a9296520a4 (DEFAULT_TABLE table=default.Table_36c481dceea34aacbb4ad6aabe5e403c [id=eba0b555e52141f1b62443826556b6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.175377 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ff7d3c7a5fcf4c2faec0d41371788b2f (DEFAULT_TABLE table=default.Table_45ec6e6d5001405faabf015cd3bd77e5 [id=89ba458c14f14a2bbe6fe1982c0043c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.175642 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:36.173063 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0db48155ac754cd3bbac5ac2e21141e6 (DEFAULT_TABLE table=default.Table_91eff2208ee74ec7b44c4908dcdbe335 [id=017537968f9146dd9eff5b3d180287ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.182533 4163 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.182698 4163 ts_tablet_manager.cc:1397] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.000s
I20260811 18:57:36.183252 4167 ts_tablet_manager.cc:1918] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.183379 4167 log.cc:1196] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a847581412494e34b2c76e605ff11772
I20260811 18:57:36.183516 4163 raft_consensus.cc:348] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.183693 4167 ts_tablet_manager.cc:1939] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.183749 4163 raft_consensus.cc:374] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.183805 4163 raft_consensus.cc:729] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.183730 3724 tablet_service.cc:1467] Processing CreateTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af (DEFAULT_TABLE table=default.Table_b7a67e56b3f641139862409e95439c6a [id=05cdc8978385423994bd961054aac04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.183981 4163 consensus_queue.cc:260] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.184327 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 28aaee10703b4993a701a6eecf52d284 (DEFAULT_TABLE table=default.Table_1fed68aa37dc45c4bd9492de217dd73c [id=5f07f3afc20146048d4e068138204639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.184769 4163 ts_tablet_manager.cc:1428] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:36.184806 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 41e4ec7cfec94a1988fc6c950421b48a (DEFAULT_TABLE table=default.Table_8e24f37301a145fe9b5029eaa654e2af [id=9a96027cb9cd4b6c9ec6c42c9d429f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.184350 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 989) took 1108 ms. Trace:
I20260811 18:57:36.185395 4162 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.185525 4162 ts_tablet_manager.cc:1397] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.000s
I20260811 18:57:36.185477 3713 tablet_service.cc:1467] Processing CreateTablet for tablet ac48afc7eec4410e8d6055ecdb794a4f (DEFAULT_TABLE table=default.Table_d6113c39441d47a8a1d07df0fbd2371e [id=a14d06dea10b47699e87570920020fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.186252 4162 raft_consensus.cc:348] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.186419 4163 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.186466 4162 raft_consensus.cc:374] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.186529 4162 raft_consensus.cc:729] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.187525 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.187918 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 56937c88026541e4b70d4f507ce03bc3 (DEFAULT_TABLE table=default.Table_1f988d05c9984be98e5f1f83939bc620 [id=b544982375764fafa02b3af75fec3b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188288 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa57a509ea14c5e935b73663b159ef7 (DEFAULT_TABLE table=default.Table_bd16e9b8092642c896a5be846347ad41 [id=297c5c3b5878432493957c51b96ea17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188395 3710 tablet_service.cc:1467] Processing CreateTablet for tablet b71b6efc42de4200bee281e7e70c6bc1 (DEFAULT_TABLE table=default.Table_567110d2149d4cbfbfcf11fbbaddd0fa [id=9816fe72f79d446588b0ea0b81a79786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188637 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 374c94a12e374d0e82180421c80d92d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.188689 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a4783168554053aa953dba5e893b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.190291 5376 raft_consensus.cc:482] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.190465 5376 raft_consensus.cc:504] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.190521 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ffd3c95b744242ba9dee25d95d8e2b68 (DEFAULT_TABLE table=default.Table_84be3c943435450c938c21da022a54f0 [id=f038b486cea94c9482f0bb57ba9e2784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.191061 5376 leader_election.cc:288] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.186755 4162 consensus_queue.cc:260] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.193763 4162 ts_tablet_manager.cc:1428] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260811 18:57:36.194058 4162 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.185148 4167 rpcz_store.cc:276] 0811 18:57:35.075693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.075774 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:36.110660 (+1034886us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:36.110670 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.139947 (+ 29277us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.183139 (+ 43192us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.184329 (+ 1190us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1034638}
I20260811 18:57:36.195297 4167 tablet_replica.cc:330] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.195495 4167 raft_consensus.cc:2229] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.195766 4167 raft_consensus.cc:2258] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:36.197017 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4451cc4d7c5940cdb41c021ae486a5a9 consensus state has no leader
I20260811 18:57:36.202581 4167 ts_tablet_manager.cc:1905] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.204754 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1314176a6f3346d7a808360af2c1cfd2" } new_table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" modify_external_catalogs: true
I20260811 18:57:36.207504 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f2c4aa71e814a5dbc9684802806666e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.208127 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f2c4aa71e814a5dbc9684802806666e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.209264 5376 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.209419 5376 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.210032 5376 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.206820 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db7b48638c894fa5ade5118c99ed4f80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.211601 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db7b48638c894fa5ade5118c99ed4f80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.228623 5490 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.228829 5490 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.229550 5490 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.229697 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 03c50dcffac54c7bac210eeaed340ca6
I20260811 18:57:36.231398 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.233515 3676 leader_election.cc:341] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.234926 3678 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.235069 3678 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.235615 5490 raft_consensus.cc:2726] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.238471 4163 tablet_bootstrap.cc:654] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.240818 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b7ce2b43b2134defb14d45112e4f5e5e (DEFAULT_TABLE table=default.Table_deb475a5050c4f3dbaece0eb1e4ac618 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.241310 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7ce2b43b2134defb14d45112e4f5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.249212 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bc171757977f4404bb736a03a87b9ecd" } new_table_name: "default.Table_f4089a325e0243d28570ad4a523a56b1" modify_external_catalogs: true
W20260811 18:57:36.252555 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:36.256958 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c284a387f502480c8faa36ea0e40a8f7" } new_table_name: "default.Table_abd75a155c204a429d050f24f1f5f0c4" modify_external_catalogs: true
I20260811 18:57:36.277464 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.278299 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.298070 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8fb6e34f004c466381fe6ab5b780568f" } modify_external_catalogs: true
I20260811 18:57:36.298440 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8fb6e34f004c466381fe6ab5b780568f" } modify_external_catalogs: true
I20260811 18:57:36.300993 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fd099e75e1ca407087ce5e01c9916c10 (DEFAULT_TABLE table=default.Table_d9a2caa47da74f95bd2276d3b1547505 [id=c3a362588c3c46d58351c3358fd65e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.301862 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd099e75e1ca407087ce5e01c9916c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.302809 5490 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.302990 5490 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.303728 5490 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.304890 4163 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.305511 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.306311 3676 leader_election.cc:341] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.306468 4163 ts_tablet_manager.cc:1397] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.121s user 0.006s sys 0.002s
W20260811 18:57:36.307597 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.307739 3678 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.308378 5490 raft_consensus.cc:2726] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.308715 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 374c94a12e374d0e82180421c80d92d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.314005 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0aee6beb2e70421083841520733087f8 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
W20260811 18:57:36.315431 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dd80633f926942e88f54b32d9a03ea3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd80633f926942e88f54b32d9a03ea3a already in progress: creating tablet
I20260811 18:57:36.316468 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b57ed7814a1e440cba362613cbab32f8 (table default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]) successfully deleted
I20260811 18:57:36.312703 4163 raft_consensus.cc:348] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.317299 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 059b325df1db4fde9d4b26b559121316 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
W20260811 18:57:36.318447 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 40898ceb18cc4f2d9104818798049ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 40898ceb18cc4f2d9104818798049ae2 already in progress: creating tablet
I20260811 18:57:36.318126 4163 raft_consensus.cc:374] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:36.319695 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a25b1eddd4a64847b51ad3cdc0238ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25b1eddd4a64847b51ad3cdc0238ff0 already in progress: creating tablet
I20260811 18:57:36.319468 4163 raft_consensus.cc:729] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.323604 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a847581412494e34b2c76e605ff11772 (table default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]) successfully deleted
I20260811 18:57:36.326602 3720 tablet_service.cc:1467] Processing CreateTablet for tablet fe080f2071964da6b54f1ee5af98d918 (DEFAULT_TABLE table=default.Table_4f4984467f654126ae2d4a7fe64c94f0 [id=afc91ffd37634c85b9909fcff10ca65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.326853 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:36.327808 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ad53b06897e4559b15d34a4d98f4005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.328415 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ad53b06897e4559b15d34a4d98f4005 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.328828 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 18a44d23ec2441b784693e1cbbd62ad8 (DEFAULT_TABLE table=default.Table_a1cb848dc1c1438aa868e39197e05009 [id=a42d5dcf61f14b7caecfbcfe5b654ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.329360 3707 tablet_service.cc:1467] Processing CreateTablet for tablet b7ce2b43b2134defb14d45112e4f5e5e (DEFAULT_TABLE table=default.Table_deb475a5050c4f3dbaece0eb1e4ac618 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.330004 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5d8c1424a045ee927b79351d236624 (DEFAULT_TABLE table=default.Table_fab5512bddad41239e285f5b77ef1958 [id=dab6b2d17a584bce88a0d6318c810076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.330765 3714 tablet_service.cc:1467] Processing CreateTablet for tablet fd099e75e1ca407087ce5e01c9916c10 (DEFAULT_TABLE table=default.Table_d9a2caa47da74f95bd2276d3b1547505 [id=c3a362588c3c46d58351c3358fd65e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.332155 3708 tablet_service.cc:1467] Processing CreateTablet for tablet ad8b3713cac84544a61e8640c07e876e (DEFAULT_TABLE table=default.Table_8488352995394b02b14172ea5eed8290 [id=fcb0065f413e4e6ab66a4b0861d89ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.331384 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1b44d53ffcc9440fa7594920a05b39a5 (DEFAULT_TABLE table=default.Table_cf1083177b984dbc8f987dc5575ae11a [id=477264043a4649afb58c7591736c35f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.333652 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b44d53ffcc9440fa7594920a05b39a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.337157 3856 tablet_service.cc:1467] Processing CreateTablet for tablet ad8b3713cac84544a61e8640c07e876e (DEFAULT_TABLE table=default.Table_8488352995394b02b14172ea5eed8290 [id=fcb0065f413e4e6ab66a4b0861d89ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.337564 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8b3713cac84544a61e8640c07e876e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.346494 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.348248 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet fe080f2071964da6b54f1ee5af98d918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.349952 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.352073 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet fe080f2071964da6b54f1ee5af98d918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.347873 4163 consensus_queue.cc:260] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.354003 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:36.357250 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8304ac7709ca4392baec4cfa78ef5b41: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.359750 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6648cd60f5214830a53129a2e1cf43f4 (DEFAULT_TABLE table=default.Table_e1f8487f0094407ba50ccdaecfebe7ee [id=3fa1075aa4514eb7a0caf0428fde2faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.360926 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.360810 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.361821 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6648cd60f5214830a53129a2e1cf43f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.363901 4163 ts_tablet_manager.cc:1428] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.057s user 0.002s sys 0.002s
I20260811 18:57:36.365665 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e5981cabc4e7ab53de86bb730c70a (DEFAULT_TABLE table=default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 [id=4ab5b403d47a4b61a8ae7a4748f6dc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.366889 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe879c4960b4e0cbd914f96255eff9d (DEFAULT_TABLE table=default.Table_44d5558ecc5b4b7cba0c0a120270f3eb [id=1b972bac79a94bb5af5833344e38f15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.367071 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 31bb11617f1548f88f534289706edb06 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.367668 3852 tablet_service.cc:1467] Processing CreateTablet for tablet eb52f30af7294caa877b3aa3be275259 (DEFAULT_TABLE table=default.Table_ef967930b08e4f7492ffa9993b43f445 [id=f0849aba28af422ca272f0757878b2dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.371505 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7df549360f014d629938e3e94659af8a" } modify_external_catalogs: true
I20260811 18:57:36.372092 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7df549360f014d629938e3e94659af8a" } modify_external_catalogs: true
I20260811 18:57:36.374293 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e5981cabc4e7ab53de86bb730c70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.375393 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe879c4960b4e0cbd914f96255eff9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.376025 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bb11617f1548f88f534289706edb06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.379521 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb52f30af7294caa877b3aa3be275259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.384557 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 9708871030cc4ea6b620fb647403c42a (DEFAULT_TABLE table=default.Table_744e4342189a4b29b424eebd6ad7542f [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.399227 4163 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:36.402590 3824 leader_election.cc:334] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.402766 3824 leader_election.cc:302] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:36.403061 5511 raft_consensus.cc:2726] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.406728 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.407013 3740 raft_consensus.cc:2454] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:36.421314 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9708871030cc4ea6b620fb647403c42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.421667 5511 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.421819 5511 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.422475 5511 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.427500 4169 ts_tablet_manager.cc:1918] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:36.427654 4169 log.cc:1196] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:36.428107 4169 ts_tablet_manager.cc:1939] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:36.429032 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 915) took 3297 ms. Trace:
I20260811 18:57:36.429152 4169 rpcz_store.cc:276] 0811 18:57:33.131271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.131357 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:35.993486 (+2862129us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:35.993500 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.069932 (+ 76432us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.427385 (+357453us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.429007 (+ 1622us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2861480}
I20260811 18:57:36.429621 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 916) took 3236 ms. Trace:
I20260811 18:57:36.429809 4169 rpcz_store.cc:276] 0811 18:57:33.192772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.194177 (+ 1405us) service_pool.cc:227] Handling call
0811 18:57:36.429469 (+3235292us) ts_tablet_manager.cc:1145] Deleting tablet 6df5545532e346f7abcdc4cd228b84da
0811 18:57:36.429484 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.429601 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3234973}
I20260811 18:57:36.430138 4169 tablet_replica.cc:330] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:36.463747 4169 raft_consensus.cc:2229] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:36.476868 5490 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.477063 5490 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.477768 5490 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.479553 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.480830 3676 leader_election.cc:341] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.481215 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.481326 3678 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.482009 5407 raft_consensus.cc:2726] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.487469 4169 pending_rounds.cc:62] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:36.487744 4169 pending_rounds.cc:68] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400181435674624 op_type: NO_OP noop_request { }
I20260811 18:57:36.488283 4169 raft_consensus.cc:2866] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:36.488715 4169 raft_consensus.cc:2258] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.503548 5407 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.503751 5407 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.504016 4163 tablet_bootstrap.cc:654] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.504433 5407 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.505332 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.506361 3676 leader_election.cc:341] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.514714 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.515007 3678 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.515555 5407 raft_consensus.cc:2726] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.548856 4169 ts_tablet_manager.cc:1905] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:36.596338 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 134828us
I20260811 18:57:36.605265 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.605551 3745 raft_consensus.cc:2454] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:36.623458 5407 raft_consensus.cc:482] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.623656 5407 raft_consensus.cc:504] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.624279 5407 leader_election.cc:288] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.627104 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:36.627429 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452ee450778e4d33adfa7e2745b5d970" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.628885 3676 leader_election.cc:341] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.629536 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
W20260811 18:57:36.630511 3678 leader_election.cc:334] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.630748 3678 leader_election.cc:302] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.631196 5407 raft_consensus.cc:2726] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.631415 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8171ab03efd442a9bf4e1176f04cb24" } modify_external_catalogs: true
I20260811 18:57:36.631711 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8171ab03efd442a9bf4e1176f04cb24" } modify_external_catalogs: true
I20260811 18:57:36.633728 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9cadd28493e444185170bcd4c314208" } modify_external_catalogs: true
I20260811 18:57:36.633996 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9cadd28493e444185170bcd4c314208" } modify_external_catalogs: true
I20260811 18:57:36.654793 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5c323e78021f457095b0ad2754e21334"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.660002 5376 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.660195 5376 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.660997 5376 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:36.666971 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 67348us
W20260811 18:57:36.669519 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 02bf791926d84522b1099c1242acddcd consensus state has no leader
I20260811 18:57:36.670042 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cde3a66762e4c0ea53da6ce0999bf3c" } new_table_name: "default.Table_61e3f65a0ee948fab69aa4add687f2bc" modify_external_catalogs: true
W20260811 18:57:36.678821 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b278420f8d2f4be8b973de3eb756f69a consensus state has no leader
I20260811 18:57:36.708385 4169 ts_tablet_manager.cc:1918] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:36.708786 4169 log.cc:1196] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:36.709833 4169 ts_tablet_manager.cc:1939] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:36.710413 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c323e78021f457095b0ad2754e21334 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.720201 5407 raft_consensus.cc:482] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.720381 5407 raft_consensus.cc:504] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.721038 5407 leader_election.cc:288] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.723822 3678 leader_election.cc:334] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.725807 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4de110bd3348f0b0dae21c9f2b6a19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.726557 3676 leader_election.cc:341] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.726708 3676 leader_election.cc:302] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.727368 5376 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.727519 5376 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.728749 5407 raft_consensus.cc:2726] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.738149 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 917) took 3508 ms. Trace:
I20260811 18:57:36.738481 4169 rpcz_store.cc:276] 0811 18:57:33.208421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.217588 (+ 9167us) service_pool.cc:227] Handling call
0811 18:57:36.430093 (+3212505us) ts_tablet_manager.cc:1145] Deleting tablet b2718cd4a98742dfa8014a8a38c69c45
0811 18:57:36.430105 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.612064 (+181959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.708225 (+ 96161us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.717182 (+ 8957us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200632}
I20260811 18:57:36.746029 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet fe203387bc9c45ba82afe6b058aa5e5f (table default.Table_e67c791ed9cd4da88ae7ce53ba362301 [id=14b4d412d33c43fdb3d20c85b82c1670]) successfully deleted
I20260811 18:57:36.746330 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.746644 3746 raft_consensus.cc:2454] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:36.747812 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6df5545532e346f7abcdc4cd228b84da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df5545532e346f7abcdc4cd228b84da already in progress: creating tablet
I20260811 18:57:36.739970 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 918) took 3487 ms. Trace:
I20260811 18:57:36.748778 4169 rpcz_store.cc:276] 0811 18:57:33.252693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.253338 (+ 645us) service_pool.cc:227] Handling call
0811 18:57:36.739281 (+3485943us) ts_tablet_manager.cc:1145] Deleting tablet 4af28c051e8243eaa73d4a5ed1abcea6
0811 18:57:36.739342 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.739703 (+ 361us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3485044}
I20260811 18:57:36.761811 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 919) took 3385 ms. Trace:
I20260811 18:57:36.765934 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a44d23ec2441b784693e1cbbd62ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.766201 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd099e75e1ca407087ce5e01c9916c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.766942 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71b6efc42de4200bee281e7e70c6bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767197 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe080f2071964da6b54f1ee5af98d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767458 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7ce2b43b2134defb14d45112e4f5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767640 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56937c88026541e4b70d4f507ce03bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767884 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db48155ac754cd3bbac5ac2e21141e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.769447 4162 tablet_bootstrap.cc:654] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.769713 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac48afc7eec4410e8d6055ecdb794a4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.770493 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8b3713cac84544a61e8640c07e876e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.774480 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5d8c1424a045ee927b79351d236624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.774560 5511 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.767292 4169 rpcz_store.cc:276] 0811 18:57:33.374994 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.375058 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:36.756311 (+3381253us) ts_tablet_manager.cc:1145] Deleting tablet 9608d6ddcad749cd9e72dd6f265db0da
0811 18:57:36.756546 (+ 235us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.760458 (+ 3912us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3379861}
I20260811 18:57:36.774712 5511 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.775048 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 934) took 2774 ms. Trace:
I20260811 18:57:36.775161 4169 rpcz_store.cc:276] 0811 18:57:34.000113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.005281 (+ 5168us) service_pool.cc:227] Handling call
0811 18:57:36.774860 (+2769579us) ts_tablet_manager.cc:1145] Deleting tablet 05cba6b392b642e59321570aac3541d0
0811 18:57:36.774880 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.775021 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2769358}
I20260811 18:57:36.775436 5511 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.775597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 935) took 2762 ms. Trace:
I20260811 18:57:36.775694 4169 rpcz_store.cc:276] 0811 18:57:34.012880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.013941 (+ 1061us) service_pool.cc:227] Handling call
0811 18:57:36.775464 (+2761523us) ts_tablet_manager.cc:1145] Deleting tablet 101994cc708a4fadb46fc8c5915a5587
0811 18:57:36.775479 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.775577 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761307}
I20260811 18:57:36.776096 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 939) took 2513 ms. Trace:
I20260811 18:57:36.777621 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd3c95b744242ba9dee25d95d8e2b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.780313 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa57a509ea14c5e935b73663b159ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.786739 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65cb8d901c04d8ba2b7a40efc7fa3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.790644 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e4ec7cfec94a1988fc6c950421b48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.796878 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90ce2b983a431383edb7a9296520a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.805383 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2698f49dd4b64fce9a411b1c3307abf2" } new_table_name: "default.Table_cbae90dc1ddd458f9780ee1f8a41c264" modify_external_catalogs: true
I20260811 18:57:36.806536 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7d3c7a5fcf4c2faec0d41371788b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.807672 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c3df98432314ff2b4cbd7004701b258" } modify_external_catalogs: true
I20260811 18:57:36.807941 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c3df98432314ff2b4cbd7004701b258" } modify_external_catalogs: true
I20260811 18:57:36.812987 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=398 overwritten=0 applied=397 ignored=0} inserts{seen=369 ignored=0} mutations{seen=440 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:57:36.812882 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c767befd17049f8a118fdbd2a89e55d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.814429 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap complete.
I20260811 18:57:36.814914 5407 raft_consensus.cc:482] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.815048 5407 raft_consensus.cc:504] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.815641 5407 leader_election.cc:288] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.816478 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28aaee10703b4993a701a6eecf52d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.818061 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 73c519528f75423195b6bdf6683fd91b
W20260811 18:57:36.818567 3678 leader_election.cc:334] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.819967 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
I20260811 18:57:36.834619 4167 ts_tablet_manager.cc:1918] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.834900 4167 log.cc:1196] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7057b8a7bae249caaed304b5c66e6871
W20260811 18:57:36.835057 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c767befd17049f8a118fdbd2a89e55d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.835494 4167 ts_tablet_manager.cc:1939] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.836822 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 990) took 1747 ms. Trace:
I20260811 18:57:36.837539 4167 rpcz_store.cc:276] 0811 18:57:35.089474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089549 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:36.194683 (+1105134us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:36.195239 (+ 556us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.803284 (+608045us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.833806 (+ 30522us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.836801 (+ 2995us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1104845}
I20260811 18:57:36.835753 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.838450 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.839022 4167 tablet_replica.cc:330] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:36.839145 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ad9e3991d0f4274a4770a8f264f403e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.839767 4167 raft_consensus.cc:2229] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.840332 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3ad5f39f59b8447cb3f15b9646e4b852
I20260811 18:57:36.841151 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a431bdf0c984b91935aed8346d63c56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.841778 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a431bdf0c984b91935aed8346d63c56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.842653 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01425dcfb75d41f6ad6349c65957ebac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.843271 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01425dcfb75d41f6ad6349c65957ebac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.845249 4167 raft_consensus.cc:2258] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.845468 4162 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.845614 4162 ts_tablet_manager.cc:1397] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.652s user 0.023s sys 0.026s
I20260811 18:57:36.845918 4167 ts_tablet_manager.cc:1905] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.846370 4162 raft_consensus.cc:348] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.846578 4162 raft_consensus.cc:374] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.846630 4162 raft_consensus.cc:729] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.846540 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fccc76d3d4284078bfd4edff47287a40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.846827 4162 consensus_queue.cc:260] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:36.847102 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fccc76d3d4284078bfd4edff47287a40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.849486 4162 ts_tablet_manager.cc:1428] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:36.854533 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e5981cabc4e7ab53de86bb730c70a (DEFAULT_TABLE table=default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 [id=4ab5b403d47a4b61a8ae7a4748f6dc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.854852 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.855206 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 1b44d53ffcc9440fa7594920a05b39a5 (DEFAULT_TABLE table=default.Table_cf1083177b984dbc8f987dc5575ae11a [id=477264043a4649afb58c7591736c35f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.776918 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:36.789129 5511 raft_consensus.cc:482] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.854873 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 6648cd60f5214830a53129a2e1cf43f4 (DEFAULT_TABLE table=default.Table_e1f8487f0094407ba50ccdaecfebe7ee [id=3fa1075aa4514eb7a0caf0428fde2faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.855969 5511 raft_consensus.cc:504] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.789256 5417 raft_consensus.cc:482] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.856685 5417 raft_consensus.cc:504] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.857185 5417 leader_election.cc:288] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.795923 5376 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.857321 4167 ts_tablet_manager.cc:1918] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.857494 4167 log.cc:1196] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2c0f6df7a291456d80538e2b76417873
I20260811 18:57:36.856593 5511 leader_election.cc:288] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.858016 4167 ts_tablet_manager.cc:1939] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.859324 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7cb3e997b4454598bdce102e92296887
W20260811 18:57:36.861800 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7bfa85fca4bd41dc83acb02ac610cf97 consensus state has no leader
I20260811 18:57:36.796537 5557 consensus_queue.cc:1035] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:36.864988 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:36.865394 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 31bb11617f1548f88f534289706edb06 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.804478 4163 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.819051 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27cda53b7ca54346a7a22bf26f5db990" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:36.866882 4163 ts_tablet_manager.cc:1397] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.471s user 0.009s sys 0.017s
W20260811 18:57:36.867228 3676 leader_election.cc:341] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.867347 3676 leader_election.cc:302] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.867707 5407 raft_consensus.cc:2726] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.868005 4163 raft_consensus.cc:348] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.868268 4163 raft_consensus.cc:374] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.868341 4163 raft_consensus.cc:729] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:36.855616 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 66614.6us
I20260811 18:57:36.868717 3824 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.868537 4163 consensus_queue.cc:260] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:36.868999 3824 leader_election.cc:341] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:36.869177 5511 raft_consensus.cc:2781] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.869274 5511 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.869417 5511 raft_consensus.cc:3037] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.869441 4163 ts_tablet_manager.cc:1428] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:36.856580 4169 rpcz_store.cc:276] 0811 18:57:34.262216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.262271 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:36.775954 (+2513683us) ts_tablet_manager.cc:1145] Deleting tablet 5ce3f2f6f6ff41a2b02492fae757ffd7
0811 18:57:36.775967 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.776055 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2513445}
I20260811 18:57:36.869287 3824 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.870730 4163 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.871065 3824 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.871423 5557 raft_consensus.cc:2781] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.871497 5557 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.871564 5557 raft_consensus.cc:3037] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.872236 5417 raft_consensus.cc:2781] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.872313 5417 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.872373 5417 raft_consensus.cc:3037] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.872378 4163 tablet_bootstrap.cc:654] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.875110 5417 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.875319 5557 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.875754 5417 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.876025 5557 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.877074 3721 tablet_service.cc:1467] Processing CreateTablet for tablet eb52f30af7294caa877b3aa3be275259 (DEFAULT_TABLE table=default.Table_ef967930b08e4f7492ffa9993b43f445 [id=f0849aba28af422ca272f0757878b2dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.877518 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe879c4960b4e0cbd914f96255eff9d (DEFAULT_TABLE table=default.Table_44d5558ecc5b4b7cba0c0a120270f3eb [id=1b972bac79a94bb5af5833344e38f15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.877959 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 9708871030cc4ea6b620fb647403c42a (DEFAULT_TABLE table=default.Table_744e4342189a4b29b424eebd6ad7542f [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.881129 5511 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.882021 5511 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.882726 3706 tablet_service.cc:1467] Processing CreateTablet for tablet dfb6ac49f1d4403dad38a3125264088a (DEFAULT_TABLE table=default.Table_c12987db235f4f4fb949580479275ec2 [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.883719 5327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:36.890534 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 943) took 2618 ms. Trace:
I20260811 18:57:36.890720 4169 rpcz_store.cc:276] 0811 18:57:34.272149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.274568 (+ 2419us) service_pool.cc:227] Handling call
0811 18:57:36.870175 (+2595607us) ts_tablet_manager.cc:1145] Deleting tablet 7216c095cc03424bbbe97a52a9600ebb
0811 18:57:36.876304 (+ 6129us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.890490 (+ 14186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2595411}
I20260811 18:57:36.892050 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3b056c714ed947198f75f2c7802ae4dd" } new_table_name: "default.Table_69bcdc9302e4490fb9ff40912f5d05b3" modify_external_catalogs: true
I20260811 18:57:36.894140 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_33c7b91319d04649821da27a7798271a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.895205 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9db0e2b871fd484f82e42660d2a09c55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.895812 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9db0e2b871fd484f82e42660d2a09c55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:36.897104 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33c7b91319d04649821da27a7798271a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.899861 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f4e1fc5686944dfa3eb57702c810eb8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:36.907940 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8f9b279c6c7344f1931b81e10248b1b6
I20260811 18:57:36.908969 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2962915ef36940f8b6dee3a420d74c59" } new_table_name: "default.Table_b24e66bb2c8b41a582869a9ee02d6745" modify_external_catalogs: true
I20260811 18:57:36.919188 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e5981cabc4e7ab53de86bb730c70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.921500 4162 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.923043 4162 tablet_bootstrap.cc:654] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:36.923965 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 52935.2us
I20260811 18:57:36.925225 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 992) took 1835 ms. Trace:
I20260811 18:57:36.925335 4167 rpcz_store.cc:276] 0811 18:57:35.089692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089881 (+ 189us) service_pool.cc:227] Handling call
0811 18:57:36.838968 (+1749087us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:36.838981 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.851221 (+ 12240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.857206 (+ 5985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.925203 (+ 67997us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1748945}
I20260811 18:57:36.925729 4167 tablet_replica.cc:330] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.925977 4167 raft_consensus.cc:2229] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.926137 4167 raft_consensus.cc:2258] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.926762 4167 ts_tablet_manager.cc:1905] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.928128 5417 consensus_queue.cc:1035] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:36.928258 3824 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:36.929114 3860 tablet_service.cc:1467] Processing CreateTablet for tablet dfb6ac49f1d4403dad38a3125264088a (DEFAULT_TABLE table=default.Table_c12987db235f4f4fb949580479275ec2 [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.929849 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.930428 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c519528f75423195b6bdf6683fd91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.930953 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.932406 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.932866 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb3e997b4454598bdce102e92296887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.933328 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.933761 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.936892 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bb11617f1548f88f534289706edb06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937175 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb52f30af7294caa877b3aa3be275259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937410 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe879c4960b4e0cbd914f96255eff9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937608 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6648cd60f5214830a53129a2e1cf43f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.940994 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b44d53ffcc9440fa7594920a05b39a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.943636 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2523fa07587246d994eca722c7032105 consensus state has no leader
I20260811 18:57:36.944106 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_83c2a159da7a4474bd29c31afe70760e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.944958 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9708871030cc4ea6b620fb647403c42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.945063 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_85a94fea83f84948a14090e2deeac956" } new_table_name: "default.Table_34fae463f93a44a2a142745f939fbf2b" modify_external_catalogs: true
I20260811 18:57:36.948592 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb6ac49f1d4403dad38a3125264088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.954025 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb6ac49f1d4403dad38a3125264088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.954200 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c519528f75423195b6bdf6683fd91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:36.954311 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet fa6509e905694f979191469d20e1de41 (table default.Table_4ecec4051b794418b73f208e526c9bbb [id=b9b4076a00c247e38bbe4223c548d8d2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fa6509e905694f979191469d20e1de41 consensus state has no leader
W20260811 18:57:36.958228 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83c2a159da7a4474bd29c31afe70760e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.960975 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.964156 5327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb005c681cf42d4ac278ec73a28e4a9, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.965165 5327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 397, Last appended: 1.398, Last appended by leader: 398, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:36.977943 5327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:36.979192 4167 ts_tablet_manager.cc:1918] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.979465 4167 log.cc:1196] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:36.979947 4169 tablet_replica.cc:330] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:36.980099 4167 ts_tablet_manager.cc:1939] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.980213 4169 raft_consensus.cc:2229] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.975515 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:36.981375 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:36.981559 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 993) took 1891 ms. Trace:
I20260811 18:57:36.981786 4167 rpcz_store.cc:276] 0811 18:57:35.089798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089959 (+ 161us) service_pool.cc:227] Handling call
0811 18:57:36.925660 (+1835701us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:36.925675 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.929996 (+ 4321us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.979075 (+ 49079us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.981427 (+ 2352us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1835514}
I20260811 18:57:36.982295 4167 tablet_replica.cc:330] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.982595 4167 raft_consensus.cc:2229] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.982774 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.982838 4167 raft_consensus.cc:2258] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.983559 4167 ts_tablet_manager.cc:1905] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.984694 4163 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.984881 4163 ts_tablet_manager.cc:1397] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.114s user 0.022s sys 0.018s
I20260811 18:57:36.985683 4163 raft_consensus.cc:348] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.985909 4163 raft_consensus.cc:374] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.985958 4163 raft_consensus.cc:729] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.986182 4163 consensus_queue.cc:260] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.986857 4163 ts_tablet_manager.cc:1428] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:36.987095 4163 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.988881 4163 tablet_bootstrap.cc:654] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.980415 4169 raft_consensus.cc:2258] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.994213 4169 ts_tablet_manager.cc:1905] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.999770 4167 ts_tablet_manager.cc:1918] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.000042 4167 log.cc:1196] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:37.000953 4167 ts_tablet_manager.cc:1939] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.002008 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 994) took 1912 ms. Trace:
I20260811 18:57:37.002226 4167 rpcz_store.cc:276] 0811 18:57:35.089906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.090187 (+ 281us) service_pool.cc:227] Handling call
0811 18:57:36.982248 (+1892061us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:36.982258 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.990750 (+ 8492us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.999632 (+ 8882us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.001981 (+ 2349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1891924}
I20260811 18:57:37.003090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 999) took 1911 ms. Trace:
I20260811 18:57:37.003250 4167 rpcz_store.cc:276] 0811 18:57:35.091592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091643 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:37.002923 (+1911280us) ts_tablet_manager.cc:1145] Deleting tablet 1c1896700e724204a37439f134a4d988
0811 18:57:37.002937 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.003071 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911119}
I20260811 18:57:37.003782 4167 tablet_replica.cc:330] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.004067 4167 raft_consensus.cc:2229] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.004297 4167 raft_consensus.cc:2258] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.005016 4167 ts_tablet_manager.cc:1905] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.010959 4162 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.011233 4162 ts_tablet_manager.cc:1397] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.090s user 0.003s sys 0.000s
I20260811 18:57:37.012946 4167 ts_tablet_manager.cc:1918] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.013173 4167 log.cc:1196] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:37.013826 4167 ts_tablet_manager.cc:1939] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.015525 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 991) took 1925 ms. Trace:
I20260811 18:57:37.015708 4167 rpcz_store.cc:276] 0811 18:57:35.089591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091874 (+ 2283us) service_pool.cc:227] Handling call
0811 18:57:37.003733 (+1911859us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:37.003745 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.008691 (+ 4946us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.012829 (+ 4138us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.015482 (+ 2653us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911699}
I20260811 18:57:37.016533 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1001) took 1924 ms. Trace:
I20260811 18:57:37.016709 4167 rpcz_store.cc:276] 0811 18:57:35.091887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091927 (+ 40us) service_pool.cc:227] Handling call
0811 18:57:37.016373 (+1924446us) ts_tablet_manager.cc:1145] Deleting tablet afa57ed239ac4afebe517cc48d879645
0811 18:57:37.016385 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.016512 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924283}
I20260811 18:57:37.017356 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1000) took 1925 ms. Trace:
I20260811 18:57:37.017508 4167 rpcz_store.cc:276] 0811 18:57:35.091691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092061 (+ 370us) service_pool.cc:227] Handling call
0811 18:57:37.017217 (+1925156us) ts_tablet_manager.cc:1145] Deleting tablet 4dd203b7eb8643f493b4642c0396e258
0811 18:57:37.017229 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.017339 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924942}
I20260811 18:57:37.018042 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1002) took 1926 ms. Trace:
I20260811 18:57:37.018190 4167 rpcz_store.cc:276] 0811 18:57:35.092046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092310 (+ 264us) service_pool.cc:227] Handling call
0811 18:57:37.017933 (+1925623us) ts_tablet_manager.cc:1145] Deleting tablet b27f9dbb52f34687a33c2ebf20e8f780
0811 18:57:37.017943 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.018028 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1925504}
I20260811 18:57:37.018639 4167 tablet_replica.cc:330] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.018900 4167 raft_consensus.cc:2229] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.019129 4167 raft_consensus.cc:2258] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.020871 4167 ts_tablet_manager.cc:1905] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:37.021548 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 85652.2us
I20260811 18:57:37.022637 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" } modify_external_catalogs: true
I20260811 18:57:37.022678 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28f0027f9b254195878661731f172f78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.022895 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" } modify_external_catalogs: true
W20260811 18:57:37.023331 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28f0027f9b254195878661731f172f78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.024983 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" } modify_external_catalogs: true
I20260811 18:57:37.025220 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" } modify_external_catalogs: true
I20260811 18:57:37.025918 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:37.026520 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4af28c051e8243eaa73d4a5ed1abcea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af28c051e8243eaa73d4a5ed1abcea6 already in progress: creating tablet
W20260811 18:57:37.027271 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5193d57970f2486795256698cdf51e6a consensus state has no leader
I20260811 18:57:37.027786 4163 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.027981 4163 ts_tablet_manager.cc:1397] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.001s
W20260811 18:57:37.031265 3678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 144 similar messages]
I20260811 18:57:37.031352 4167 ts_tablet_manager.cc:1918] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.031287 4163 raft_consensus.cc:348] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.031509 4163 raft_consensus.cc:374] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.031540 4167 log.cc:1196] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0bfb54175c834517b7af291c942333cd
I20260811 18:57:37.031566 4163 raft_consensus.cc:729] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.031777 4163 consensus_queue.cc:260] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.031960 4167 ts_tablet_manager.cc:1939] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.032402 4163 ts_tablet_manager.cc:1428] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:37.032722 4163 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.034277 4163 tablet_bootstrap.cc:654] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.038614 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0cdd7e4b15384ec8844f2e31a5496d8a consensus state has no leader
I20260811 18:57:37.040246 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 998) took 1948 ms. Trace:
I20260811 18:57:37.040449 4167 rpcz_store.cc:276] 0811 18:57:35.091498 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092542 (+ 1044us) service_pool.cc:227] Handling call
0811 18:57:37.018597 (+1926055us) ts_tablet_manager.cc:1145] Deleting tablet 0bfb54175c834517b7af291c942333cd
0811 18:57:37.018607 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.025884 (+ 7277us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.031213 (+ 5329us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.040217 (+ 9004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1925910}
I20260811 18:57:37.040652 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:37.045743 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9608d6ddcad749cd9e72dd6f265db0da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9608d6ddcad749cd9e72dd6f265db0da already in progress: creating tablet
I20260811 18:57:37.044847 4162 raft_consensus.cc:348] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.049124 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 05cba6b392b642e59321570aac3541d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05cba6b392b642e59321570aac3541d0 already in progress: creating tablet
I20260811 18:57:37.050031 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:37.050321 4162 raft_consensus.cc:374] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.050441 4162 raft_consensus.cc:729] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.047686 4167 tablet_replica.cc:330] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.051267 4167 raft_consensus.cc:2229] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.051453 4167 raft_consensus.cc:2258] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.051713 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5aa83c98b49c466cb01cdf92154ad384" } modify_external_catalogs: true
I20260811 18:57:37.051965 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5aa83c98b49c466cb01cdf92154ad384" } modify_external_catalogs: true
I20260811 18:57:37.052119 4167 ts_tablet_manager.cc:1905] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:37.055528 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 101994cc708a4fadb46fc8c5915a5587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 101994cc708a4fadb46fc8c5915a5587 already in progress: creating tablet
I20260811 18:57:37.050695 4162 consensus_queue.cc:260] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.060976 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:37.061220 3745 raft_consensus.cc:2454] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:37.066664 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:37.068222 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb3e997b4454598bdce102e92296887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:37.069195 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 already in progress: creating tablet
I20260811 18:57:37.062675 4162 ts_tablet_manager.cc:1428] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.051s user 0.002s sys 0.001s
I20260811 18:57:37.071342 4162 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.071462 4163 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.071615 4163 ts_tablet_manager.cc:1397] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20260811 18:57:37.072695 4163 raft_consensus.cc:348] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.072906 4163 raft_consensus.cc:374] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.072961 4163 raft_consensus.cc:729] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.072798 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2c977f4c94664fcb88bb338b5c79edfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.072928 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1cb848dc1c1438aa868e39197e05009" } new_table_name: "default.Table_236ea19297e44d6ea717601dbb1a318b" modify_external_catalogs: true
I20260811 18:57:37.073184 4163 consensus_queue.cc:260] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.073450 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c977f4c94664fcb88bb338b5c79edfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.073865 4163 ts_tablet_manager.cc:1428] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:37.074270 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_772fdfa6598e4b2ca2aa3a4b28746b24" } new_table_name: "default.Table_075181f96b7b42059289018e831e2671" modify_external_catalogs: true
I20260811 18:57:37.074747 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7f4e1fc5686944dfa3eb57702c810eb8
I20260811 18:57:37.074975 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fccc76d3d4284078bfd4edff47287a40" } modify_external_catalogs: true
I20260811 18:57:37.075356 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fccc76d3d4284078bfd4edff47287a40" } modify_external_catalogs: true
I20260811 18:57:37.075462 4163 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.075520 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ea579c33f8b14e9fa7fa389ce7f45931
I20260811 18:57:37.076016 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 7f4e1fc5686944dfa3eb57702c810eb8 deleted and replaced by tablet b192929b6d154885a0cc44c00f1add8e
I20260811 18:57:37.076206 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7057b8a7bae249caaed304b5c66e6871 (table default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]) successfully deleted
W20260811 18:57:37.076647 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7216c095cc03424bbbe97a52a9600ebb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7216c095cc03424bbbe97a52a9600ebb already in progress: creating tablet
I20260811 18:57:37.071395 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:37.072371 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.078099 3745 raft_consensus.cc:3037] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.072512 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.078264 3740 raft_consensus.cc:3037] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.080873 3740 raft_consensus.cc:2454] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.080994 3745 raft_consensus.cc:2454] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.075767 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.081493 3743 raft_consensus.cc:3037] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.083768 4162 tablet_bootstrap.cc:654] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.086907 4163 tablet_bootstrap.cc:654] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.089295 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2c0f6df7a291456d80538e2b76417873 (table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]) successfully deleted
I20260811 18:57:37.090605 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8cce97bfdb1d41e685e091c8c5cc6946 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:37.098848 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ce94d37efd2245bf8cc71fdd90bd9735 (table default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]) successfully deleted
W20260811 18:57:37.099189 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1c1896700e724204a37439f134a4d988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c1896700e724204a37439f134a4d988 already in progress: creating tablet
I20260811 18:57:37.099560 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0cccebfc54484a9b9abffcce4e459fb5 (table default.Table_d88f576cc3ca4c6eadc6e071aacc15b2 [id=5fc5ade3f38c438d9b501127ac866b88]) successfully deleted
W20260811 18:57:37.099783 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet afa57ed239ac4afebe517cc48d879645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa57ed239ac4afebe517cc48d879645 already in progress: creating tablet
W20260811 18:57:37.100032 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4dd203b7eb8643f493b4642c0396e258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4dd203b7eb8643f493b4642c0396e258 already in progress: creating tablet
W20260811 18:57:37.100275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b27f9dbb52f34687a33c2ebf20e8f780 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b27f9dbb52f34687a33c2ebf20e8f780 already in progress: creating tablet
I20260811 18:57:37.100473 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0bfb54175c834517b7af291c942333cd (table default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]) successfully deleted
I20260811 18:57:37.103860 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:37.104127 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:37.108741 4163 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.108927 4163 ts_tablet_manager.cc:1397] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:37.109790 4163 raft_consensus.cc:348] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.110065 4163 raft_consensus.cc:374] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.110142 4163 raft_consensus.cc:729] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.110369 4163 consensus_queue.cc:260] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.110795 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.084043 3743 raft_consensus.cc:2454] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.101410 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.113791 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e" } modify_external_catalogs: true
I20260811 18:57:37.111059 4163 ts_tablet_manager.cc:1428] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:37.116528 3824 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.118716 3824 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.118856 3824 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.121697 3717 tablet_service.cc:1467] Processing CreateTablet for tablet eb2ce551cae74423ba2a7b521fe1e44f (DEFAULT_TABLE table=default.Table_7249f865dd9d4e0fa204c278006d4cee [id=ce14e61175f7486aa71cabde45e4fc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.121974 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 026245dfb0284d95a1afae43c77074c5 (DEFAULT_TABLE table=default.Table_def64923a6b24aa0a7b9ac4a66dcace5 [id=1e2eeed8d9aa4ce0935705782a7a7e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.122177 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2ce551cae74423ba2a7b521fe1e44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.122423 4163 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.122505 3721 tablet_service.cc:1467] Processing CreateTablet for tablet f5a969dbd34f414ba3a0b45748510632 (DEFAULT_TABLE table=default.Table_1ec52777b5484eb18531a70c114a32f6 [id=0d5ee633d4ab43a58bf9009459ee7699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.122825 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a969dbd34f414ba3a0b45748510632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.124181 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d21973bc8e514a089c2beb24a686060f
I20260811 18:57:37.124931 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2f2f5ce2b1d14a9a988cff644e196088
I20260811 18:57:37.125514 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e" } modify_external_catalogs: true
I20260811 18:57:37.126631 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 38f64f32463148349a2b0d403ae881e9 (DEFAULT_TABLE table=default.Table_cce74eefcdbb44369f1915d3f40f54e2 [id=43162c1c44f442fb912c654af76eac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.127101 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026245dfb0284d95a1afae43c77074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.127511 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f64f32463148349a2b0d403ae881e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.128603 4163 tablet_bootstrap.cc:654] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.132777 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet a3973dfdaac746a6af7a402277d2711b from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.134048 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 20c4f222b5054527b18f2874dacc5558 (DEFAULT_TABLE table=default.Table_b807102dd88842e2b6e466a147c5b675 [id=c97451803afd4beab9583182c36fbac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.134472 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20c4f222b5054527b18f2874dacc5558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.135169 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 558dc0f1c46840d0991d08d792f903d5 (DEFAULT_TABLE table=default.Table_458dd09db5894e0fae4cc31deb113bdd [id=aa118bfdb3b440a1bc81ca195f425546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135419 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5baa02ab900a4a8893c8e40d6ae5cf58 (DEFAULT_TABLE table=default.Table_7407d7434a3b4290889506a933ae2f05 [id=fe40a820d5cd401c8fc8393b02de9689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135957 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 3edf59b74ee245be852ece5ba50a1054 (DEFAULT_TABLE table=default.Table_6aacb785fd444621820be42007d55668 [id=996a202868ee4f6dadf4be343005550a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135515 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558dc0f1c46840d0991d08d792f903d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.136770 4169 ts_tablet_manager.cc:1918] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.136885 4169 log.cc:1196] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:37.137182 4169 ts_tablet_manager.cc:1939] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.137578 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 950) took 2841 ms. Trace:
I20260811 18:57:37.137670 4169 rpcz_store.cc:276] 0811 18:57:34.296426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.296581 (+ 155us) service_pool.cc:227] Handling call
0811 18:57:36.891109 (+2594528us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:36.979870 (+ 88761us) spinlock_profiling.cc:243] Waited 84.2 ms on lock 0x618000005118. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d090f80fd 00007f4d090f916e 00007f4d1c04dcfd 00007f4d1bfcb883 00007f4d1826b6e6 00007f4d1826ca37 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c
0811 18:57:36.979886 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.997662 (+ 17776us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.136659 (+138997us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.137558 (+ 899us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":193548160,"tablet-delete.queue_time_us":2594307}
I20260811 18:57:37.138190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 948) took 2844 ms. Trace:
I20260811 18:57:37.140309 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a385abaeb70347dcac06a5cedbdddeb4 (DEFAULT_TABLE table=default.Table_5437383c0bbb4384825f578e83c10acc [id=78cd09fbad264f7385b142dcf7ff57b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.140679 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a385abaeb70347dcac06a5cedbdddeb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.142446 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3edf59b74ee245be852ece5ba50a1054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.144925 5557 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.145313 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:37.146224 3824 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.146466 3824 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.146646 3824 leader_election.cc:334] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.146724 3824 leader_election.cc:302] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.149000 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5baa02ab900a4a8893c8e40d6ae5cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.151755 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2afc6c2e3a254c57840c4e891d796bf5" } new_table_name: "default.Table_3f202b05f18d4434bb51caa2cd6d23b8" modify_external_catalogs: true
I20260811 18:57:37.152149 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1ec50daf17974aef9e292bab9fc018cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.152814 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ec50daf17974aef9e292bab9fc018cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.154618 5557 raft_consensus.cc:2726] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.155756 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb6981748ef0423aaaa9a85bacef95be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.156366 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb6981748ef0423aaaa9a85bacef95be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.157218 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f13412c5533e4d9ebb03d46521d325ea
W20260811 18:57:37.157351 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18a44d23ec2441b784693e1cbbd62ad8 consensus state has no leader
I20260811 18:57:37.157639 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c364560f47804860afd5635ea9b6885d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.157784 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_faccbd3f194043708dbbd12a50f037cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.158214 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_faccbd3f194043708dbbd12a50f037cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.158214 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c364560f47804860afd5635ea9b6885d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.158972 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1365655a76d7454c8415102e75960152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.159518 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1365655a76d7454c8415102e75960152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.160760 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:37.161374 5557 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.161514 5557 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.162153 5557 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.163656 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
I20260811 18:57:37.168506 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_606ca53b362a44b4ba72954536b85633"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.169100 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_606ca53b362a44b4ba72954536b85633 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.170614 3824 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:37.172700 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa5a3d1ada554d778d507382c8836133 consensus state has no leader
I20260811 18:57:37.173540 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f2b66196716a4ba3893b6b8bfec2b8c0
I20260811 18:57:37.174399 5557 raft_consensus.cc:2781] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:37.174504 5557 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:37.174595 5557 raft_consensus.cc:3037] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.174940 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9485300898e145ba9645d6e5ede8d7f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.175549 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9485300898e145ba9645d6e5ede8d7f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.138266 4169 rpcz_store.cc:276] 0811 18:57:34.293992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.297681 (+ 3689us) service_pool.cc:227] Handling call
0811 18:57:37.138054 (+2840373us) ts_tablet_manager.cc:1145] Deleting tablet 7be6d9578e7847c2822278486119390c
0811 18:57:37.138066 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.138173 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2840182}
I20260811 18:57:37.177310 5557 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.178560 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 55f97f37fa7c483cacaeed54f93e7a24 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.177913 5557 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.177678 4169 tablet_replica.cc:330] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.179741 4163 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.179890 4163 ts_tablet_manager.cc:1397] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.000s
I20260811 18:57:37.180387 4167 ts_tablet_manager.cc:1918] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.179792 4169 raft_consensus.cc:2229] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.180702 4163 raft_consensus.cc:348] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.180891 4163 raft_consensus.cc:374] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.180953 4163 raft_consensus.cc:729] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.181103 4169 pending_rounds.cc:62] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:37.181190 4169 pending_rounds.cc:68] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400179772579840 op_type: NO_OP noop_request { }
W20260811 18:57:37.178697 3824 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:37.181161 4163 consensus_queue.cc:260] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.181560 3824 leader_election.cc:341] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 968fa8b8912b40a8bdba77db1d39c5c0
W20260811 18:57:37.181744 3824 leader_election.cc:341] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:37.181900 3824 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.182271 4163 ts_tablet_manager.cc:1428] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.182667 4163 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.181406 4169 raft_consensus.cc:2866] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:37.184665 4169 raft_consensus.cc:2258] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.185238 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 764f87ff39df4706bc05e496fa2c7f29 (DEFAULT_TABLE table=default.Table_8edcf621b1064a75906a7ffe63e1a9d0 [id=673f9c8b623b4a8f8f34608f0487ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.185616 4163 tablet_bootstrap.cc:654] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.185717 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764f87ff39df4706bc05e496fa2c7f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.187810 3719 tablet_service.cc:1467] Processing CreateTablet for tablet dcad9c8052af4cbda4926c7c9430ea2a (DEFAULT_TABLE table=default.Table_d3410fcb02604cdabfae4dfa57a0bac0 [id=8489eaa51fe9406b94e1a7054e307bbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.188822 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcad9c8052af4cbda4926c7c9430ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.189055 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.189110 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 with delete_type TABLET_DATA_DELETED (replaced by tablet b192929b6d154885a0cc44c00f1add8e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.189370 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 7671139202af4a17976295f521828650 (DEFAULT_TABLE table=default.Table_15068805ee264778ac398274e74d630a [id=3744324baeba44dab08e66b32215651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.189402 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 31df6bcee1fb4324b20384bf3d3edee7 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190018 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc649a70b624d0cb6839f65000dacf9 (DEFAULT_TABLE table=default.Table_8752a36a67ad4fb380132326a813d172 [id=7259d5d4e7764c1b8d3a9d286f7bbd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190410 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 (DEFAULT_TABLE table=default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e [id=920721dee68b404bb09a2e453baec9c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190748 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 3cdaa303c70a44a19658664c8c9f9dd5 (DEFAULT_TABLE table=default.Table_ec98c10d285144029e63c99243d4d9fe [id=b73816870ec0459ca8092fcde1a27fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190662 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc84fd57013443ca0a0e7752a213566 (DEFAULT_TABLE table=default.Table_6f167c64b15d4d18a642c45b00f75658 [id=1a44fd5ad046463dbd4d84fb4e154005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190699 3716 tablet_service.cc:1467] Processing CreateTablet for tablet f10e19a2978a4a6799064cb36295c93c (DEFAULT_TABLE table=default.Table_7811427611da4cd1a27acc552c72c374 [id=b92c51f24dec492d8735bfb37ead3523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190699 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 (DEFAULT_TABLE table=default.Table_0084f6fe30164d4d9dcc8ad20699380e [id=48c5b345af51445eac883afc0ae3704f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190500 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1e182c4df64e7f9fb1ffb452046f7f (DEFAULT_TABLE table=default.Table_3f83f2ec5f2847c9820675f2f29f0674 [id=0428c361975e441a9d8a4e228aeb0a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.191093 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet d21973bc8e514a089c2beb24a686060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.190148 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 3c55a7b27133454881bcddc544c22faa (DEFAULT_TABLE table=default.Table_9c421939f1354150bddc1156b40cf5fe [id=5063c88916b845fcb99d66fd8a45f042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.191548 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31df6bcee1fb4324b20384bf3d3edee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.191514 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.191850 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6710084f21734dea9c7ce1f43d1bf587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.191990 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cdaa303c70a44a19658664c8c9f9dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192139 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc84fd57013443ca0a0e7752a213566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192466 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b66196716a4ba3893b6b8bfec2b8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192975 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10e19a2978a4a6799064cb36295c93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.193373 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc649a70b624d0cb6839f65000dacf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.194895 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdbe37b812846fdb089ec081000125c (DEFAULT_TABLE table=default.Table_9df97a2d74d040ef8f3dd56e41b8c28b [id=e2a3e504be604e95af18e61f23c07911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.196645 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 10bc2fcd64574327acf04347fa9ca0f4 (DEFAULT_TABLE table=default.Table_fa66e20bff3745ca961cfec0dc3a65c1 [id=f91b4d545c044e87a53411a16e80c42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.187983 4167 log.cc:1196] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cf6c8d542ae2489ba466ffadd518db12
I20260811 18:57:37.199440 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1e182c4df64e7f9fb1ffb452046f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.199635 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7671139202af4a17976295f521828650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.199833 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c55a7b27133454881bcddc544c22faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.200795 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 5143412d3c2349fcb97dc527891b49e4 (DEFAULT_TABLE table=default.Table_153744b340b44d8c9e939deedadaced9 [id=da2277b2b7e94f26a71d0e68075766d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.186627 5557 raft_consensus.cc:2781] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.201195 5557 raft_consensus.cc:686] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.201485 5557 consensus_queue.cc:237] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.204388 4167 ts_tablet_manager.cc:1939] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.204500 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.204784 3718 tablet_service.cc:1467] Processing CreateTablet for tablet b9f31daf440647949978feb54b97543d (DEFAULT_TABLE table=default.Table_767c174a258144229f562c9dda12392b [id=4ef58bf17e6b402b96fcd79052cd35dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.207845 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdbe37b812846fdb089ec081000125c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.186826 3824 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.210718 5511 raft_consensus.cc:2781] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.200917 4169 ts_tablet_manager.cc:1905] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.210883 5511 raft_consensus.cc:686] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.211114 5511 consensus_queue.cc:237] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.210601 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bc2fcd64574327acf04347fa9ca0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.226027 4162 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.226182 4162 ts_tablet_manager.cc:1397] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.155s user 0.004s sys 0.000s
I20260811 18:57:37.227031 4162 raft_consensus.cc:348] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.227252 4162 raft_consensus.cc:374] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.227305 4162 raft_consensus.cc:729] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.227531 4162 consensus_queue.cc:260] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.229245 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0790be2c84c14fb3aeb2add67667646e" } new_table_name: "default.Table_293a5368b1724dea8e5591b378e82995" modify_external_catalogs: true
I20260811 18:57:37.230702 4162 ts_tablet_manager.cc:1428] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260811 18:57:37.231689 3824 consensus_peers.cc:487] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b57ed7814a1e440cba362613cbab32f8. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.232072 3824 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.232327 5376 raft_consensus.cc:2781] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.239619 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
I20260811 18:57:37.246719 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5143412d3c2349fcb97dc527891b49e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.247170 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.248267 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.257578 4162 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.258368 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1020) took 2131 ms. Trace:
I20260811 18:57:37.258509 4167 rpcz_store.cc:276] 0811 18:57:35.126861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.126903 (+ 42us) service_pool.cc:227] Handling call
0811 18:57:37.041544 (+1914641us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:37.041558 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.107400 (+ 65842us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.180247 (+ 72847us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.258318 (+ 78071us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1914342}
I20260811 18:57:37.259001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1018) took 2132 ms. Trace:
I20260811 18:57:37.259109 4167 rpcz_store.cc:276] 0811 18:57:35.126705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.126774 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:37.258868 (+2132094us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:37.258878 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.258982 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2131338}
I20260811 18:57:37.259495 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1019) took 2132 ms. Trace:
I20260811 18:57:37.259593 4167 rpcz_store.cc:276] 0811 18:57:35.126797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.130400 (+ 3603us) service_pool.cc:227] Handling call
0811 18:57:37.259372 (+2128972us) ts_tablet_manager.cc:1145] Deleting tablet f86bfa65a8774b51bc7e7fe59442bf2f
0811 18:57:37.259385 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.259480 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2128775}
I20260811 18:57:37.259954 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1022) took 2018 ms. Trace:
I20260811 18:57:37.259986 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.260035 4167 rpcz_store.cc:276] 0811 18:57:35.241228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.241290 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:37.259813 (+2018523us) ts_tablet_manager.cc:1145] Deleting tablet f1342b8cd65b4c31a47b27a79c678d2b
0811 18:57:37.259827 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.259940 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2018287}
I20260811 18:57:37.260357 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1023) took 2018 ms. Trace:
I20260811 18:57:37.260443 4167 rpcz_store.cc:276] 0811 18:57:35.241879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.241932 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.260241 (+2018309us) ts_tablet_manager.cc:1145] Deleting tablet e5f47ccc193d48efa15d60a06b148e00
0811 18:57:37.260249 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.260342 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2018131}
I20260811 18:57:37.260802 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1024) took 2015 ms. Trace:
I20260811 18:57:37.260910 4167 rpcz_store.cc:276] 0811 18:57:35.245320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.245377 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:37.260681 (+2015304us) ts_tablet_manager.cc:1145] Deleting tablet fcb1c9787d3f4fd181f86deca28ad160
0811 18:57:37.260695 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.260786 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2015098}
I20260811 18:57:37.261183 4167 tablet_replica.cc:330] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.261392 4167 raft_consensus.cc:2229] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.261572 4167 raft_consensus.cc:2258] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.262480 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 20c4f222b5054527b18f2874dacc5558 (DEFAULT_TABLE table=default.Table_b807102dd88842e2b6e466a147c5b675 [id=c97451803afd4beab9583182c36fbac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.262948 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20c4f222b5054527b18f2874dacc5558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.265319 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a3973dfdaac746a6af7a402277d2711b
I20260811 18:57:37.266134 3547 catalog_manager.cc:6565] ReplaceTablet: tablet a3973dfdaac746a6af7a402277d2711b deleted and replaced by tablet 16db56987b2c44c9b5d3445073226ab9
I20260811 18:57:37.266388 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:37.266728 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_755a5ecf2290451b8befb8e19c0cda4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.266899 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b" } new_table_name: "default.Table_161c7e0df7b04150bf644353aac4604e" modify_external_catalogs: true
W20260811 18:57:37.267468 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_755a5ecf2290451b8befb8e19c0cda4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.272868 4167 ts_tablet_manager.cc:1905] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.276481 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f31daf440647949978feb54b97543d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.279004 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" } modify_external_catalogs: true
I20260811 18:57:37.279250 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" } modify_external_catalogs: true
W20260811 18:57:37.286058 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4f21761b37b42dbb814bd54cca89452 consensus state has no leader
I20260811 18:57:37.286651 5376 raft_consensus.cc:686] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.287096 5376 consensus_queue.cc:237] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.289307 4167 ts_tablet_manager.cc:1918] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:37.294714 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61f4112fcb7445b3803456e59a24ad0c consensus state has no leader
I20260811 18:57:37.296682 4167 log.cc:1196] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:37.303375 4167 ts_tablet_manager.cc:1939] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.305589 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1021) took 2065 ms. Trace:
I20260811 18:57:37.306528 4167 rpcz_store.cc:276] 0811 18:57:35.240249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.256817 (+ 16568us) service_pool.cc:227] Handling call
0811 18:57:37.261136 (+2004319us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:37.261150 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.276876 (+ 15726us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.289196 (+ 12320us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.305559 (+ 16363us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2004046}
I20260811 18:57:37.309882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1025) took 2017 ms. Trace:
I20260811 18:57:37.310055 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:37.311770 3860 tablet_service.cc:1467] Processing CreateTablet for tablet eb2ce551cae74423ba2a7b521fe1e44f (DEFAULT_TABLE table=default.Table_7249f865dd9d4e0fa204c278006d4cee [id=ce14e61175f7486aa71cabde45e4fc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.312248 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2ce551cae74423ba2a7b521fe1e44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.312963 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 15dd14e2b4bd4a08b5b08f532a55291b
I20260811 18:57:37.314217 4167 rpcz_store.cc:276] 0811 18:57:35.292084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.292497 (+ 413us) service_pool.cc:227] Handling call
0811 18:57:37.309717 (+2017220us) ts_tablet_manager.cc:1145] Deleting tablet 302cf69ea7c14fdaac72f6a9a9aff113
0811 18:57:37.309730 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.309860 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2017004}
I20260811 18:57:37.317173 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1026) took 1910 ms. Trace:
I20260811 18:57:37.317744 4167 rpcz_store.cc:276] 0811 18:57:35.406857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.406949 (+ 92us) service_pool.cc:227] Handling call
0811 18:57:37.316989 (+1910040us) ts_tablet_manager.cc:1145] Deleting tablet 36861ae786c549d687e687fe853d1801
0811 18:57:37.317003 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.317149 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1909778}
I20260811 18:57:37.319607 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 38f64f32463148349a2b0d403ae881e9 (DEFAULT_TABLE table=default.Table_cce74eefcdbb44369f1915d3f40f54e2 [id=43162c1c44f442fb912c654af76eac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.320168 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f64f32463148349a2b0d403ae881e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.325865 5593 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.326136 5593 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.327049 5593 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.327244 5596 tablet.cc:1722] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:37.328758 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c" } modify_external_catalogs: true
I20260811 18:57:37.328931 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 026245dfb0284d95a1afae43c77074c5 (DEFAULT_TABLE table=default.Table_def64923a6b24aa0a7b9ac4a66dcace5 [id=1e2eeed8d9aa4ce0935705782a7a7e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.328975 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c" } modify_external_catalogs: true
I20260811 18:57:37.329334 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026245dfb0284d95a1afae43c77074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.330786 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13786cfeafbc4312bf84f93b204add4f" } modify_external_catalogs: true
I20260811 18:57:37.331007 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13786cfeafbc4312bf84f93b204add4f" } modify_external_catalogs: true
I20260811 18:57:37.332078 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f5a969dbd34f414ba3a0b45748510632 (DEFAULT_TABLE table=default.Table_1ec52777b5484eb18531a70c114a32f6 [id=0d5ee633d4ab43a58bf9009459ee7699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.332500 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a969dbd34f414ba3a0b45748510632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.334934 4163 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.335067 4163 ts_tablet_manager.cc:1397] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.152s user 0.010s sys 0.011s
I20260811 18:57:37.335800 4163 raft_consensus.cc:348] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.336014 4163 raft_consensus.cc:374] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.336141 4163 raft_consensus.cc:729] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.336313 4163 consensus_queue.cc:260] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.336826 4163 ts_tablet_manager.cc:1428] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:37.337061 4163 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.338649 4163 tablet_bootstrap.cc:654] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.345635 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1027) took 1934 ms. Trace:
I20260811 18:57:37.345784 4167 rpcz_store.cc:276] 0811 18:57:35.410785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.410860 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:37.318586 (+1907726us) ts_tablet_manager.cc:1145] Deleting tablet 46505ba544104458888dbc2a37115dcc
0811 18:57:37.318600 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.345585 (+ 26985us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1907513}
I20260811 18:57:37.346241 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1028) took 1935 ms. Trace:
I20260811 18:57:37.346333 4167 rpcz_store.cc:276] 0811 18:57:35.410888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.410953 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:37.346094 (+1935141us) ts_tablet_manager.cc:1145] Deleting tablet 18c599d945044e0088bc66c8472d2495
0811 18:57:37.346108 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.346221 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1934938}
I20260811 18:57:37.346642 4167 tablet_replica.cc:330] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.346851 4167 raft_consensus.cc:2229] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.347007 4167 raft_consensus.cc:2258] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.347584 4167 ts_tablet_manager.cc:1905] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.350383 4163 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.350610 4163 ts_tablet_manager.cc:1397] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260811 18:57:37.351648 4163 raft_consensus.cc:348] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.351970 4163 raft_consensus.cc:374] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.352069 4163 raft_consensus.cc:729] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.352309 4163 consensus_queue.cc:260] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.353163 4163 ts_tablet_manager.cc:1428] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.353485 4163 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.355240 4162 tablet_bootstrap.cc:654] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.357286 5596 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37348 (request call id 1061) took 1029 ms. Trace:
I20260811 18:57:37.358695 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a385abaeb70347dcac06a5cedbdddeb4 (DEFAULT_TABLE table=default.Table_5437383c0bbb4384825f578e83c10acc [id=78cd09fbad264f7385b142dcf7ff57b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.359525 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a385abaeb70347dcac06a5cedbdddeb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.361946 4163 tablet_bootstrap.cc:654] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.359157 5596 rpcz_store.cc:276] 0811 18:57:36.327453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.327506 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.357261 (+1029755us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:36.327978 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:36.328082 (+ 104us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:36.328091 (+ 9us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:36.328098 (+ 7us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:36.328138 (+ 40us) alter_schema_op.cc:126] START. Timestamp: 1786474656328080
0811 18:57:36.328141 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:36.328328 (+ 187us) log.cc:842] Serialized 173 byte log entry
0811 18:57:37.263812 (+935484us) op_driver.cc:444] REPLICATION: finished
0811 18:57:37.295775 (+ 31963us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:37.328011 (+ 32236us) log.cc:842] Serialized 5 byte log entry
0811 18:57:37.333864 (+ 5853us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.334115 (+ 251us) log.cc:842] Serialized 14 byte log entry
0811 18:57:37.334165 (+ 50us) alter_schema_op.cc:71] Released schema lock
0811 18:57:37.334168 (+ 3us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":31897,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":8691,"prepare.run_wall_time_us":62140,"raft.queue_time_us":167,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":223,"replication_time_us":935639,"thread_start_us":93417,"threads_started":2,"wal-append.queue_time_us":59623}]]}
I20260811 18:57:37.366290 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914d7e775ebf4b26beed57cb21140166" } modify_external_catalogs: true
I20260811 18:57:37.366544 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914d7e775ebf4b26beed57cb21140166" } modify_external_catalogs: true
I20260811 18:57:37.366688 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d6113c39441d47a8a1d07df0fbd2371e" } new_table_name: "default.Table_037a37bbcc9044bdb397875f1abe75a0" modify_external_catalogs: true
I20260811 18:57:37.367110 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3b465c02cbcd4236a62ea1c357497b89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.367599 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b465c02cbcd4236a62ea1c357497b89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.368351 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 558dc0f1c46840d0991d08d792f903d5 (DEFAULT_TABLE table=default.Table_458dd09db5894e0fae4cc31deb113bdd [id=aa118bfdb3b440a1bc81ca195f425546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.368834 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558dc0f1c46840d0991d08d792f903d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.371512 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b908c7d655a467bbda1f9d2fdc1b99a" } new_table_name: "default.Table_fd1672ab9ee548b499043927ab259054" modify_external_catalogs: true
I20260811 18:57:37.374892 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe2f572c66cd4b9a8656e0fd439d50c7" } new_table_name: "default.Table_b67fc3dfee4e429c908d07053340f80c" modify_external_catalogs: true
I20260811 18:57:37.360247 4167 ts_tablet_manager.cc:1918] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.375891 4167 log.cc:1196] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:37.376502 4167 ts_tablet_manager.cc:1939] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.376912 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1029) took 1952 ms. Trace:
I20260811 18:57:37.377010 4167 rpcz_store.cc:276] 0811 18:57:35.424226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.424284 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:37.346595 (+1922311us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:37.346604 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.356860 (+ 10256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.360155 (+ 3295us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.376890 (+ 16735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1921323}
I20260811 18:57:37.377518 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1032) took 1884 ms. Trace:
I20260811 18:57:37.377614 4167 rpcz_store.cc:276] 0811 18:57:35.492951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.493260 (+ 309us) service_pool.cc:227] Handling call
0811 18:57:37.377351 (+1884091us) ts_tablet_manager.cc:1145] Deleting tablet 8eaafc7677344cac8d9616996c5f416c
0811 18:57:37.377364 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.377500 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1883850}
I20260811 18:57:37.378016 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1035) took 1829 ms. Trace:
I20260811 18:57:37.378125 4167 rpcz_store.cc:276] 0811 18:57:35.548826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.548885 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:37.377893 (+1829008us) ts_tablet_manager.cc:1145] Deleting tablet e2fb15b72c1d49d2ac4961bc04655a29
0811 18:57:37.377905 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.377996 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1828787}
I20260811 18:57:37.378607 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1036) took 1818 ms. Trace:
I20260811 18:57:37.378688 4167 rpcz_store.cc:276] 0811 18:57:35.560468 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.560534 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:37.378446 (+1817912us) ts_tablet_manager.cc:1145] Deleting tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
0811 18:57:37.378458 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.378589 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1817670}
I20260811 18:57:37.378921 4167 tablet_replica.cc:330] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.379145 4167 raft_consensus.cc:2229] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:37.379539 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d721b0e55384d8a8efd88a0e47b1504: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.379820 4167 raft_consensus.cc:2258] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.380239 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d" } modify_external_catalogs: true
I20260811 18:57:37.380455 4167 ts_tablet_manager.cc:1905] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.380445 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d" } modify_external_catalogs: true
I20260811 18:57:37.383245 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7fdacc021a974bd981dc30e4915378a3 (DEFAULT_TABLE table=default.Table_83c2a159da7a4474bd29c31afe70760e [id=110ac0c6ffd4495ea578641ae12779cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.383759 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fdacc021a974bd981dc30e4915378a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.386466 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet a3973dfdaac746a6af7a402277d2711b with delete_type TABLET_DATA_DELETED (replaced by tablet 16db56987b2c44c9b5d3445073226ab9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.387095 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.387487 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92a533c58b43c69fd65a1ec335cb3a (DEFAULT_TABLE table=default.Table_01425dcfb75d41f6ad6349c65957ebac [id=794cbc360aaf4b05a62e609896638744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.388913 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b92a533c58b43c69fd65a1ec335cb3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.392858 4167 ts_tablet_manager.cc:1918] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:37.395269 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5baa02ab900a4a8893c8e40d6ae5cf58 (DEFAULT_TABLE table=default.Table_7407d7434a3b4290889506a933ae2f05 [id=fe40a820d5cd401c8fc8393b02de9689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.395804 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5baa02ab900a4a8893c8e40d6ae5cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.396703 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0495c5b770b34d54afb1379a6e36204c
I20260811 18:57:37.405655 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 3edf59b74ee245be852ece5ba50a1054 (DEFAULT_TABLE table=default.Table_6aacb785fd444621820be42007d55668 [id=996a202868ee4f6dadf4be343005550a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.406075 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3edf59b74ee245be852ece5ba50a1054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.407898 4167 log.cc:1196] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:37.408384 4167 ts_tablet_manager.cc:1939] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.408710 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1037) took 1845 ms. Trace:
I20260811 18:57:37.408785 4167 rpcz_store.cc:276] 0811 18:57:35.563055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.563130 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:37.378879 (+1815749us) ts_tablet_manager.cc:1145] Deleting tablet b1ab2eee21264b1aa5ba00528cae3843
0811 18:57:37.378891 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.389558 (+ 10667us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.392776 (+ 3218us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.408691 (+ 15915us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1815552}
I20260811 18:57:37.410048 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 764f87ff39df4706bc05e496fa2c7f29 (DEFAULT_TABLE table=default.Table_8edcf621b1064a75906a7ffe63e1a9d0 [id=673f9c8b623b4a8f8f34608f0487ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.410486 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764f87ff39df4706bc05e496fa2c7f29. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.411727 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac48afc7eec4410e8d6055ecdb794a4f consensus state has no leader
I20260811 18:57:37.413708 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 75ceba7bfc8240268b16c789f07813d1
W20260811 18:57:37.414462 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 86a57b040ab44812a2001705493f2bcd consensus state has no leader
I20260811 18:57:37.414594 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5a75a480f38047d880b0c1fa51403786
I20260811 18:57:37.387914 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 5665e9b2813b4151981a7d487db9e7cd (DEFAULT_TABLE table=default.Table_9db0e2b871fd484f82e42660d2a09c55 [id=867e36c5f215455f808898586ca092f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.415015 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5665e9b2813b4151981a7d487db9e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.420342 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13af4d419ae542cc961c75f28efd0204 consensus state has no leader
I20260811 18:57:37.388209 3719 tablet_service.cc:1467] Processing CreateTablet for tablet ec4405eb41ff4a7484fcc24648f0e56a (DEFAULT_TABLE table=default.Table_28f0027f9b254195878661731f172f78 [id=6bd6c71ba5e94958a2f5758edfa35d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.421029 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec4405eb41ff4a7484fcc24648f0e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.387596 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf3d47a038f423190b343d94cd2694f (DEFAULT_TABLE table=default.Table_33c7b91319d04649821da27a7798271a [id=0a529f726a2a4bb297cdac716eccbc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.422641 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf3d47a038f423190b343d94cd2694f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.425943 3851 tablet_service.cc:1467] Processing CreateTablet for tablet dcad9c8052af4cbda4926c7c9430ea2a (DEFAULT_TABLE table=default.Table_d3410fcb02604cdabfae4dfa57a0bac0 [id=8489eaa51fe9406b94e1a7054e307bbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.426363 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcad9c8052af4cbda4926c7c9430ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.427591 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 28af2d2c61254b0ca399dddb05e9ee70 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.429241 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d32212c817c94fad8379462b027e7d3d
I20260811 18:57:37.433231 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b9f31daf440647949978feb54b97543d (DEFAULT_TABLE table=default.Table_767c174a258144229f562c9dda12392b [id=4ef58bf17e6b402b96fcd79052cd35dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.433553 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 with delete_type TABLET_DATA_DELETED (replaced by tablet b192929b6d154885a0cc44c00f1add8e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.433719 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f31daf440647949978feb54b97543d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.434408 4162 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.434543 4162 ts_tablet_manager.cc:1397] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.177s user 0.001s sys 0.003s
I20260811 18:57:37.435180 4162 raft_consensus.cc:348] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.435407 4162 raft_consensus.cc:374] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.435460 4162 raft_consensus.cc:729] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.436805 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.437371 4167 tablet_replica.cc:330] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.437563 4167 raft_consensus.cc:2229] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.437700 4167 raft_consensus.cc:2258] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.438210 4167 ts_tablet_manager.cc:1905] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.443656 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.444672 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdbe37b812846fdb089ec081000125c (DEFAULT_TABLE table=default.Table_9df97a2d74d040ef8f3dd56e41b8c28b [id=e2a3e504be604e95af18e61f23c07911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.445106 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdbe37b812846fdb089ec081000125c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.452689 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 7671139202af4a17976295f521828650 (DEFAULT_TABLE table=default.Table_15068805ee264778ac398274e74d630a [id=3744324baeba44dab08e66b32215651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.453200 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7671139202af4a17976295f521828650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.464879 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc649a70b624d0cb6839f65000dacf9 (DEFAULT_TABLE table=default.Table_8752a36a67ad4fb380132326a813d172 [id=7259d5d4e7764c1b8d3a9d286f7bbd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.465320 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc649a70b624d0cb6839f65000dacf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.467257 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1e182c4df64e7f9fb1ffb452046f7f (DEFAULT_TABLE table=default.Table_3f83f2ec5f2847c9820675f2f29f0674 [id=0428c361975e441a9d8a4e228aeb0a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.470081 4169 ts_tablet_manager.cc:1918] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:37.470202 4169 log.cc:1196] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:37.470646 4169 ts_tablet_manager.cc:1939] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.470997 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 5143412d3c2349fcb97dc527891b49e4 (DEFAULT_TABLE table=default.Table_153744b340b44d8c9e939deedadaced9 [id=da2277b2b7e94f26a71d0e68075766d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.471310 4162 consensus_queue.cc:260] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.475890 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cf6c8d542ae2489ba466ffadd518db12 (table default.Table_70493900fc9346389c1d4b71c611b962 [id=97d17499bd684f0abf98e0582fa06f5d]) successfully deleted
W20260811 18:57:37.476186 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d37e81d376e14907b13746cca831a96b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d37e81d376e14907b13746cca831a96b already in progress: creating tablet
W20260811 18:57:37.476392 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f86bfa65a8774b51bc7e7fe59442bf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f86bfa65a8774b51bc7e7fe59442bf2f already in progress: creating tablet
W20260811 18:57:37.476570 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1342b8cd65b4c31a47b27a79c678d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1342b8cd65b4c31a47b27a79c678d2b already in progress: creating tablet
W20260811 18:57:37.476739 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e5f47ccc193d48efa15d60a06b148e00 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f47ccc193d48efa15d60a06b148e00 already in progress: creating tablet
W20260811 18:57:37.476915 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fcb1c9787d3f4fd181f86deca28ad160 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb1c9787d3f4fd181f86deca28ad160 already in progress: creating tablet
I20260811 18:57:37.477432 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7b404e99e9414c67b3cd5aff12203a53 (table default.Table_d8397de033b74ce9accaa4085f59d7e6 [id=bebb3efbbc56416597abdf4a0b3952ed]) successfully deleted
I20260811 18:57:37.477542 4162 ts_tablet_manager.cc:1428] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
W20260811 18:57:37.477622 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 302cf69ea7c14fdaac72f6a9a9aff113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302cf69ea7c14fdaac72f6a9a9aff113 already in progress: creating tablet
W20260811 18:57:37.477800 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 36861ae786c549d687e687fe853d1801 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36861ae786c549d687e687fe853d1801 already in progress: creating tablet
I20260811 18:57:37.477864 4162 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.479489 4162 tablet_bootstrap.cc:654] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.480113 3855 tablet_service.cc:1467] Processing CreateTablet for tablet f10e19a2978a4a6799064cb36295c93c (DEFAULT_TABLE table=default.Table_7811427611da4cd1a27acc552c72c374 [id=b92c51f24dec492d8735bfb37ead3523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.483053 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88737be020144b20a2c4df22b9bf7b52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.483420 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ba12b590ed224215a8ea56022e39e6c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.483669 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88737be020144b20a2c4df22b9bf7b52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.483786 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba12b590ed224215a8ea56022e39e6c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.483003 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e407c33cb78457c831222ccfdc05d9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.484406 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e407c33cb78457c831222ccfdc05d9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.484359 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eb78d7140fc94f75b13bca56099bde32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.484778 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb78d7140fc94f75b13bca56099bde32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.485867 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.485874 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.485975 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc84fd57013443ca0a0e7752a213566 (DEFAULT_TABLE table=default.Table_6f167c64b15d4d18a642c45b00f75658 [id=1a44fd5ad046463dbd4d84fb4e154005]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.486280 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.486440 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet d21973bc8e514a089c2beb24a686060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486500 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 (DEFAULT_TABLE table=default.Table_0084f6fe30164d4d9dcc8ad20699380e [id=48c5b345af51445eac883afc0ae3704f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.486521 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486913 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet a3973dfdaac746a6af7a402277d2711b with delete_type TABLET_DATA_DELETED (replaced by tablet 16db56987b2c44c9b5d3445073226ab9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486927 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 3cdaa303c70a44a19658664c8c9f9dd5 (DEFAULT_TABLE table=default.Table_ec98c10d285144029e63c99243d4d9fe [id=b73816870ec0459ca8092fcde1a27fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.485531 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 (DEFAULT_TABLE table=default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e [id=920721dee68b404bb09a2e453baec9c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.487560 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 7fdacc021a974bd981dc30e4915378a3 (DEFAULT_TABLE table=default.Table_83c2a159da7a4474bd29c31afe70760e [id=110ac0c6ffd4495ea578641ae12779cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.488265 3862 tablet_service.cc:1467] Processing CreateTablet for tablet ec4405eb41ff4a7484fcc24648f0e56a (DEFAULT_TABLE table=default.Table_28f0027f9b254195878661731f172f78 [id=6bd6c71ba5e94958a2f5758edfa35d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.488867 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.489048 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92a533c58b43c69fd65a1ec335cb3a (DEFAULT_TABLE table=default.Table_01425dcfb75d41f6ad6349c65957ebac [id=794cbc360aaf4b05a62e609896638744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.489655 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf3d47a038f423190b343d94cd2694f (DEFAULT_TABLE table=default.Table_33c7b91319d04649821da27a7798271a [id=0a529f726a2a4bb297cdac716eccbc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.489794 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 5665e9b2813b4151981a7d487db9e7cd (DEFAULT_TABLE table=default.Table_9db0e2b871fd484f82e42660d2a09c55 [id=867e36c5f215455f808898586ca092f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.491252 3861 tablet_service.cc:1467] Processing CreateTablet for tablet ba29218db89a4814ac4e2f559884e34e (DEFAULT_TABLE table=default.Table_9a431bdf0c984b91935aed8346d63c56 [id=36e0f6431382481fae88e9cfec7f513c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.491591 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 79bf7d16b6024c9fb853c852d1c44f13 (DEFAULT_TABLE table=default.Table_5c323e78021f457095b0ad2754e21334 [id=dad48d97579f4912953013414d46c23b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.492729 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 31df6bcee1fb4324b20384bf3d3edee7 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.493160 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 3c55a7b27133454881bcddc544c22faa (DEFAULT_TABLE table=default.Table_9c421939f1354150bddc1156b40cf5fe [id=5063c88916b845fcb99d66fd8a45f042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.493858 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ef097030b2146bb9df7f7d2bffde6b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.494462 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ef097030b2146bb9df7f7d2bffde6b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.495373 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 10bc2fcd64574327acf04347fa9ca0f4 (DEFAULT_TABLE table=default.Table_fa66e20bff3745ca961cfec0dc3a65c1 [id=f91b4d545c044e87a53411a16e80c42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.495916 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9db0e2b871fd484f82e42660d2a09c55" } new_table_name: "default.Table_0ff3ccffaec842d2a6057786e0d521d3" modify_external_catalogs: true
W20260811 18:57:37.497931 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 46505ba544104458888dbc2a37115dcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46505ba544104458888dbc2a37115dcc already in progress: creating tablet
W20260811 18:57:37.498260 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 18c599d945044e0088bc66c8472d2495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18c599d945044e0088bc66c8472d2495 already in progress: creating tablet
I20260811 18:57:37.498605 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
W20260811 18:57:37.498787 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8eaafc7677344cac8d9616996c5f416c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8eaafc7677344cac8d9616996c5f416c already in progress: creating tablet
W20260811 18:57:37.498921 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e2fb15b72c1d49d2ac4961bc04655a29 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2fb15b72c1d49d2ac4961bc04655a29 already in progress: creating tablet
W20260811 18:57:37.499060 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae already in progress: creating tablet
I20260811 18:57:37.499442 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:37.500269 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_31dac67fa2c146b0956b5a3b25672da3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.500829 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31dac67fa2c146b0956b5a3b25672da3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.492370 3824 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.502231 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ba29218db89a4814ac4e2f559884e34e (DEFAULT_TABLE table=default.Table_9a431bdf0c984b91935aed8346d63c56 [id=36e0f6431382481fae88e9cfec7f513c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.502702 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba29218db89a4814ac4e2f559884e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.502848 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bc38e6760754449fbd6209d765cce70c (DEFAULT_TABLE table=default.Table_2ad9e3991d0f4274a4770a8f264f403e [id=9beaec986d304a5493a2d84fe239f803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.503153 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc38e6760754449fbd6209d765cce70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.503859 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 76a3c8bc00124a328d7eb80294e08b40 (DEFAULT_TABLE table=default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e [id=64199234b0924a10b5c42ad4d92c44ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.504217 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a3c8bc00124a328d7eb80294e08b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.506671 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b192929b6d154885a0cc44c00f1add8e (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.507191 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b192929b6d154885a0cc44c00f1add8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.507622 5557 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.507783 5557 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:37.508452 5557 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:37.501727 3824 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.508777 3824 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.508981 3824 leader_election.cc:334] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.509083 3824 leader_election.cc:302] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.510324 5557 raft_consensus.cc:2726] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.506696 3706 tablet_service.cc:1467] Processing CreateTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 (DEFAULT_TABLE table=default.Table_5ad53b06897e4559b15d34a4d98f4005 [id=99cf45a7b48c49b9b61196f14a84330e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.510649 3824 leader_election.cc:334] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.510751 3824 leader_election.cc:302] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.510852 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.513604 4167 ts_tablet_manager.cc:1918] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.513718 4167 log.cc:1196] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:37.514402 5557 raft_consensus.cc:2726] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.515077 3824 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.515305 3824 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.515396 3824 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.515340 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca13a41112144c79ff0612e9c9fd663 (DEFAULT_TABLE table=default.Table_9c767befd17049f8a118fdbd2a89e55d [id=57cfd7ce65924312b163d92e3418bc05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.515729 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca13a41112144c79ff0612e9c9fd663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.515756 4167 ts_tablet_manager.cc:1939] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.516467 5557 raft_consensus.cc:2726] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.518242 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.518514 3740 raft_consensus.cc:3037] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.519922 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 9bee89926dd74b54aaf40675bbae5d85 (DEFAULT_TABLE table=default.Table_8f2c4aa71e814a5dbc9684802806666e [id=df3f586e76a84ad2a47fb6a4d47ce274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.521392 3740 raft_consensus.cc:2454] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.529625 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f86907d0e6cf4cca9b263435070c705e (DEFAULT_TABLE table=default.Table_db7b48638c894fa5ade5118c99ed4f80 [id=681e2d3ad28d47978115767346af16bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.530161 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86907d0e6cf4cca9b263435070c705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.534713 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ceba7bfc8240268b16c789f07813d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.534817 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 0495c5b770b34d54afb1379a6e36204c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.535144 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a75a480f38047d880b0c1fa51403786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.538825 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_35bcb0c44a4840879a1f1d1386bf24c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.539628 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35bcb0c44a4840879a1f1d1386bf24c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.540887 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e" } new_table_name: "default.Table_d24b30f82e744f838b01bb0f37a99644" modify_external_catalogs: true
I20260811 18:57:37.541394 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 43ad6993b63e4bbd98c8cc52f1a53451 (table default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]) successfully deleted
W20260811 18:57:37.541653 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7be6d9578e7847c2822278486119390c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7be6d9578e7847c2822278486119390c already in progress: creating tablet
I20260811 18:57:37.542467 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet d32212c817c94fad8379462b027e7d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.544103 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1e182c4df64e7f9fb1ffb452046f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.544656 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6710084f21734dea9c7ce1f43d1bf587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.544881 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b92a533c58b43c69fd65a1ec335cb3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545094 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc84fd57013443ca0a0e7752a213566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545282 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fdacc021a974bd981dc30e4915378a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545526 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cdaa303c70a44a19658664c8c9f9dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545737 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec4405eb41ff4a7484fcc24648f0e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.549306 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf3d47a038f423190b343d94cd2694f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.552603 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bf7d16b6024c9fb853c852d1c44f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.556257 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba29218db89a4814ac4e2f559884e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.558210 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 79bf7d16b6024c9fb853c852d1c44f13 (DEFAULT_TABLE table=default.Table_5c323e78021f457095b0ad2754e21334 [id=dad48d97579f4912953013414d46c23b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.559547 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b66196716a4ba3893b6b8bfec2b8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.562565 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5665e9b2813b4151981a7d487db9e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.563179 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10e19a2978a4a6799064cb36295c93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.568455 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bf7d16b6024c9fb853c852d1c44f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.576347 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 067717418be34f7d9efe1c3be7f98830 (DEFAULT_TABLE table=default.Table_6c18f70552714e0f9a63822c3b6fb0a5 [id=203d5f31364f4e5bbc0413442f419953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.576834 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5143412d3c2349fcb97dc527891b49e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.580201 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bee89926dd74b54aaf40675bbae5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.583666 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c55a7b27133454881bcddc544c22faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.584062 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bc2fcd64574327acf04347fa9ca0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.585193 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9bee89926dd74b54aaf40675bbae5d85 (DEFAULT_TABLE table=default.Table_8f2c4aa71e814a5dbc9684802806666e [id=df3f586e76a84ad2a47fb6a4d47ce274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.585722 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bee89926dd74b54aaf40675bbae5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.586474 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31df6bcee1fb4324b20384bf3d3edee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.590724 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 067717418be34f7d9efe1c3be7f98830 (DEFAULT_TABLE table=default.Table_6c18f70552714e0f9a63822c3b6fb0a5 [id=203d5f31364f4e5bbc0413442f419953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.590894 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067717418be34f7d9efe1c3be7f98830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.591174 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067717418be34f7d9efe1c3be7f98830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.596189 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: OK, instance_id { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" instance_seqno: 1786474650422925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:37.596462 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:37.602156 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.604092 5611 tablet.cc:1722] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:37.605310 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1040) took 2001 ms. Trace:
I20260811 18:57:37.606423 4167 rpcz_store.cc:276] 0811 18:57:35.603688 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.603757 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:37.437316 (+1833559us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:37.437329 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.440985 (+ 3656us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.513521 (+ 72536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.605276 (+ 91755us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1833270}
I20260811 18:57:37.607419 4162 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.612739 4162 ts_tablet_manager.cc:1397] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.135s user 0.004s sys 0.000s
I20260811 18:57:37.613596 4162 raft_consensus.cc:348] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.613945 4162 raft_consensus.cc:374] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.614056 4162 raft_consensus.cc:729] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.614305 4162 consensus_queue.cc:260] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.615134 4162 ts_tablet_manager.cc:1428] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:37.615458 4162 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.617005 4162 tablet_bootstrap.cc:654] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.617254 4167 tablet_replica.cc:330] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:37.621827 3824 consensus_peers.cc:487] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.623472 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 951) took 3322 ms. Trace:
I20260811 18:57:37.623575 4169 rpcz_store.cc:276] 0811 18:57:34.300739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.300963 (+ 224us) service_pool.cc:227] Handling call
0811 18:57:37.177631 (+2876668us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:37.177643 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.224190 (+ 46547us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.470007 (+245817us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.623452 (+153445us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14483,"tablet-delete.queue_time_us":2876413}
I20260811 18:57:37.623931 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 949) took 3329 ms. Trace:
I20260811 18:57:37.624002 4169 rpcz_store.cc:276] 0811 18:57:34.295383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.307452 (+ 12069us) service_pool.cc:227] Handling call
0811 18:57:37.623839 (+3316387us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:37.623850 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.623916 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3316091}
I20260811 18:57:37.624248 4169 tablet_replica.cc:330] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.624444 4169 raft_consensus.cc:2229] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.624593 4169 raft_consensus.cc:2258] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.625113 4169 ts_tablet_manager.cc:1905] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.631210 4167 raft_consensus.cc:2229] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.631561 4167 raft_consensus.cc:2258] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.632315 4167 ts_tablet_manager.cc:1905] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.635433 5407 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.635610 5407 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.636333 5407 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.637246 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.638396 3676 leader_election.cc:341] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:37.649623 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 76a3c8bc00124a328d7eb80294e08b40 (DEFAULT_TABLE table=default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e [id=64199234b0924a10b5c42ad4d92c44ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.650111 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a3c8bc00124a328d7eb80294e08b40. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.651126 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76a3c8bc00124a328d7eb80294e08b40 consensus state has no leader
I20260811 18:57:37.653438 3555 catalog_manager.cc:5495] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.653848 5557 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.653981 5557 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.654616 5557 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.656323 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:37.657521 4169 ts_tablet_manager.cc:1918] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:37.659471 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5665e9b2813b4151981a7d487db9e7cd consensus state has no leader
I20260811 18:57:37.659896 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c925e9ac591346b1a5142f338c68bcc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.660514 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c925e9ac591346b1a5142f338c68bcc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.662737 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" } modify_external_catalogs: true
I20260811 18:57:37.662989 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" } modify_external_catalogs: true
I20260811 18:57:37.654117 3555 catalog_manager.cc:5495] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.670305 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:37.670624 4169 log.cc:1196] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:37.671171 4169 ts_tablet_manager.cc:1939] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.671609 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 952) took 3335 ms. Trace:
I20260811 18:57:37.671766 4169 rpcz_store.cc:276] 0811 18:57:34.335899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.335958 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:37.624200 (+3288242us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:37.624212 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.653292 (+ 29080us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.657412 (+ 4120us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.671589 (+ 14177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3272828}
I20260811 18:57:37.672248 4169 tablet_replica.cc:330] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.672535 4169 raft_consensus.cc:2229] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.672706 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" } modify_external_catalogs: true
I20260811 18:57:37.660971 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca13a41112144c79ff0612e9c9fd663 (DEFAULT_TABLE table=default.Table_9c767befd17049f8a118fdbd2a89e55d [id=57cfd7ce65924312b163d92e3418bc05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.673265 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca13a41112144c79ff0612e9c9fd663. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.645305 3678 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.674422 3678 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.674968 5407 raft_consensus.cc:2726] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.663993 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f86907d0e6cf4cca9b263435070c705e (DEFAULT_TABLE table=default.Table_db7b48638c894fa5ade5118c99ed4f80 [id=681e2d3ad28d47978115767346af16bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.678735 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86907d0e6cf4cca9b263435070c705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.683063 5557 raft_consensus.cc:482] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.683247 5557 raft_consensus.cc:504] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.683867 5557 leader_election.cc:288] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:37.684128 3824 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:37.685956 3824 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.686197 5557 raft_consensus.cc:2781] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.686353 5557 raft_consensus.cc:686] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.686570 5557 consensus_queue.cc:237] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.689379 4162 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.689788 4162 ts_tablet_manager.cc:1397] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.074s user 0.003s sys 0.000s
I20260811 18:57:37.690651 4162 raft_consensus.cc:348] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.690992 4162 raft_consensus.cc:374] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.691118 4162 raft_consensus.cc:729] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.691401 4162 consensus_queue.cc:260] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.692291 4162 ts_tablet_manager.cc:1428] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.692659 4162 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.695223 4162 tablet_bootstrap.cc:654] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.695581 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" } modify_external_catalogs: true
I20260811 18:57:37.699676 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_52e514412ff342e3b34ba958e863d4ba" } modify_external_catalogs: true
I20260811 18:57:37.700947 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_52e514412ff342e3b34ba958e863d4ba" } modify_external_catalogs: true
I20260811 18:57:37.700181 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24c4140a00244c8d8446d91705a8d4a8" } modify_external_catalogs: true
I20260811 18:57:37.702235 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24c4140a00244c8d8446d91705a8d4a8" } modify_external_catalogs: true
I20260811 18:57:37.700474 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_75b5605dc2a14c9daf6d68e7094bcb39" } new_table_name: "default.Table_b2e37753037145439b8e47e544940455" modify_external_catalogs: true
I20260811 18:57:37.705478 4162 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.705730 4162 ts_tablet_manager.cc:1397] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260811 18:57:37.706557 4162 raft_consensus.cc:348] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.706933 4162 raft_consensus.cc:374] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.707063 4162 raft_consensus.cc:729] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.707374 4162 consensus_queue.cc:260] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.708312 4162 ts_tablet_manager.cc:1428] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.708673 4162 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.700791 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8076b90bff8d49c08328583af352ae7c" } new_table_name: "default.Table_440c0b981c8a42c28bf5f00f50ad7296" modify_external_catalogs: true
I20260811 18:57:37.710312 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e8780219db840aeb566b570452e221b" } new_table_name: "default.Table_b373e0051180486da1efff0b123d200e" modify_external_catalogs: true
I20260811 18:57:37.710529 4162 tablet_bootstrap.cc:654] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.716399 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
W20260811 18:57:37.717265 3824 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:37.717851 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3195082643264e0d90f70e1007ab9b24
W20260811 18:57:37.717841 3824 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 059b325df1db4fde9d4b26b559121316
I20260811 18:57:37.718505 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" } modify_external_catalogs: true
I20260811 18:57:37.718767 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" } modify_external_catalogs: true
I20260811 18:57:37.719408 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d495354c0994fb4a74c376078e41a26" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:37.719868 4169 pending_rounds.cc:62] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:37.719985 4169 pending_rounds.cc:68] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400181036277760 op_type: NO_OP noop_request { }
I20260811 18:57:37.720201 4169 raft_consensus.cc:2866] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:37.720392 4169 raft_consensus.cc:2258] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.721150 4169 ts_tablet_manager.cc:1905] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.727438 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d2326f1112004b75bdb9d55298314884"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.728170 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2326f1112004b75bdb9d55298314884 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.732092 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 221d0d8d69f142a29ec7065a82b3f769 (table default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]) successfully deleted
W20260811 18:57:37.732369 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 490f34d7a403416fbab64e46973746de because the tablet was not found. No further retry: Not found: Tablet not found: 490f34d7a403416fbab64e46973746de
I20260811 18:57:37.740178 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.753002 3871 tablet_service.cc:1467] Processing CreateTablet for tablet bc38e6760754449fbd6209d765cce70c (DEFAULT_TABLE table=default.Table_2ad9e3991d0f4274a4770a8f264f403e [id=9beaec986d304a5493a2d84fe239f803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.755726 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cbf6ce167fb348be8d61372791f0cbca (table default.Table_1558f156b5eb46a1858ca7673e4a9747 [id=dda7fb95dc1549ddbc88d8525ff023e8]) successfully deleted
I20260811 18:57:37.759416 4167 ts_tablet_manager.cc:1918] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.760648 4167 log.cc:1196] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5d495354c0994fb4a74c376078e41a26
I20260811 18:57:37.764215 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b7b38d3b9570467a9128bd7fe73acf2f
I20260811 18:57:37.764810 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b4de110bd3348f0b0dae21c9f2b6a19 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.765595 4167 ts_tablet_manager.cc:1939] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.767083 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1049) took 2105 ms. Trace:
I20260811 18:57:37.767607 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc38e6760754449fbd6209d765cce70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.767798 4167 rpcz_store.cc:276] 0811 18:57:35.661443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.661634 (+ 191us) service_pool.cc:227] Handling call
0811 18:57:37.617203 (+1955569us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:37.617215 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.645066 (+ 27851us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.759272 (+114206us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.767059 (+ 7787us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9606,"tablet-delete.queue_time_us":1955407}
I20260811 18:57:37.769824 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 429eb5824f3f4b2b8da6e17ba7c69b92
I20260811 18:57:37.758180 3852 tablet_service.cc:1467] Processing CreateTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 (DEFAULT_TABLE table=default.Table_5ad53b06897e4559b15d34a4d98f4005 [id=99cf45a7b48c49b9b61196f14a84330e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.760504 4163 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.773669 4163 ts_tablet_manager.cc:1397] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.420s user 0.004s sys 0.007s
I20260811 18:57:37.775707 4163 raft_consensus.cc:348] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.776067 4163 raft_consensus.cc:374] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.776254 4163 raft_consensus.cc:729] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.776602 4163 consensus_queue.cc:260] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.777442 4163 ts_tablet_manager.cc:1428] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:37.777813 4163 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.780731 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 0495c5b770b34d54afb1379a6e36204c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.787012 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1057) took 1967 ms. Trace:
I20260811 18:57:37.787123 4167 rpcz_store.cc:276] 0811 18:57:35.819520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.819590 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:37.786849 (+1967259us) ts_tablet_manager.cc:1145] Deleting tablet 46a15e7a0c0f4283ac5e575d3c15c9da
0811 18:57:37.786863 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.786987 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1967018}
I20260811 18:57:37.787484 4167 tablet_replica.cc:330] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.787703 4167 raft_consensus.cc:2229] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.788089 4167 raft_consensus.cc:2258] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.788708 4167 ts_tablet_manager.cc:1905] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.794013 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f89eebbc40234794a081734f834294ab
I20260811 18:57:37.768417 4162 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.796281 4162 ts_tablet_manager.cc:1397] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.088s user 0.000s sys 0.009s
I20260811 18:57:37.797200 4162 raft_consensus.cc:348] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.797539 4162 raft_consensus.cc:374] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.797665 4162 raft_consensus.cc:729] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.797976 4162 consensus_queue.cc:260] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.798754 4162 ts_tablet_manager.cc:1428] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:37.804453 5407 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.804620 5407 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.805269 5407 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.805840 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c99ceca57cb245f4a3a6cb4b36f1306d (table default.Table_8a27cf06565547a39a8eee2420c9a31b [id=0a4e4826c2cf4250a34e7e26eece8e3a]) successfully deleted
I20260811 18:57:37.806586 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:37.807022 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5d495354c0994fb4a74c376078e41a26 (table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]) successfully deleted
W20260811 18:57:37.807201 3676 leader_election.cc:341] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:37.807292 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 46a15e7a0c0f4283ac5e575d3c15c9da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46a15e7a0c0f4283ac5e575d3c15c9da already in progress: creating tablet
I20260811 18:57:37.809242 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 97e6e6b26a11432eb739e4219d062f7d (DEFAULT_TABLE table=default.Table_1ec50daf17974aef9e292bab9fc018cd [id=c571e0e9444e48859e1ccaf8b4abc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.809825 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 275ec54207504e9f8267925e2224b2ab (DEFAULT_TABLE table=default.Table_1365655a76d7454c8415102e75960152 [id=d634ab6079f74d08ac11be356a004ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.810479 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 961f8adc3ef24257b669288c44fe2511 (DEFAULT_TABLE table=default.Table_fb6981748ef0423aaaa9a85bacef95be [id=b5aed8fa1431445da78c1e9cfb7a19de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.811491 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed4bc2ff0d743bbbe8383d235e1f398 (DEFAULT_TABLE table=default.Table_9485300898e145ba9645d6e5ede8d7f1 [id=089de6d15c3245d1a9af59a8444fbeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.811025 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 87c390ea36844d91a115a4c6a64e6563 (DEFAULT_TABLE table=default.Table_2c977f4c94664fcb88bb338b5c79edfe [id=7d23ddd1558d48488f96456d33f7e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812178 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7 (DEFAULT_TABLE table=default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 [id=9a0911a26573434cb56ca228b312c33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812584 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1932990f8a014b74999971d07c1cbe84 (DEFAULT_TABLE table=default.Table_606ca53b362a44b4ba72954536b85633 [id=b620868692c840f58e1a970a496e8a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.813021 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e883852aaf6c41c79e87e4286da16a27 (DEFAULT_TABLE table=default.Table_755a5ecf2290451b8befb8e19c0cda4b [id=94f0c49c6b794f85b2a460c8789c6c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812584 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 03109d96439141228d8c26b491e2dd8c (DEFAULT_TABLE table=default.Table_c364560f47804860afd5635ea9b6885d [id=30094b497eb9441caec9b0296d80858d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.814970 3678 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.815114 3678 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.815631 5407 raft_consensus.cc:2726] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.816679 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 16db56987b2c44c9b5d3445073226ab9 (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.817122 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 75daed89122c406caa4bd9bbaa67d9b8 (DEFAULT_TABLE table=default.Table_faccbd3f194043708dbbd12a50f037cf [id=14137b353d6b409ab3461d37309cdff7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.754408 3824 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.784461 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ceba7bfc8240268b16c789f07813d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.785375 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a75a480f38047d880b0c1fa51403786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.817670 3824 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.818017 5557 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.818379 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 63979.2us
W20260811 18:57:37.818610 3824 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.818801 3824 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.819072 5557 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.820184 3824 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.820374 3824 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.820624 5557 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.820962 3824 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.824334 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 75daed89122c406caa4bd9bbaa67d9b8 (DEFAULT_TABLE table=default.Table_faccbd3f194043708dbbd12a50f037cf [id=14137b353d6b409ab3461d37309cdff7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.830133 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
W20260811 18:57:37.830719 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a135bd92ab5b4f76bad9b3eedcfada29 consensus state has no leader
I20260811 18:57:37.831346 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3195082643264e0d90f70e1007ab9b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.765017 3860 tablet_service.cc:1467] Processing CreateTablet for tablet b192929b6d154885a0cc44c00f1add8e (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.786206 4163 tablet_bootstrap.cc:654] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.787927 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet d32212c817c94fad8379462b027e7d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.824334 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 87c390ea36844d91a115a4c6a64e6563 (DEFAULT_TABLE table=default.Table_2c977f4c94664fcb88bb338b5c79edfe [id=7d23ddd1558d48488f96456d33f7e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.786612 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.835459 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75daed89122c406caa4bd9bbaa67d9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.838652 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b192929b6d154885a0cc44c00f1add8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.839010 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a5bbd3274980470b9fe71457b49bdd62
W20260811 18:57:37.839672 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:37.846837 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c390ea36844d91a115a4c6a64e6563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.854897 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 16db56987b2c44c9b5d3445073226ab9 (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.855742 3859 tablet_service.cc:1467] Processing CreateTablet for tablet e883852aaf6c41c79e87e4286da16a27 (DEFAULT_TABLE table=default.Table_755a5ecf2290451b8befb8e19c0cda4b [id=94f0c49c6b794f85b2a460c8789c6c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.856423 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 03109d96439141228d8c26b491e2dd8c (DEFAULT_TABLE table=default.Table_c364560f47804860afd5635ea9b6885d [id=30094b497eb9441caec9b0296d80858d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.856845 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03109d96439141228d8c26b491e2dd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.860293 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b67fc3dfee4e429c908d07053340f80c" } new_table_name: "default.Table_543e49bc9d8a40e8a06df8d7f48479b7" modify_external_catalogs: true
I20260811 18:57:37.862666 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.863265 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5dde73e565645cfbaf60a60e2d0e71a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.865164 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 3a9efd2f19d74a68a8536fd057377738 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.865245 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9a431bdf0c984b91935aed8346d63c56" } new_table_name: "default.Table_f0da3b5bd2b14fd3961e325d592c86b6" modify_external_catalogs: true
I20260811 18:57:37.865867 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bb1820be09714d0b976eda8e0151671e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.866611 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb1820be09714d0b976eda8e0151671e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.869143 5557 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.869350 5557 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.870122 5557 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.871446 5407 raft_consensus.cc:482] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.871598 5407 raft_consensus.cc:504] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.872257 5407 leader_election.cc:288] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.873103 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37e81d376e14907b13746cca831a96b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.873828 3676 leader_election.cc:341] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:37.875228 3678 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.875416 3678 leader_election.cc:302] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.875701 5407 raft_consensus.cc:2726] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.878239 4169 ts_tablet_manager.cc:1918] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:37.878458 4169 log.cc:1196] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6f3060822021431584a494e6941be756
I20260811 18:57:37.878984 4169 ts_tablet_manager.cc:1939] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.879438 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 953) took 3543 ms. Trace:
W20260811 18:57:37.884646 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13af4d419ae542cc961c75f28efd0204 consensus state has no leader
I20260811 18:57:37.894330 4162 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.895992 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c6490a83dec455bac3fe91b50da8a95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.896322 4162 tablet_bootstrap.cc:654] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.896727 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c6490a83dec455bac3fe91b50da8a95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.898893 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e6e6b26a11432eb739e4219d062f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.902622 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275ec54207504e9f8267925e2224b2ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.904897 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_908f52c0075e4f6c8f60b6c84f13e004"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.905141 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b882f1fc510a41558369a5ced806d29a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.905077 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5dc02b59b48d45c8bee6458f21962eeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.905547 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b882f1fc510a41558369a5ced806d29a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.905632 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5dc02b59b48d45c8bee6458f21962eeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.905552 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_908f52c0075e4f6c8f60b6c84f13e004 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.906611 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961f8adc3ef24257b669288c44fe2511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.911015 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed4bc2ff0d743bbbe8383d235e1f398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.911329 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.912690 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba29218db89a4814ac4e2f559884e34e consensus state has no leader
I20260811 18:57:37.914598 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1932990f8a014b74999971d07c1cbe84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.918509 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c390ea36844d91a115a4c6a64e6563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.879951 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7 (DEFAULT_TABLE table=default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 [id=9a0911a26573434cb56ca228b312c33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.880749 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 275ec54207504e9f8267925e2224b2ab (DEFAULT_TABLE table=default.Table_1365655a76d7454c8415102e75960152 [id=d634ab6079f74d08ac11be356a004ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.880749 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 961f8adc3ef24257b669288c44fe2511 (DEFAULT_TABLE table=default.Table_fb6981748ef0423aaaa9a85bacef95be [id=b5aed8fa1431445da78c1e9cfb7a19de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.881287 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 97e6e6b26a11432eb739e4219d062f7d (DEFAULT_TABLE table=default.Table_1ec50daf17974aef9e292bab9fc018cd [id=c571e0e9444e48859e1ccaf8b4abc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.882372 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 3195082643264e0d90f70e1007ab9b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.882671 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.882946 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883167 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883442 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet f89eebbc40234794a081734f834294ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883607 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bbd3274980470b9fe71457b49bdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.922127 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e" } modify_external_catalogs: true
I20260811 18:57:37.922406 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e" } modify_external_catalogs: true
I20260811 18:57:37.923509 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e883852aaf6c41c79e87e4286da16a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.923735 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03109d96439141228d8c26b491e2dd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.928686 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16db56987b2c44c9b5d3445073226ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.930639 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75daed89122c406caa4bd9bbaa67d9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.942574 4162 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.942759 4162 ts_tablet_manager.cc:1397] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.001s
I20260811 18:57:37.943439 4162 raft_consensus.cc:348] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.943674 4162 raft_consensus.cc:374] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.943738 4162 raft_consensus.cc:729] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.881287 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed4bc2ff0d743bbbe8383d235e1f398 (DEFAULT_TABLE table=default.Table_9485300898e145ba9645d6e5ede8d7f1 [id=089de6d15c3245d1a9af59a8444fbeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.943923 4162 consensus_queue.cc:260] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.944489 4162 ts_tablet_manager.cc:1428] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:37.944789 4162 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.946717 4162 tablet_bootstrap.cc:654] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.956127 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet b4c7f500a5444bae9a015a5e903c0622 on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:37.956415 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 65234acc3d5a462683edbe8f223eeefc on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:37.957391 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 476da7e2681540a89824b58fb7237df9 (table default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 476da7e2681540a89824b58fb7237df9 consensus state has no leader
I20260811 18:57:37.892078 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 1932990f8a014b74999971d07c1cbe84 (DEFAULT_TABLE table=default.Table_606ca53b362a44b4ba72954536b85633 [id=b620868692c840f58e1a970a496e8a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.897558 4163 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.960824 4163 ts_tablet_manager.cc:1397] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.183s user 0.004s sys 0.000s
I20260811 18:57:37.961752 4163 raft_consensus.cc:348] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.962019 4163 raft_consensus.cc:374] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.962080 4163 raft_consensus.cc:729] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.961826 3555 catalog_manager.cc:5495] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.962313 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:37.962280 4163 consensus_queue.cc:260] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.962630 3555 catalog_manager.cc:5495] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.962924 4163 ts_tablet_manager.cc:1428] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.963027 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
I20260811 18:57:37.963243 4163 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.906963 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16db56987b2c44c9b5d3445073226ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.963547 3552 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca] alter complete (version 1)
I20260811 18:57:37.969794 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961f8adc3ef24257b669288c44fe2511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971791 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed4bc2ff0d743bbbe8383d235e1f398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971654 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_149093c5dbb2402ebc6afe1a069fb195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.971933 5593 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.972062 5593 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:37.972276 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_149093c5dbb2402ebc6afe1a069fb195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.972647 5593 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.973845 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e6e6b26a11432eb739e4219d062f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971670 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb" } modify_external_catalogs: true
I20260811 18:57:37.974563 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb" } modify_external_catalogs: true
I20260811 18:57:37.976344 4163 tablet_bootstrap.cc:654] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.976713 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.976948 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet f89eebbc40234794a081734f834294ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.977135 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e883852aaf6c41c79e87e4286da16a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.982489 5407 raft_consensus.cc:482] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.982656 5407 raft_consensus.cc:504] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.983392 5407 leader_election.cc:288] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.984050 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8f5b87062349c4b19ec99fa0175f24" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.984668 3676 leader_election.cc:341] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:37.989032 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.995959 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275ec54207504e9f8267925e2224b2ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.999301 3678 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.999449 3678 leader_election.cc:302] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.999776 5407 raft_consensus.cc:2726] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.002820 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1932990f8a014b74999971d07c1cbe84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.007182 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bbd3274980470b9fe71457b49bdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.010516 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.011792 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6710084f21734dea9c7ce1f43d1bf587
I20260811 18:57:38.012734 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e9f0e443ab84424dac307b745b496986
I20260811 18:57:38.013394 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" } modify_external_catalogs: true
I20260811 18:57:38.013612 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" } modify_external_catalogs: true
I20260811 18:57:38.015297 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_788572afacab482590222fc4d084ef0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.015763 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_614252f40e34465080bb490a1a7050ed" } modify_external_catalogs: true
W20260811 18:57:38.015900 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_788572afacab482590222fc4d084ef0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.015928 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_614252f40e34465080bb490a1a7050ed" } modify_external_catalogs: true
I20260811 18:57:38.017884 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fe6c2229e23048eb81c61314102eab38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.018476 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe6c2229e23048eb81c61314102eab38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.921648 4169 rpcz_store.cc:276] 0811 18:57:34.336043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336162 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:37.672190 (+3336028us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:37.672211 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.724559 (+ 52348us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.878120 (+153561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.879420 (+ 1300us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3320668}
I20260811 18:57:38.026991 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 954) took 3690 ms. Trace:
I20260811 18:57:38.027138 4169 rpcz_store.cc:276] 0811 18:57:34.336181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336330 (+ 149us) service_pool.cc:227] Handling call
0811 18:57:38.026786 (+3690456us) ts_tablet_manager.cc:1145] Deleting tablet dd80633f926942e88f54b32d9a03ea3a
0811 18:57:38.026802 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.026962 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675076}
I20260811 18:57:38.027465 4169 tablet_replica.cc:330] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.027691 4169 raft_consensus.cc:2229] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.027870 4169 raft_consensus.cc:2258] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:37.913493 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.032150 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bfea0cd619024204b9bff1e2dd62b7b0
W20260811 18:57:38.032105 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 118631us
I20260811 18:57:38.033958 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 69059a0667234a408cc1c09585df63bc
I20260811 18:57:38.036274 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e407c33cb78457c831222ccfdc05d9d" } new_table_name: "default.Table_cb4ec5ebc85e4562a65d10436234b4c2" modify_external_catalogs: true
I20260811 18:57:38.036890 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b373e0051180486da1efff0b123d200e" } new_table_name: "default.Table_144f54709c324231aab49a10d3daab83" modify_external_catalogs: true
I20260811 18:57:38.041230 5407 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.041393 5407 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.041949 5407 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.043035 3678 consensus_peers.cc:487] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.043247 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be8ef75ca590418486730a869708b637" } new_table_name: "default.Table_f747b7645d4643a5bd6215c8ff94bb1e" modify_external_catalogs: true
I20260811 18:57:38.043084 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:38.043507 3888 raft_consensus.cc:2454] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:38.043344 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1afb343a7aa84568ad8ad65cab128123"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.044051 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1afb343a7aa84568ad8ad65cab128123 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.044016 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.044296 5407 raft_consensus.cc:2781] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.044363 5407 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.044415 5407 raft_consensus.cc:3037] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.048430 5407 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.049606 5407 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.051025 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.051859 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:38.052096 3888 raft_consensus.cc:3037] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:38.053393 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.054664 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a" } modify_external_catalogs: true
I20260811 18:57:38.054883 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a" } modify_external_catalogs: true
I20260811 18:57:38.056417 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.056757 3745 raft_consensus.cc:2379] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
I20260811 18:57:38.064256 4169 ts_tablet_manager.cc:1905] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.066097 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 82b04e37fc264fc08e0ed9517356a17d from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:38.068427 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae3750e5870411e8fe7dedae2fe3ec0 consensus state has no leader
W20260811 18:57:38.069460 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b48a82a7182b4bf89fb9fd3830829909 has no consensus state
W20260811 18:57:38.070034 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:38.079357 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 82b04e37fc264fc08e0ed9517356a17d
I20260811 18:57:38.079733 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 8ab3b72a62db4781806808f3af2ce46e
I20260811 18:57:38.079933 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 82b04e37fc264fc08e0ed9517356a17d deleted and replaced by tablet 94857ef729eb435c97d089b8ab108cea
I20260811 18:57:38.087909 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bfaa8ab4fcd40efb60e1718e424bccf" } new_table_name: "default.Table_8b0f40c58a1c4f028c865253aba93296" modify_external_catalogs: true
I20260811 18:57:38.088680 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.089140 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e6562b8db6d54b42a89d19df04cab07c (DEFAULT_TABLE table=default.Table_88737be020144b20a2c4df22b9bf7b52 [id=2ef647fe25634721afac1f7c2d4a359a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.089584 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6562b8db6d54b42a89d19df04cab07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.093039 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet d11b889e9d044a049d0586382c263951 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.072790 3888 raft_consensus.cc:2454] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:38.102594 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.103048 5407 raft_consensus.cc:2781] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:38.101482 3824 leader_election.cc:341] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: Tablet not RUNNING: STOPPED
W20260811 18:57:38.104046 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b540ae350254346a675b54118c5cfcb consensus state has no leader
I20260811 18:57:38.103906 3824 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.107209 4163 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.107493 4163 ts_tablet_manager.cc:1397] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.144s user 0.006s sys 0.000s
I20260811 18:57:38.110123 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d11b889e9d044a049d0586382c263951
I20260811 18:57:38.110711 3555 catalog_manager.cc:6565] ReplaceTablet: tablet d11b889e9d044a049d0586382c263951 deleted and replaced by tablet e4441c724b394e8092fb19c0d2427233
I20260811 18:57:38.108707 4163 raft_consensus.cc:348] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.111124 4163 raft_consensus.cc:374] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.111196 4163 raft_consensus.cc:729] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.111613 4163 consensus_queue.cc:260] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.112296 5407 raft_consensus.cc:686] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:38.112381 4163 ts_tablet_manager.cc:1428] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:38.112601 5407 consensus_queue.cc:237] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:38.117178 3824 leader_election.cc:341] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:38.117766 5557 raft_consensus.cc:2726] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.118477 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6f3060822021431584a494e6941be756 (table default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]) successfully deleted
I20260811 18:57:38.119063 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.119263 3745 raft_consensus.cc:2454] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:38.121982 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9914a933914f4377ba70e0b57c70edff" } new_table_name: "default.Table_c3327dac0fcc458d9b96d9eeb6098f22" modify_external_catalogs: true
I20260811 18:57:38.124890 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_80140471fa194b22bf8347836f95fdca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.125450 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80140471fa194b22bf8347836f95fdca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.126811 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0765ca548a9649e88a2ad253c392107b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.127434 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0765ca548a9649e88a2ad253c392107b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.129698 5407 raft_consensus.cc:482] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.129899 5407 raft_consensus.cc:504] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.131803 5407 leader_election.cc:288] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.133719 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18242f762d8407db0e2f8716890feb4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.133864 3678 leader_election.cc:334] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.134460 3676 leader_election.cc:341] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.134594 3676 leader_election.cc:302] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:38.136392 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dd80633f926942e88f54b32d9a03ea3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd80633f926942e88f54b32d9a03ea3a already in progress: creating tablet
W20260811 18:57:38.136404 3824 leader_election.cc:334] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.136533 3824 leader_election.cc:302] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.136890 5593 raft_consensus.cc:2726] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.140314 4167 ts_tablet_manager.cc:1918] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:38.140457 4167 log.cc:1196] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ef47211aa8864d13a55551da47804c19
I20260811 18:57:38.140848 4167 ts_tablet_manager.cc:1939] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.141681 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1058) took 2321 ms. Trace:
I20260811 18:57:38.141793 4167 rpcz_store.cc:276] 0811 18:57:35.820344 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.820397 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.787439 (+1967042us) ts_tablet_manager.cc:1145] Deleting tablet ef47211aa8864d13a55551da47804c19
0811 18:57:37.787450 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.058537 (+271087us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.140193 (+ 81656us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.141659 (+ 1466us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1966867}
I20260811 18:57:38.142239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1059) took 2321 ms. Trace:
I20260811 18:57:38.142359 4167 rpcz_store.cc:276] 0811 18:57:35.820890 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.820941 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:38.142114 (+2321173us) ts_tablet_manager.cc:1145] Deleting tablet 22e1a33348e34e35bab786f8966c408b
0811 18:57:38.142125 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.142221 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2321006}
I20260811 18:57:38.142706 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1060) took 2316 ms. Trace:
I20260811 18:57:38.142781 4167 rpcz_store.cc:276] 0811 18:57:35.826006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.826058 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:38.142591 (+2316533us) ts_tablet_manager.cc:1145] Deleting tablet 6f804703e20c4214b8b2a31efeaef59e
0811 18:57:38.142603 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.142678 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2316343}
I20260811 18:57:38.143088 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1061) took 2303 ms. Trace:
I20260811 18:57:38.143157 4167 rpcz_store.cc:276] 0811 18:57:35.839641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.839720 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:38.142994 (+2303274us) ts_tablet_manager.cc:1145] Deleting tablet 9b931eca7ba04c11af6571cb418f882e
0811 18:57:38.143004 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.143075 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2303001}
W20260811 18:57:38.143236 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ee5acbaa9f54d4ba523b603641e4a97 consensus state has no leader
I20260811 18:57:38.143421 4167 tablet_replica.cc:330] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.143640 4167 raft_consensus.cc:2229] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.143828 4167 raft_consensus.cc:2258] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.144398 4167 ts_tablet_manager.cc:1905] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.152639 4169 ts_tablet_manager.cc:1918] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.152808 4169 log.cc:1196] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:38.153169 4169 ts_tablet_manager.cc:1939] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.153549 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 955) took 3817 ms. Trace:
I20260811 18:57:38.153659 4169 rpcz_store.cc:276] 0811 18:57:34.336295 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336370 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:38.027416 (+3691046us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:38.027426 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.148682 (+121256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.152521 (+ 3839us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.153529 (+ 1008us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675678}
I20260811 18:57:38.154126 4169 tablet_replica.cc:330] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.157522 4163 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.157789 5407 raft_consensus.cc:2726] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.159246 4169 raft_consensus.cc:2229] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.159449 4169 raft_consensus.cc:2258] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.160427 4163 tablet_bootstrap.cc:654] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.162245 4162 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.162809 4169 ts_tablet_manager.cc:1905] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.163126 4162 ts_tablet_manager.cc:1397] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.218s user 0.016s sys 0.007s
I20260811 18:57:38.167901 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8dcb132b5b5474b8ff081e54a55dd3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.167842 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.170575 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2dffa8e09c6948c092cfdd4d888e5f81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.175407 4162 raft_consensus.cc:348] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:38.175740 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 64085.1us
I20260811 18:57:38.169569 3860 tablet_service.cc:1467] Processing CreateTablet for tablet e6562b8db6d54b42a89d19df04cab07c (DEFAULT_TABLE table=default.Table_88737be020144b20a2c4df22b9bf7b52 [id=2ef647fe25634721afac1f7c2d4a359a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.176230 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6562b8db6d54b42a89d19df04cab07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.176481 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.171165 3871 tablet_service.cc:1467] Processing CreateTablet for tablet c590c98e39f84ea09b26bbe8640ca93d (DEFAULT_TABLE table=default.Table_ba12b590ed224215a8ea56022e39e6c0 [id=54bb76f0477a4a589ad12b460d4c7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.179677 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c590c98e39f84ea09b26bbe8640ca93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.172567 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 915ab20a6b404a1f89080ce78214001d (DEFAULT_TABLE table=default.Table_3b465c02cbcd4236a62ea1c357497b89 [id=5469d608a7c04d9b99adea950d56bdd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.182922 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915ab20a6b404a1f89080ce78214001d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.173125 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.175369 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.175710 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0e443ab84424dac307b745b496986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.188751 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 69059a0667234a408cc1c09585df63bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.189231 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3799766ede4e4c8bb4e3ab35a723ebc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.189877 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8dcb132b5b5474b8ff081e54a55dd3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.190742 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.190918 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2dffa8e09c6948c092cfdd4d888e5f81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.191640 4163 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.191836 4163 ts_tablet_manager.cc:1397] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.001s
I20260811 18:57:38.195255 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.197880 4163 raft_consensus.cc:348] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:38.202310 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8c07150e31994bdf9a00ac2e7a954691 (table default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]) successfully deleted
I20260811 18:57:38.207077 4163 raft_consensus.cc:374] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.207221 4163 raft_consensus.cc:729] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.207492 4163 consensus_queue.cc:260] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:38.202279 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet d11b889e9d044a049d0586382c263951 with delete_type TABLET_DATA_DELETED (replaced by tablet e4441c724b394e8092fb19c0d2427233) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.214776 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.215029 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b04e37fc264fc08e0ed9517356a17d with delete_type TABLET_DATA_DELETED (replaced by tablet 94857ef729eb435c97d089b8ab108cea) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.205906 3715 tablet_service.cc:1467] Processing CreateTablet for tablet c590c98e39f84ea09b26bbe8640ca93d (DEFAULT_TABLE table=default.Table_ba12b590ed224215a8ea56022e39e6c0 [id=54bb76f0477a4a589ad12b460d4c7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.215575 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c590c98e39f84ea09b26bbe8640ca93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.206147 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.206279 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.216662 4162 raft_consensus.cc:374] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.216871 4162 raft_consensus.cc:729] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.217494 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80" } modify_external_catalogs: true
I20260811 18:57:38.217657 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.217733 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80" } modify_external_catalogs: true
I20260811 18:57:38.206620 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 915ab20a6b404a1f89080ce78214001d (DEFAULT_TABLE table=default.Table_3b465c02cbcd4236a62ea1c357497b89 [id=5469d608a7c04d9b99adea950d56bdd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.218472 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915ab20a6b404a1f89080ce78214001d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.218683 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_46d76d83b3174f5da337585b00af5f68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.219293 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46d76d83b3174f5da337585b00af5f68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.221252 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3020dd34e96c426fbbca98e320405d0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.221757 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3020dd34e96c426fbbca98e320405d0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.222316 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91bea743ec0b491299f45f3ac23cb734" } modify_external_catalogs: true
I20260811 18:57:38.222497 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91bea743ec0b491299f45f3ac23cb734" } modify_external_catalogs: true
I20260811 18:57:38.223379 4163 ts_tablet_manager.cc:1428] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.030s user 0.001s sys 0.001s
I20260811 18:57:38.224223 4162 consensus_queue.cc:260] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.224956 4162 ts_tablet_manager.cc:1428] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.050s user 0.001s sys 0.001s
I20260811 18:57:38.236351 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8" } modify_external_catalogs: true
I20260811 18:57:38.236601 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8" } modify_external_catalogs: true
I20260811 18:57:38.240341 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24401a03e8d54050a74da4ed3bbb60fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.241240 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24401a03e8d54050a74da4ed3bbb60fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.241572 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc1e05602be443528f758fba1ac0195f" } modify_external_catalogs: true
I20260811 18:57:38.241809 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc1e05602be443528f758fba1ac0195f" } modify_external_catalogs: true
I20260811 18:57:38.246124 4162 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.247702 3552 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 43fcb5054edb47a3bea8bcd1fbef036e
W20260811 18:57:38.249675 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:38.249855 3824 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.249996 3824 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.250083 3824 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.250393 5593 raft_consensus.cc:2726] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.251508 4163 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.252681 4167 ts_tablet_manager.cc:1918] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.252858 4167 log.cc:1196] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/19243ab3d6744aaa87a133f1462cd2fa
I20260811 18:57:38.253206 4167 ts_tablet_manager.cc:1939] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.253723 4163 tablet_bootstrap.cc:654] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.254091 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1062) took 2388 ms. Trace:
I20260811 18:57:38.261983 4163 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.262388 4163 ts_tablet_manager.cc:1397] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260811 18:57:38.263266 4163 raft_consensus.cc:348] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.263515 4163 raft_consensus.cc:374] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.263572 4163 raft_consensus.cc:729] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.263839 4163 consensus_queue.cc:260] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.264636 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ef47211aa8864d13a55551da47804c19 (table default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c]) successfully deleted
I20260811 18:57:38.265864 4162 tablet_bootstrap.cc:654] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.266589 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 22e1a33348e34e35bab786f8966c408b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22e1a33348e34e35bab786f8966c408b already in progress: creating tablet
W20260811 18:57:38.266829 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6f804703e20c4214b8b2a31efeaef59e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f804703e20c4214b8b2a31efeaef59e already in progress: creating tablet
W20260811 18:57:38.267043 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9b931eca7ba04c11af6571cb418f882e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b931eca7ba04c11af6571cb418f882e already in progress: creating tablet
I20260811 18:57:38.268746 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269059 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 69059a0667234a408cc1c09585df63bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269263 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b04e37fc264fc08e0ed9517356a17d with delete_type TABLET_DATA_DELETED (replaced by tablet 94857ef729eb435c97d089b8ab108cea) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269455 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet d11b889e9d044a049d0586382c263951 with delete_type TABLET_DATA_DELETED (replaced by tablet e4441c724b394e8092fb19c0d2427233) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269833 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0e443ab84424dac307b745b496986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.270392 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 31bb11617f1548f88f534289706edb06 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.274178 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5437383c0bbb4384825f578e83c10acc" } new_table_name: "default.Table_5d568786ed494eae8f240cd653776099" modify_external_catalogs: true
I20260811 18:57:38.275642 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_51094e8e45fe427789a0f442148332f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.276449 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_325467f49f2648b5a83d5fc618fb302d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.277148 3678 consensus_peers.cc:487] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.277582 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_325467f49f2648b5a83d5fc618fb302d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.277762 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51094e8e45fe427789a0f442148332f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.280122 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_af28562bfbf94af98e28afef24fda608" } new_table_name: "default.Table_af422068a6ec4b0e8bb6f9851275fefc" modify_external_catalogs: true
I20260811 18:57:38.254220 4167 rpcz_store.cc:276] 0811 18:57:35.865809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.865876 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:38.143381 (+2277505us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:38.143390 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.203452 (+ 60062us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.252565 (+ 49113us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.254061 (+ 1496us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2277282}
I20260811 18:57:38.283133 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a055754403774bc0a94fda2f957c70d5" } new_table_name: "default.Table_18ec139c6f2e4ba6b43bc08eb73f85c6" modify_external_catalogs: true
I20260811 18:57:38.292543 3824 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:38.292811 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a385abaeb70347dcac06a5cedbdddeb4 consensus state has no leader
I20260811 18:57:38.293274 4167 tablet_replica.cc:330] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.293228 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34d2346219e14bd58aa02228ad109763" } modify_external_catalogs: true
I20260811 18:57:38.293462 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34d2346219e14bd58aa02228ad109763" } modify_external_catalogs: true
I20260811 18:57:38.293525 4167 raft_consensus.cc:2229] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:38.293898 4167 raft_consensus.cc:2258] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.295122 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.296507 5593 raft_consensus.cc:2781] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.296629 5593 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.296715 5593 raft_consensus.cc:3037] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.297103 4167 ts_tablet_manager.cc:1905] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.297307 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:38.299571 5593 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.300262 5593 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.296502 4169 ts_tablet_manager.cc:1918] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.302549 4169 log.cc:1196] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:38.302910 4169 ts_tablet_manager.cc:1939] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.303278 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 956) took 3966 ms. Trace:
I20260811 18:57:38.303378 4169 rpcz_store.cc:276] 0811 18:57:34.336408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336548 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:38.154058 (+3817510us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:38.154073 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.258486 (+104413us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.296353 (+ 37867us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.303257 (+ 6904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3802172}
I20260811 18:57:38.303730 4169 tablet_replica.cc:330] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.303956 4169 raft_consensus.cc:2229] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:38.304250 4169 pending_rounds.cc:62] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:38.304308 4169 pending_rounds.cc:68] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400182754709504 op_type: NO_OP noop_request { }
I20260811 18:57:38.304437 4169 raft_consensus.cc:2866] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:38.306036 4169 raft_consensus.cc:2258] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.307075 4162 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.307423 4162 ts_tablet_manager.cc:1397] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.004s
I20260811 18:57:38.307739 4163 ts_tablet_manager.cc:1428] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.045s user 0.002s sys 0.000s
I20260811 18:57:38.308605 4162 raft_consensus.cc:348] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.308987 4162 raft_consensus.cc:374] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.309110 4162 raft_consensus.cc:729] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.309458 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e79b6e054e914fd39ca0a6282eb0ce17" } new_table_name: "default.Table_9beab5ab1dab4912a4c7ff553fd620f8" modify_external_catalogs: true
I20260811 18:57:38.310650 5561 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.310784 5561 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.311297 4162 consensus_queue.cc:260] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.311482 5561 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.312083 4162 ts_tablet_manager.cc:1428] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 18:57:38.319370 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.321998 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb1cb45730b14b998a8f547983b83823 consensus state has no leader
I20260811 18:57:38.325484 3550 catalog_manager.cc:5495] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:38.325999 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
W20260811 18:57:38.326489 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c142d8ef40947e29f3c94e1e8e4c517 consensus state has no leader
I20260811 18:57:38.332681 4162 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.334846 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.335638 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:38.335899 3745 raft_consensus.cc:3037] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.337106 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:38.338855 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b3f0cdcee681460084662e01152ad9dd (table default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]) successfully deleted
W20260811 18:57:38.340088 3824 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.340559 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.340838 3740 raft_consensus.cc:2454] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:38.343282 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 31bb11617f1548f88f534289706edb06
I20260811 18:57:38.343706 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 31bb11617f1548f88f534289706edb06 deleted and replaced by tablet 60f82aa67c614c6abcac348b67255904
I20260811 18:57:38.343703 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 76506ad179a9408c9e4cf8e4275ebddd
I20260811 18:57:38.343869 4169 ts_tablet_manager.cc:1905] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:38.343317 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23c85b2315254e4c8d9d69fec4cc9aa2 consensus state has no leader
I20260811 18:57:38.345959 4162 tablet_bootstrap.cc:654] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.347374 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 112d72e213dc415c959f8562bb83b31b (DEFAULT_TABLE table=default.Table_0c6490a83dec455bac3fe91b50da8a95 [id=d985a8592d304a808a31df85c9bba953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.347920 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112d72e213dc415c959f8562bb83b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.350590 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_149093c5dbb2402ebc6afe1a069fb195" } new_table_name: "default.Table_7cd9fda507254d029c62090b18422521" modify_external_catalogs: true
I20260811 18:57:38.350737 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9042bc231f0841a4a370be5e6251dd6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.351383 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9042bc231f0841a4a370be5e6251dd6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.352990 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bce1d6623de1467198e1dfaf763f08cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.353330 4167 ts_tablet_manager.cc:1918] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:38.353453 4167 log.cc:1196] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e3021c51525846e199f279f988248af0
W20260811 18:57:38.353523 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bce1d6623de1467198e1dfaf763f08cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.353873 4167 ts_tablet_manager.cc:1939] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.354749 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1063) took 2467 ms. Trace:
I20260811 18:57:38.355444 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c7f500a5444bae9a015a5e903c0622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.355875 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 7fad52ead22f42dc9dd487557c9acce8 (DEFAULT_TABLE table=default.Table_bb1820be09714d0b976eda8e0151671e [id=f6550593f6194360b4e6d3d7ca1b2466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.356269 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fad52ead22f42dc9dd487557c9acce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.357465 4163 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.358933 4163 tablet_bootstrap.cc:654] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.360378 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b48a82a7182b4bf89fb9fd3830829909 (DEFAULT_TABLE table=default.Table_cb4ec5ebc85e4562a65d10436234b4c2 [id=70fa03d99296435383f2d465179a431a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.360736 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48a82a7182b4bf89fb9fd3830829909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.370527 4167 rpcz_store.cc:276] 0811 18:57:35.887344 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.887413 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:38.282686 (+2395273us) ts_tablet_manager.cc:1145] Deleting tablet e3021c51525846e199f279f988248af0
0811 18:57:38.293207 (+ 10521us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.300894 (+ 7687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.353213 (+ 52319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.354722 (+ 1509us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2395034}
I20260811 18:57:38.371742 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 8dc07c28ccfd48ee9c33e49442c6047b (table default.Table_7f65859fb42941e3a8375f65182f9ec4 [id=cabc6cc20754488eb152c0d4f686dce0]). Expected version 2 got 0
I20260811 18:57:38.371945 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet a811d4488d0e4541808a5e1aa44d663b (table default.Table_fb8dd203bf3941fbacfd7dad1418e566 [id=a71b71af19a44182aea7e742062558ca]). Expected version 1 got 0
I20260811 18:57:38.372364 4169 ts_tablet_manager.cc:1918] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:38.372505 4169 log.cc:1196] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:38.373301 4167 tablet_replica.cc:330] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.373530 4167 raft_consensus.cc:2229] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.373740 4167 raft_consensus.cc:2258] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.374495 4167 ts_tablet_manager.cc:1905] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.376283 4162 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.376559 4162 ts_tablet_manager.cc:1397] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.001s
W20260811 18:57:38.377035 3678 consensus_peers.cc:487] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.377713 4162 raft_consensus.cc:348] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.377967 4162 raft_consensus.cc:374] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.378044 4162 raft_consensus.cc:729] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.378271 4162 consensus_queue.cc:260] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.381407 4162 ts_tablet_manager.cc:1428] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:38.382083 4169 ts_tablet_manager.cc:1939] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:38.382951 5570 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 10282287177661667202) tablet 00000000000000000000000000000000 cur holder 0x617000081f80
I20260811 18:57:38.389910 4162 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.391688 4162 tablet_bootstrap.cc:654] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.393942 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 958) took 4057 ms. Trace:
I20260811 18:57:38.394326 3745 raft_consensus.cc:2454] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:38.394150 4169 rpcz_store.cc:276] 0811 18:57:34.336659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336708 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:38.303670 (+3966962us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:38.303682 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.349776 (+ 46094us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.372240 (+ 22464us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.393914 (+ 21674us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3951728}
I20260811 18:57:38.405823 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_97a83e019035401aa2d468bf20c5ec11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.406572 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97a83e019035401aa2d468bf20c5ec11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.406481 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b081a3f48c9048ecb88f1bd4571f934b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.407029 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b081a3f48c9048ecb88f1bd4571f934b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.408725 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_644eac4afde147dea1b6dc8cd69dc719"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.409207 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_644eac4afde147dea1b6dc8cd69dc719 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.399564 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.400170 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8de4374609f04020b23ea6fe111a7cd7 (DEFAULT_TABLE table=default.Table_eb78d7140fc94f75b13bca56099bde32 [id=06a4c34fa672484692f4051cf30cf242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.417409 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de4374609f04020b23ea6fe111a7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.420979 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 4451cc4d7c5940cdb41c021ae486a5a9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.401789 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bb11617f1548f88f534289706edb06 with delete_type TABLET_DATA_DELETED (replaced by tablet 60f82aa67c614c6abcac348b67255904) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.402595 3824 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.424618 4169 tablet_replica.cc:330] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.425923 5561 raft_consensus.cc:2781] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.426012 5561 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.426092 5561 raft_consensus.cc:3037] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.426306 4169 raft_consensus.cc:2229] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.428118 4169 raft_consensus.cc:2258] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.428826 4169 ts_tablet_manager.cc:1905] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:38.430831 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b203df4124364737839bf4c1ae072988: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.428861 5561 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.429677 3824 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.431579 5593 raft_consensus.cc:2781] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:38.431625 5561 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.431728 5593 raft_consensus.cc:686] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:38.431861 5557 consensus_queue.cc:1035] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.435388 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:38.441047 4162 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.441488 4162 ts_tablet_manager.cc:1397] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.000s
I20260811 18:57:38.443475 4162 raft_consensus.cc:348] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.444121 4162 raft_consensus.cc:374] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.444327 4162 raft_consensus.cc:729] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.444823 4162 consensus_queue.cc:260] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.431931 5593 consensus_queue.cc:237] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.453809 4162 ts_tablet_manager.cc:1428] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260811 18:57:38.459852 4167 ts_tablet_manager.cc:1918] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.460119 4167 log.cc:1196] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:38.461138 4162 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.463297 5622 raft_consensus.cc:482] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.463449 5622 raft_consensus.cc:504] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.464118 5622 leader_election.cc:288] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.466634 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f75dc515d7f3473b8e43f20255a70e3e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.467654 3676 leader_election.cc:341] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.467527 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_04519ba2021244b6922bea0e5b79ecb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.468242 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04519ba2021244b6922bea0e5b79ecb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.468281 3678 leader_election.cc:334] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.468389 3678 leader_election.cc:302] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.468873 5622 raft_consensus.cc:2726] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.469924 4167 ts_tablet_manager.cc:1939] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.472296 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 112d72e213dc415c959f8562bb83b31b (DEFAULT_TABLE table=default.Table_0c6490a83dec455bac3fe91b50da8a95 [id=d985a8592d304a808a31df85c9bba953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.472498 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 8de4374609f04020b23ea6fe111a7cd7 (DEFAULT_TABLE table=default.Table_eb78d7140fc94f75b13bca56099bde32 [id=06a4c34fa672484692f4051cf30cf242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.472807 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112d72e213dc415c959f8562bb83b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.473076 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d40b42e92ef24ade9a8d09a1ac910d82 (DEFAULT_TABLE table=default.Table_908f52c0075e4f6c8f60b6c84f13e004 [id=430c769710734a608fa3e3aa44488eb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.473420 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40b42e92ef24ade9a8d09a1ac910d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.474630 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7fad52ead22f42dc9dd487557c9acce8 (DEFAULT_TABLE table=default.Table_bb1820be09714d0b976eda8e0151671e [id=f6550593f6194360b4e6d3d7ca1b2466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.475149 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fad52ead22f42dc9dd487557c9acce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.478395 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de4374609f04020b23ea6fe111a7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.478443 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c7f500a5444bae9a015a5e903c0622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.478878 3705 tablet_service.cc:1467] Processing CreateTablet for tablet b48a82a7182b4bf89fb9fd3830829909 (DEFAULT_TABLE table=default.Table_cb4ec5ebc85e4562a65d10436234b4c2 [id=70fa03d99296435383f2d465179a431a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.479247 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48a82a7182b4bf89fb9fd3830829909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.482460 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bb11617f1548f88f534289706edb06 with delete_type TABLET_DATA_DELETED (replaced by tablet 60f82aa67c614c6abcac348b67255904) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.482717 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.482522 4162 tablet_bootstrap.cc:654] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.490511 5622 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.490724 5622 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.491488 5622 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.494828 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.495952 3676 leader_election.cc:341] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.496351 3678 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.496526 3678 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.496814 5407 raft_consensus.cc:482] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.496944 5407 raft_consensus.cc:504] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.497555 5407 leader_election.cc:288] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.497691 5407 raft_consensus.cc:2726] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.499106 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eb43127dda814a718b338906fc2a3c47
I20260811 18:57:38.498993 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8304ac7709ca4392baec4cfa78ef5b41" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:38.499452 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_724b9320db2141f39b2511549a1aa8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.499927 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba12b590ed224215a8ea56022e39e6c0" } modify_external_catalogs: true
I20260811 18:57:38.500123 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba12b590ed224215a8ea56022e39e6c0" } modify_external_catalogs: true
W20260811 18:57:38.500676 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f404fe05a8674605bb8038d676443046 has no consensus state
I20260811 18:57:38.501120 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4fd6bf06dd964e719ce1470aec9f60bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.501729 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fd6bf06dd964e719ce1470aec9f60bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.502576 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ee5b7d1e32514ad48177371582e85241
W20260811 18:57:38.504055 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_724b9320db2141f39b2511549a1aa8d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.505227 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dbebdb613fa84efc9722613af36d52c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.505878 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbebdb613fa84efc9722613af36d52c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.509019 3676 leader_election.cc:341] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.509541 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_661f1ebd9f4c4cb9be591c5e1c203a86" } new_table_name: "default.Table_062f4ec20c7f4f41ace0731964dc4d96" modify_external_catalogs: true
I20260811 18:57:38.511410 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1064) took 2493 ms. Trace:
I20260811 18:57:38.512936 4167 rpcz_store.cc:276] 0811 18:57:36.018088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.018154 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:38.373241 (+2355087us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:38.373254 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.389349 (+ 16095us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.459714 (+ 70365us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.511359 (+ 51645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2354813}
I20260811 18:57:38.513612 4167 tablet_replica.cc:330] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.515020 4167 raft_consensus.cc:2229] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.515771 4169 ts_tablet_manager.cc:1918] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.516065 4167 raft_consensus.cc:2258] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.518306 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 302cf69ea7c14fdaac72f6a9a9aff113 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:38.518786 3678 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.519058 3678 leader_election.cc:302] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.519641 4167 ts_tablet_manager.cc:1905] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.519869 4169 log.cc:1196] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:38.520365 4169 ts_tablet_manager.cc:1939] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.519842 5622 raft_consensus.cc:2726] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.523401 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 957) took 4186 ms. Trace:
I20260811 18:57:38.520520 5630 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.523713 5630 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.524482 5630 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.524681 4169 rpcz_store.cc:276] 0811 18:57:34.336524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.343747 (+ 7223us) service_pool.cc:227] Handling call
0811 18:57:38.424558 (+4080811us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:38.424573 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.453897 (+ 29324us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.515646 (+ 61749us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.523378 (+ 7732us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4065677}
I20260811 18:57:38.525413 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 961) took 3555 ms. Trace:
I20260811 18:57:38.525573 4169 rpcz_store.cc:276] 0811 18:57:34.970408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.970794 (+ 386us) service_pool.cc:227] Handling call
0811 18:57:38.525261 (+3554467us) ts_tablet_manager.cc:1145] Deleting tablet a25b1eddd4a64847b51ad3cdc0238ff0
0811 18:57:38.525275 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.525393 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3554326}
I20260811 18:57:38.526049 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 960) took 3555 ms. Trace:
W20260811 18:57:38.528191 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 96460.2us
I20260811 18:57:38.528615 4169 rpcz_store.cc:276] 0811 18:57:34.970271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.970355 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:38.525916 (+3555561us) ts_tablet_manager.cc:1145] Deleting tablet 40898ceb18cc4f2d9104818798049ae2
0811 18:57:38.525929 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.526031 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43,"tablet-delete.queue_time_us":3554979}
W20260811 18:57:38.529294 3824 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.528913 5557 consensus_queue.cc:1035] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.529649 4169 tablet_replica.cc:330] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.530625 4169 raft_consensus.cc:2229] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.530635 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_605185b9f67e456ba194522b24112e39" } new_table_name: "default.Table_e71f95e50a3b4ac188f0ca2b82fc95bb" modify_external_catalogs: true
I20260811 18:57:38.530871 4169 raft_consensus.cc:2258] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:38.529820 3824 consensus_peers.cc:487] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.532382 4162 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.532622 4162 ts_tablet_manager.cc:1397] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.002s
I20260811 18:57:38.532665 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 1e1374a80d15444cb0faff339275aa8d (table default.Table_647ae82e63464976abb7f26c9526a349 [id=e0ef202b868f47269e4f2a63cb5a390a]) successfully deleted
I20260811 18:57:38.532779 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a6af8048b8f14af899345d8213953792"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.532967 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a25b1eddd4a64847b51ad3cdc0238ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25b1eddd4a64847b51ad3cdc0238ff0 already in progress: creating tablet
I20260811 18:57:38.533278 3852 tablet_service.cc:1467] Processing CreateTablet for tablet d40b42e92ef24ade9a8d09a1ac910d82 (DEFAULT_TABLE table=default.Table_908f52c0075e4f6c8f60b6c84f13e004 [id=430c769710734a608fa3e3aa44488eb9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:38.533416 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6af8048b8f14af899345d8213953792 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.533751 4162 raft_consensus.cc:348] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.534055 4162 raft_consensus.cc:374] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.534173 4162 raft_consensus.cc:729] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.534356 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 6168d1f0fffc4fad91581d1795d721ce (DEFAULT_TABLE table=default.Table_5ef097030b2146bb9df7f7d2bffde6b6 [id=0291bde7daf64dc480710af744852acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.534462 4162 consensus_queue.cc:260] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.535069 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:38.535333 3745 raft_consensus.cc:3037] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.536222 4162 ts_tablet_manager.cc:1428] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:38.540298 3745 raft_consensus.cc:2454] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:38.542137 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 49203e259d17449984dcd528df985537 (DEFAULT_TABLE table=default.Table_d2326f1112004b75bdb9d55298314884 [id=9833505c53af4634911465bd22485158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.542380 4162 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.543453 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 071a6513025341caaa0cac048d3caccd (DEFAULT_TABLE table=default.Table_b882f1fc510a41558369a5ced806d29a [id=d744d4b03cb64b5e8eb2b02f7387733b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.545169 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f10006748b084a6bbd1a2f6a55cca9b9 (DEFAULT_TABLE table=default.Table_35bcb0c44a4840879a1f1d1386bf24c1 [id=955bf462395b45a3b6fea1fce8dad61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.547215 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4451cc4d7c5940cdb41c021ae486a5a9
I20260811 18:57:38.547715 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 4451cc4d7c5940cdb41c021ae486a5a9 deleted and replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5
I20260811 18:57:38.547878 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40b42e92ef24ade9a8d09a1ac910d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.551918 4162 tablet_bootstrap.cc:654] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.553933 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071a6513025341caaa0cac048d3caccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.562042 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d" } modify_external_catalogs: true
I20260811 18:57:38.563921 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d" } modify_external_catalogs: true
I20260811 18:57:38.567185 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b069c186694049998175e3d83241b318" } new_table_name: "default.Table_9fbe81922b5e4d228c3b17450feb18d5" modify_external_catalogs: true
I20260811 18:57:38.568940 5622 raft_consensus.cc:482] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.569108 5622 raft_consensus.cc:504] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.569746 5622 leader_election.cc:288] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.571097 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4dc2df4c6464d71bb13fe25e0046758" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.572019 3676 leader_election.cc:341] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.573967 3678 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.574091 3678 leader_election.cc:302] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.574546 5407 raft_consensus.cc:2726] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.579466 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49203e259d17449984dcd528df985537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.582646 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10006748b084a6bbd1a2f6a55cca9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.586362 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6168d1f0fffc4fad91581d1795d721ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.617215 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c590c98e39f84ea09b26bbe8640ca93d
I20260811 18:57:38.632541 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" } modify_external_catalogs: true
I20260811 18:57:38.632828 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" } modify_external_catalogs: true
I20260811 18:57:38.632997 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acd269829eb743178d173c6cc3d7301e" } new_table_name: "default.Table_d990f007030d4d1e82827f549e934fa9" modify_external_catalogs: true
I20260811 18:57:38.634455 5622 raft_consensus.cc:482] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.634529 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cb54beb34a654fbdb9c7632901bb0b9e
W20260811 18:57:38.634655 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 547f6c33d6414bcfa3eeff20cb9a0302 consensus state has no leader
I20260811 18:57:38.634647 5622 raft_consensus.cc:504] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.635471 5622 leader_election.cc:288] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.637477 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8319453f68a54da9813747152ef08756 consensus state has no leader
I20260811 18:57:38.639803 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42405aec21e3405486b6c658438b5fb2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.641501 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
W20260811 18:57:38.641667 3676 leader_election.cc:341] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.646564 3678 leader_election.cc:334] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.646759 3678 leader_election.cc:302] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.647899 5622 raft_consensus.cc:2726] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:38.655084 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec27862124b74126a62f8588ec1237fa consensus state has no leader
I20260811 18:57:38.666321 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5b7d1e32514ad48177371582e85241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.666814 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 with delete_type TABLET_DATA_DELETED (replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.667248 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c590c98e39f84ea09b26bbe8640ca93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.667593 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.668151 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 40898ceb18cc4f2d9104818798049ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 40898ceb18cc4f2d9104818798049ae2 already in progress: creating tablet
W20260811 18:57:38.668720 3824 consensus_peers.cc:487] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.670769 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd (DEFAULT_TABLE table=default.Table_31dac67fa2c146b0956b5a3b25672da3 [id=4d02195d37e64519a56460e55acf9ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.671229 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.673641 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fa49814fb45f481a9c16e30ea947c79a (DEFAULT_TABLE table=default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb [id=22079dd74afc459081d1eb743f58c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.674105 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa49814fb45f481a9c16e30ea947c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.678985 4163 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.679157 4163 ts_tablet_manager.cc:1397] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.322s user 0.015s sys 0.010s
I20260811 18:57:38.680097 4163 raft_consensus.cc:348] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.680867 4163 raft_consensus.cc:374] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.680967 4163 raft_consensus.cc:729] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.681282 4163 consensus_queue.cc:260] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.682075 4163 ts_tablet_manager.cc:1428] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:38.685063 4169 ts_tablet_manager.cc:1905] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.690974 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:38.693135 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a7946503bec9446d8deb02448eb231d5 (DEFAULT_TABLE table=default.Table_c925e9ac591346b1a5142f338c68bcc6 [id=c5d66f3cc70d498eb0db5c57bb06442d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.697223 3860 tablet_service.cc:1467] Processing CreateTablet for tablet de822817a9634958ba0d1a3128691dba (DEFAULT_TABLE table=default.Table_5dc02b59b48d45c8bee6458f21962eeb [id=7ee28ea901454542818bc026827867b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.697592 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet eb43127dda814a718b338906fc2a3c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.697854 5593 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.698113 5593 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.698781 5593 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.700403 3555 catalog_manager.cc:5495] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:38.701009 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:38.702051 4167 ts_tablet_manager.cc:1918] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.721114 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8e26f75764214905965fa0f514cb0028"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.721329 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_440c0b981c8a42c28bf5f00f50ad7296" } new_table_name: "default.Table_9db7061f3798449280da2e921e2181bf" modify_external_catalogs: true
W20260811 18:57:38.721794 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e26f75764214905965fa0f514cb0028 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.727005 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ee627ac9abf47889b5fed972cecd97e" } modify_external_catalogs: true
I20260811 18:57:38.728312 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b" } modify_external_catalogs: true
I20260811 18:57:38.735134 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bf1e0bf055c44f45835974098ec141f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.738173 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 09366eccf38a4876840cc5ddddbbb103
I20260811 18:57:38.742483 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b" } modify_external_catalogs: true
I20260811 18:57:38.751243 5617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:38.751678 5617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:38.751713 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:38.752161 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:38.739476 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ee627ac9abf47889b5fed972cecd97e" } modify_external_catalogs: true
W20260811 18:57:38.762194 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf1e0bf055c44f45835974098ec141f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.762766 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_53bcd19697ad49b9aec577b1646c0d3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.763475 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53bcd19697ad49b9aec577b1646c0d3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.764575 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9919772109274064aee942f6a7f1220f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.765128 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9919772109274064aee942f6a7f1220f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.767928 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7946503bec9446d8deb02448eb231d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.771272 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de822817a9634958ba0d1a3128691dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.776389 5593 consensus_queue.cc:1035] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.788113 5278 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 0) took 7995 ms. Trace:
I20260811 18:57:38.797987 4163 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.800503 5557 raft_consensus.cc:482] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.800740 5557 raft_consensus.cc:504] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.801632 5557 leader_election.cc:288] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:38.802495 3824 leader_election.cc:341] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:38.803143 3824 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.803402 5557 raft_consensus.cc:2781] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:38.803511 5557 raft_consensus.cc:686] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:38.803743 5557 consensus_queue.cc:237] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.805439 5617 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:38.741443 5622 raft_consensus.cc:482] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.812592 4167 log.cc:1196] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:38.812608 5622 raft_consensus.cc:504] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.813050 4167 ts_tablet_manager.cc:1939] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.813462 5622 leader_election.cc:288] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.772256 3678 consensus_peers.cc:487] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.814049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1065) took 2795 ms. Trace:
I20260811 18:57:38.814146 4167 rpcz_store.cc:276] 0811 18:57:36.018784 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.018838 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:38.513557 (+2494719us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:38.513571 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.527740 (+ 14169us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.701924 (+174184us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.814024 (+112100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2494528}
I20260811 18:57:38.814561 4167 tablet_replica.cc:330] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.814790 4167 raft_consensus.cc:2229] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.815603 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa8a4a1c0cb4834be501bdd7d468472" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.816725 3676 leader_election.cc:341] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.818182 3678 leader_election.cc:334] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.818320 3678 leader_election.cc:302] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.820303 5622 raft_consensus.cc:2726] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.828917 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2486985deae44613a5f8d342c8236784"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.841296 3824 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.841697 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.841843 3824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 183 similar messages]
W20260811 18:57:38.842450 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.842581 3824 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.860497 4163 tablet_bootstrap.cc:654] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.866780 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2486985deae44613a5f8d342c8236784 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.869310 3678 consensus_peers.cc:487] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.870216 3678 consensus_peers.cc:487] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.871809 5646 raft_consensus.cc:2726] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.873688 5642 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:38.876310 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:38.884279 5646 raft_consensus.cc:482] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.884495 5646 raft_consensus.cc:504] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.885391 5646 leader_election.cc:288] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.886869 4162 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.887050 4162 ts_tablet_manager.cc:1397] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.345s user 0.025s sys 0.025s
I20260811 18:57:38.887842 4162 raft_consensus.cc:348] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.888108 4162 raft_consensus.cc:374] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.888170 4162 raft_consensus.cc:729] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.888448 4162 consensus_queue.cc:260] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.889104 4162 ts_tablet_manager.cc:1428] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:38.889415 4162 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.891542 4162 tablet_bootstrap.cc:654] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.897831 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:38.898624 4167 raft_consensus.cc:2258] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.899343 4167 ts_tablet_manager.cc:1905] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.900476 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ad53b06897e4559b15d34a4d98f4005" } modify_external_catalogs: true
I20260811 18:57:38.900734 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ad53b06897e4559b15d34a4d98f4005" } modify_external_catalogs: true
I20260811 18:57:38.908847 4163 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.909041 4163 ts_tablet_manager.cc:1397] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.111s user 0.003s sys 0.000s
I20260811 18:57:38.910053 4163 raft_consensus.cc:348] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.910313 4163 raft_consensus.cc:374] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.910408 4163 raft_consensus.cc:729] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.910645 4163 consensus_queue.cc:260] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.901520 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.923786 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.926374 4169 ts_tablet_manager.cc:1918] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.926548 4169 log.cc:1196] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:38.926905 4169 ts_tablet_manager.cc:1939] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.927706 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8072b2d13cbc45249f43411e9022d006 (DEFAULT_TABLE table=default.Table_46d76d83b3174f5da337585b00af5f68 [id=40394961c1b348b4b67b3322da0eb2c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.928375 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 812cc50dcac740c5ab7633abc04c1d9e (DEFAULT_TABLE table=default.Table_3020dd34e96c426fbbca98e320405d0f [id=7f1262b7d1754c54aa332bc31a4db6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.928424 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.928942 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 81f52a745f6243ae9b1ef2dd33b06d3a (DEFAULT_TABLE table=default.Table_fe6c2229e23048eb81c61314102eab38 [id=57de9419daff451a9ca6485e9230e621]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:38.929075 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eae6865a9edf4e0ebe72f448ac7ab770 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.929365 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 09366eccf38a4876840cc5ddddbbb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.929854 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 258d08de21434edc86f0cab49f839294 (DEFAULT_TABLE table=default.Table_2dffa8e09c6948c092cfdd4d888e5f81 [id=58c3e7941755444c978bd70d45db09f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.931057 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (table default.Table_c71b8760db044bad851b4bcd819bf75f [id=7a869d9623bc4e07938b2d26688723a5]). Expected version 2 got 0
I20260811 18:57:38.936331 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 407b07d906d4447dbae5fd45d44e3ef8 (DEFAULT_TABLE table=default.Table_24401a03e8d54050a74da4ed3bbb60fd [id=936d78e258b341f98d02313652a325f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.937434 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f404fe05a8674605bb8038d676443046 (DEFAULT_TABLE table=default.Table_7cd9fda507254d029c62090b18422521 [id=50dde5de39e14c068d494813f4e68a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.938439 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 408c18a220d94809a5c43e1bfca310c8 (DEFAULT_TABLE table=default.Table_325467f49f2648b5a83d5fc618fb302d [id=4dcf548dd1e84122a82fa8971e993312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.938609 3865 tablet_service.cc:1467] Processing CreateTablet for tablet f87a8741e8ec481ea21284abe2d6c479 (DEFAULT_TABLE table=default.Table_788572afacab482590222fc4d084ef0f [id=77992422c86e47c79d4e1aea0a66b089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.939399 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d3e28a19c145403485376b057bad4c9e (DEFAULT_TABLE table=default.Table_51094e8e45fe427789a0f442148332f0 [id=7aa13fc26748491db551f61611f7c62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.940733 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 60f82aa67c614c6abcac348b67255904 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.942132 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9c048122b5434ec594fbf5b2665edfab (DEFAULT_TABLE table=default.Table_0765ca548a9649e88a2ad253c392107b [id=46788ef8b34947bbbffd5f651a41a102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.943665 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b752724648a499980ad71fd596a731f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.943863 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 94857ef729eb435c97d089b8ab108cea (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.948807 5557 raft_consensus.cc:482] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.949553 5557 raft_consensus.cc:504] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.952075 5557 leader_election.cc:288] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.953348 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b07d906d4447dbae5fd45d44e3ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.953963 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c18a220d94809a5c43e1bfca310c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.954150 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e28a19c145403485376b057bad4c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.954457 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f404fe05a8674605bb8038d676443046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.958539 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f82aa67c614c6abcac348b67255904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.962598 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f87a8741e8ec481ea21284abe2d6c479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.982892 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c048122b5434ec594fbf5b2665edfab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:38.986589 3678 consensus_peers.cc:487] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.987039 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94857ef729eb435c97d089b8ab108cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.987723 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 962) took 4016 ms. Trace:
W20260811 18:57:39.010433 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b752724648a499980ad71fd596a731f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.011677 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 930b6e8839e24ec9a69854349bc3ecf8 (table default.Table_9db7061f3798449280da2e921e2181bf [id=d7d11af2eee54ab880a15c525b7ad0a9], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:39.027179 4169 rpcz_store.cc:276] 0811 18:57:34.971091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.971687 (+ 596us) service_pool.cc:227] Handling call
0811 18:57:38.529595 (+3557908us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:38.529607 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.716429 (+186822us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.926070 (+209641us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.987700 (+ 61630us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3557491}
I20260811 18:57:39.029419 4169 tablet_replica.cc:330] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.030401 4169 raft_consensus.cc:2229] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.031582 4169 raft_consensus.cc:2258] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:38.988991 3824 consensus_peers.cc:487] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:39.033349 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44328.3us
I20260811 18:57:39.019375 4163 ts_tablet_manager.cc:1428] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.110s user 0.004s sys 0.000s
I20260811 18:57:39.024945 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812cc50dcac740c5ab7633abc04c1d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.788347 5278 rpcz_store.cc:276] 0811 18:57:30.792355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.792444 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:36.978279 (+6185835us) raft_consensus.cc:1457] Updating replica for 662 ops
0811 18:57:37.340756 (+362477us) raft_consensus.cc:1520] Early marking committed up to index 398
0811 18:57:37.344297 (+ 3541us) raft_consensus.cc:1525] Triggering prepare for 662 ops
0811 18:57:37.345239 (+ 942us) write_op.cc:270] Start()
0811 18:57:37.345297 (+ 58us) write_op.cc:276] Timestamp: P: 1786474635810894 usec, L: 0
0811 18:57:37.353495 (+ 8198us) write_op.cc:270] Start()
0811 18:57:37.353546 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635811701 usec, L: 0
0811 18:57:37.359002 (+ 5456us) write_op.cc:270] Start()
0811 18:57:37.359037 (+ 35us) write_op.cc:276] Timestamp: P: 1786474635812439 usec, L: 0
0811 18:57:37.360195 (+ 1158us) write_op.cc:270] Start()
0811 18:57:37.360225 (+ 30us) write_op.cc:276] Timestamp: P: 1786474635813111 usec, L: 0
0811 18:57:37.361375 (+ 1150us) write_op.cc:270] Start()
0811 18:57:37.361404 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635813734 usec, L: 0
0811 18:57:37.368292 (+ 6888us) write_op.cc:270] Start()
0811 18:57:37.368343 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635860354 usec, L: 0
0811 18:57:37.370113 (+ 1770us) write_op.cc:270] Start()
0811 18:57:37.370210 (+ 97us) write_op.cc:276] Timestamp: P: 1786474635876830 usec, L: 0
0811 18:57:37.370893 (+ 683us) write_op.cc:270] Start()
0811 18:57:37.370922 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635916988 usec, L: 0
0811 18:57:37.371482 (+ 560us) write_op.cc:270] Start()
0811 18:57:37.371511 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635988940 usec, L: 0
0811 18:57:37.375527 (+ 4016us) write_op.cc:270] Start()
0811 18:57:37.375571 (+ 44us) write_op.cc:276] Timestamp: P: 1786474635990013 usec, L: 0
0811 18:57:37.376256 (+ 685us) write_op.cc:270] Start()
0811 18:57:37.376285 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635991607 usec, L: 0
0811 18:57:37.376949 (+ 664us) write_op.cc:270] Start()
0811 18:57:37.376977 (+ 28us) write_op.cc:276] Timestamp: P: 1786474635992238 usec, L: 0
0811 18:57:37.378434 (+ 1457us) write_op.cc:270] Start()
0811 18:57:37.378460 (+ 26us) write_op.cc:276] Timestamp: P: 1786474635992863 usec, L: 0
0811 18:57:37.379111 (+ 651us) write_op.cc:270] Start()
0811 18:57:37.379197 (+ 86us) write_op.cc:276] Timestamp: P: 1786474636034210 usec, L: 0
0811 18:57:37.380503 (+ 1306us) write_op.cc:270] Start()
0811 18:57:37.380526 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636035501 usec, L: 0
0811 18:57:37.381522 (+ 996us) write_op.cc:270] Start()
0811 18:57:37.381551 (+ 29us) write_op.cc:276] Timestamp: P: 1786474636036316 usec, L: 0
0811 18:57:37.381757 (+ 206us) write_op.cc:270] Start()
0811 18:57:37.381836 (+ 79us) write_op.cc:276] Timestamp: P: 1786474636037305 usec, L: 0
0811 18:57:37.385531 (+ 3695us) write_op.cc:270] Start()
0811 18:57:37.385565 (+ 34us) write_op.cc:276] Timestamp: P: 1786474636037997 usec, L: 0
0811 18:57:37.385799 (+ 234us) write_op.cc:270] Start()
0811 18:57:37.385820 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636070832 usec, L: 0
0811 18:57:37.385932 (+ 112us) write_op.cc:270] Start()
0811 18:57:37.385948 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636071659 usec, L: 0
0811 18:57:37.386067 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.386084 (+ 17us) write_op.cc:276] Timestamp: P: 1786474636072248 usec, L: 0
0811 18:57:37.386179 (+ 95us) write_op.cc:270] Start()
0811 18:57:37.386193 (+ 14us) write_op.cc:276] Timestamp: P: 1786474636072755 usec, L: 0
0811 18:57:37.386289 (+ 96us) write_op.cc:270] Start()
0811 18:57:37.386304 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636075267 usec, L: 0
0811 18:57:37.387746 (+ 1442us) write_op.cc:270] Start()
0811 18:57:37.387779 (+ 33us) write_op.cc:276] Timestamp: P: 1786474636083654 usec, L: 0
0811 18:57:37.387975 (+ 196us) write_op.cc:270] Start()
0811 18:57:37.387999 (+ 24us) write_op.cc:276] Timestamp: P: 1786474636214582 usec, L: 0
0811 18:57:37.388165 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.388188 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636264097 usec, L: 0
0811 18:57:37.401971 (+ 13783us) write_op.cc:270] Start()
0811 18:57:37.402033 (+ 62us) write_op.cc:276] Timestamp: P: 1786474636272706 usec, L: 0
0811 18:57:37.402267 (+ 234us) write_op.cc:270] Start()
0811 18:57:37.402292 (+ 25us) write_op.cc:276] Timestamp: P: 1786474636310865 usec, L: 0
0811 18:57:37.402463 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.402486 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636320577 usec, L: 0
0811 18:57:37.402642 (+ 156us) write_op.cc:270] Start()
0811 18:57:37.402664 (+ 22us) write_op.cc:276] Timestamp: P: 1786474636321455 usec, L: 0
0811 18:57:37.408058 (+ 5394us) write_op.cc:270] Start()
0811 18:57:37.408104 (+ 46us) write_op.cc:276] Timestamp: P: 1786474636322172 usec, L: 0
0811 18:57:37.408334 (+ 230us) write_op.cc:270] Start()
0811 18:57:37.408354 (+ 20us) write_op.cc:276] Timestamp: P: 1786474636328657 usec, L: 0
0811 18:57:37.408466 (+ 112us) write_op.cc:270] Start()
0811 18:57:37.408482 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636330376 usec, L: 0
0811 18:57:37.408582 (+ 100us) write_op.cc:270] Start()
0811 18:57:37.408598 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636339931 usec, L: 0
0811 18:57:37.408717 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.408734 (+ 17us) write_op.cc:276] Timestamp: P: 1786474636342761 usec, L: 0
0811 18:57:37.408837 (+ 103us) write_op.cc:270] Start()
0811 18:57:37.408852 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636344470 usec, L: 0
0811 18:57:37.408957 (+ 105us) write_op.cc:270] Start()
0811 18:57:37.408986 (+ 29us) write_op.cc:276] Timestamp: P: 1786474636347176 usec, L: 0
0811 18:57:37.409102 (+ 116us) write_op.cc:270] Start()
0811 18:57:37.409117 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636564973 usec, L: 0
0811 18:57:37.409217 (+ 100us) write_op.cc:270] Start()
0811 18:57:37.409232 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636576210 usec, L: 0
0811 18:57:37.409331 (+ 99us) write_op.cc:270] Start()
0811 18:57:37.409346 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636579204 usec, L: 0
0811 18:57:37.409443 (+ 97us) write_op.cc:270] Start()
0811 18:57:37.409459 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636640162 usec, L: 0
0811 18:57:37.409578 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.409594 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636703566 usec, L: 0
0811 18:57:37.409713 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.409728 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636711299 usec, L: 0
0811 18:57:37.409825 (+ 97us) write_op.cc:270] Start()
0811 18:57:37.409840 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636719825 usec, L: 0
0811 18:57:37.409942 (+ 102us) write_op.cc:270] Start()
0811 18:57:37.409957 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636721967 usec, L: 0
0811 18:57:37.410229 (+ 272us) write_op.cc:270] Start()
0811 18:57:37.410251 (+ 22us) write_op.cc:276] Timestamp: P: 1786474636793721 usec, L: 0
0811 18:57:37.410385 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.410404 (+ 19us) write_op.cc:276] Timestamp: P: 1786474636795286 usec, L: 0
0811 18:57:37.410511 (+ 107us) write_op.cc:270] Start()
0811 18:57:37.410527 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636827331 usec, L: 0
0811 18:57:37.410684 (+ 157us) write_op.cc:270] Start()
0811 18:57:37.410705 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636863900 usec, L: 0
0811 18:57:37.410931 (+ 226us) write_op.cc:270] Start()
0811 18:57:37.410951 (+ 20us) write_op.cc:276] Timestamp: P: 1786474636865795 usec, L: 0
0811 18:57:37.411068 (+ 117us) write_op.cc:270] Start()
0811 18:57:37.411084 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636886677 usec, L: 0
0811 18:57:37.411183 (+ 99us) write_op.cc:270] Start()
0811 18:57:37.411198 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636898148 usec, L: 0
0811 18:57:37.411305 (+ 107us) write_op.cc:270] Start()
0811 18:57:37.411320 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636899078 usec, L: 0
0811 18:57:37.413936 (+ 2616us) write_op.cc:270] Start()
0811 18:57:37.413971 (+ 35us) write_op.cc:276] Timestamp: P: 1786474636899680 usec, L: 0
0811 18:57:37.414165 (+ 194us) write_op.cc:270] Start()
0811 18:57:37.414186 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636900267 usec, L: 0
0811 18:57:37.414367 (+ 181us) write_op.cc:270] Start()
0811 18:57:37.414392 (+ 25us) write_op.cc:276] Timestamp: P: 1786474636900847 usec, L: 0
0811 18:57:37.414556 (+ 164us) write_op.cc:270] Start()
0811 18:57:37.414579 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636995805 usec, L: 0
0811 18:57:37.414737 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.414764 (+ 27us) write_op.cc:276] Timestamp: P: 1786474636996759 usec, L: 0
0811 18:57:37.416086 (+ 1322us) write_op.cc:270] Start()
0811 18:57:37.416188 (+ 102us) write_op.cc:276] Timestamp: P: 1786474636997412 usec, L: 0
0811 18:57:37.416416 (+ 228us) write_op.cc:270] Start()
0811 18:57:37.416469 (+ 53us) write_op.cc:276] Timestamp: P: 1786474637165325 usec, L: 0
0811 18:57:37.416651 (+ 182us) write_op.cc:270] Start()
0811 18:57:37.416692 (+ 41us) write_op.cc:276] Timestamp: P: 1786474637166611 usec, L: 0
0811 18:57:37.418816 (+ 2124us) write_op.cc:270] Start()
0811 18:57:37.418851 (+ 35us) write_op.cc:276] Timestamp: P: 1786474637183576 usec, L: 0
0811 18:57:37.419729 (+ 878us) write_op.cc:270] Start()
0811 18:57:37.419759 (+ 30us) write_op.cc:276] Timestamp: P: 1786474637197348 usec, L: 0
0811 18:57:37.419930 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.419952 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637198400 usec, L: 0
0811 18:57:37.420118 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.420138 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637223619 usec, L: 0
0811 18:57:37.420280 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.420302 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637226023 usec, L: 0
0811 18:57:37.420437 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.420458 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637227353 usec, L: 0
0811 18:57:37.420593 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.423062 (+ 2469us) write_op.cc:276] Timestamp: P: 1786474637256743 usec, L: 0
0811 18:57:37.423274 (+ 212us) write_op.cc:270] Start()
0811 18:57:37.423301 (+ 27us) write_op.cc:276] Timestamp: P: 1786474637261701 usec, L: 0
0811 18:57:37.423453 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.423798 (+ 345us) write_op.cc:276] Timestamp: P: 1786474637456592 usec, L: 0
0811 18:57:37.423968 (+ 170us) write_op.cc:270] Start()
0811 18:57:37.423993 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637499754 usec, L: 0
0811 18:57:37.424137 (+ 144us) write_op.cc:270] Start()
0811 18:57:37.424162 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637527823 usec, L: 0
0811 18:57:37.424322 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.424344 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637529119 usec, L: 0
0811 18:57:37.424504 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.424526 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637530126 usec, L: 0
0811 18:57:37.429495 (+ 4969us) write_op.cc:270] Start()
0811 18:57:37.429534 (+ 39us) write_op.cc:276] Timestamp: P: 1786474637530649 usec, L: 0
0811 18:57:37.429729 (+ 195us) write_op.cc:270] Start()
0811 18:57:37.429753 (+ 24us) write_op.cc:276] Timestamp: P: 1786474637531214 usec, L: 0
0811 18:57:37.429898 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.429923 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637531724 usec, L: 0
0811 18:57:37.430062 (+ 139us) write_op.cc:270] Start()
0811 18:57:37.430098 (+ 36us) write_op.cc:276] Timestamp: P: 1786474637532251 usec, L: 0
0811 18:57:37.430245 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.430301 (+ 56us) write_op.cc:276] Timestamp: P: 1786474637532782 usec, L: 0
0811 18:57:37.430550 (+ 249us) write_op.cc:270] Start()
0811 18:57:37.430577 (+ 27us) write_op.cc:276] Timestamp: P: 1786474637533284 usec, L: 0
0811 18:57:37.430763 (+ 186us) write_op.cc:270] Start()
0811 18:57:37.430789 (+ 26us) write_op.cc:276] Timestamp: P: 1786474637651573 usec, L: 0
0811 18:57:37.430938 (+ 149us) write_op.cc:270] Start()
0811 18:57:37.430961 (+ 23us) write_op.cc:276] Timestamp: P: 1786474637673259 usec, L: 0
0811 18:57:37.433983 (+ 3022us) write_op.cc:270] Start()
0811 18:57:37.434015 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637691547 usec, L: 0
0811 18:57:37.434194 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.434216 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637774199 usec, L: 0
0811 18:57:37.435209 (+ 993us) write_op.cc:270] Start()
0811 18:57:37.435241 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637782845 usec, L: 0
0811 18:57:37.435412 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.435432 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637806484 usec, L: 0
0811 18:57:37.435563 (+ 131us) write_op.cc:270] Start()
0811 18:57:37.435585 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637820252 usec, L: 0
0811 18:57:37.435737 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.435763 (+ 26us) write_op.cc:276] Timestamp: P: 1786474637820988 usec, L: 0
0811 18:57:37.435897 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.435919 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637854994 usec, L: 0
0811 18:57:37.436126 (+ 207us) write_op.cc:270] Start()
0811 18:57:37.436147 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637874197 usec, L: 0
0811 18:57:37.436311 (+ 164us) write_op.cc:270] Start()
0811 18:57:37.436332 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637891043 usec, L: 0
0811 18:57:37.436462 (+ 130us) write_op.cc:270] Start()
0811 18:57:37.436482 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637939412 usec, L: 0
0811 18:57:37.436614 (+ 132us) write_op.cc:270] Start()
0811 18:57:37.436646 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637942606 usec, L: 0
0811 18:57:37.436772 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.436791 (+ 19us) write_op.cc:276] Timestamp: P: 1786474637966850 usec, L: 0
0811 18:57:37.436937 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.436957 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637980702 usec, L: 0
0811 18:57:37.437109 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.437130 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637998597 usec, L: 0
0811 18:57:37.437275 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.437296 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638017625 usec, L: 0
0811 18:57:37.437404 (+ 108us) write_op.cc:270] Start()
0811 18:57:37.437424 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638131431 usec, L: 0
0811 18:57:37.437552 (+ 128us) write_op.cc:270] Start()
0811 18:57:37.437571 (+ 19us) write_op.cc:276] Timestamp: P: 1786474638169916 usec, L: 0
0811 18:57:37.447267 (+ 9696us) write_op.cc:270] Start()
0811 18:57:37.447331 (+ 64us) write_op.cc:276] Timestamp: P: 1786474638189043 usec, L: 0
0811 18:57:37.447604 (+ 273us) write_op.cc:270] Start()
0811 18:57:37.447646 (+ 42us) write_op.cc:276] Timestamp: P: 1786474638190523 usec, L: 0
0811 18:57:37.447829 (+ 183us) write_op.cc:270] Start()
0811 18:57:37.447853 (+ 24us) write_op.cc:276] Timestamp: P: 1786474638229450 usec, L: 0
0811 18:57:37.448005 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.448028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638242469 usec, L: 0
0811 18:57:37.448167 (+ 139us) write_op.cc:270] Start()
0811 18:57:37.448189 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638273772 usec, L: 0
0811 18:57:37.448331 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.448354 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638293460 usec, L: 0
0811 18:57:37.448494 (+ 140us) write_op.cc:270] Start()
0811 18:57:37.448516 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638303494 usec, L: 0
0811 18:57:37.448670 (+ 154us) write_op.cc:270] Start()
0811 18:57:37.448691 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638328460 usec, L: 0
0811 18:57:37.448833 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.448854 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638355124 usec, L: 0
0811 18:57:37.448989 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.449010 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638356761 usec, L: 0
0811 18:57:37.449151 (+ 141us) write_op.cc:270] Start()
0811 18:57:37.449173 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638384373 usec, L: 0
0811 18:57:37.449310 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.449331 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638385461 usec, L: 0
0811 18:57:37.449453 (+ 122us) write_op.cc:270] Start()
0811 18:57:37.449473 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638386482 usec, L: 0
0811 18:57:37.449636 (+ 163us) write_op.cc:270] Start()
0811 18:57:37.449658 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638387925 usec, L: 0
0811 18:57:37.449791 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.449812 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638425419 usec, L: 0
0811 18:57:37.449954 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.449975 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638505835 usec, L: 0
0811 18:57:37.450133 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.450157 (+ 24us) write_op.cc:276] Timestamp: P: 1786474638520071 usec, L: 0
0811 18:57:37.450302 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.450323 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638545392 usec, L: 0
0811 18:57:37.454595 (+ 4272us) write_op.cc:270] Start()
0811 18:57:37.454634 (+ 39us) write_op.cc:276] Timestamp: P: 1786474638546829 usec, L: 0
0811 18:57:37.454779 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.454801 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638549740 usec, L: 0
0811 18:57:37.455596 (+ 795us) write_op.cc:270] Start()
0811 18:57:37.455644 (+ 48us) write_op.cc:276] Timestamp: P: 1786474638550716 usec, L: 0
0811 18:57:37.455826 (+ 182us) write_op.cc:270] Start()
0811 18:57:37.455847 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638614412 usec, L: 0
0811 18:57:37.456020 (+ 173us) write_op.cc:270] Start()
0811 18:57:37.456043 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638665699 usec, L: 0
0811 18:57:37.456234 (+ 191us) write_op.cc:270] Start()
0811 18:57:37.456262 (+ 28us) write_op.cc:276] Timestamp: P: 1786474638667216 usec, L: 0
0811 18:57:37.456422 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.456442 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638757069 usec, L: 0
0811 18:57:37.456568 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.456587 (+ 19us) write_op.cc:276] Timestamp: P: 1786474638759093 usec, L: 0
0811 18:57:37.460279 (+ 3692us) write_op.cc:270] Start()
0811 18:57:37.460311 (+ 32us) write_op.cc:276] Timestamp: P: 1786474638760108 usec, L: 0
0811 18:57:37.460476 (+ 165us) write_op.cc:270] Start()
0811 18:57:37.460494 (+ 18us) write_op.cc:276] Timestamp: P: 1786474638762390 usec, L: 0
0811 18:57:37.460642 (+ 148us) write_op.cc:270] Start()
0811 18:57:37.460660 (+ 18us) write_op.cc:276] Timestamp: P: 1786474638763339 usec, L: 0
0811 18:57:37.460857 (+ 197us) write_op.cc:270] Start()
0811 18:57:37.460874 (+ 17us) write_op.cc:276] Timestamp: P: 1786474638764255 usec, L: 0
0811 18:57:37.460988 (+ 114us) write_op.cc:270] Start()
0811 18:57:37.461018 (+ 30us) write_op.cc:276] Timestamp: P: 1786474638779863 usec, L: 0
0811 18:57:37.523213 (+ 62195us) write_op.cc:270] Start()
0811 18:57:37.523619 (+ 406us) write_op.cc:276] Timestamp: P: 1786474638801484 usec, L: 0
0811 18:57:37.526558 (+ 2939us) write_op.cc:270] Start()
0811 18:57:37.536976 (+ 10418us) write_op.cc:276] Timestamp: P: 1786474638843754 usec, L: 0
0811 18:57:37.555395 (+ 18419us) write_op.cc:270] Start()
0811 18:57:37.555474 (+ 79us) write_op.cc:276] Timestamp: P: 1786474638887467 usec, L: 0
0811 18:57:37.555802 (+ 328us) write_op.cc:270] Start()
0811 18:57:37.555836 (+ 34us) write_op.cc:276] Timestamp: P: 1786474638891001 usec, L: 0
0811 18:57:37.556035 (+ 199us) write_op.cc:270] Start()
0811 18:57:37.556063 (+ 28us) write_op.cc:276] Timestamp: P: 1786474638899786 usec, L: 0
0811 18:57:37.556288 (+ 225us) write_op.cc:270] Start()
0811 18:57:37.556315 (+ 27us) write_op.cc:276] Timestamp: P: 1786474638950524 usec, L: 0
0811 18:57:37.556491 (+ 176us) write_op.cc:270] Start()
0811 18:57:37.556514 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638957680 usec, L: 0
0811 18:57:37.556651 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.556673 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638975842 usec, L: 0
0811 18:57:37.556793 (+ 120us) write_op.cc:270] Start()
0811 18:57:37.556818 (+ 25us) write_op.cc:276] Timestamp: P: 1786474638981818 usec, L: 0
0811 18:57:37.556941 (+ 123us) write_op.cc:270] Start()
0811 18:57:37.556960 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639000496 usec, L: 0
0811 18:57:37.557086 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.557105 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639035540 usec, L: 0
0811 18:57:37.557251 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.557271 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639056811 usec, L: 0
0811 18:57:37.557404 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.557427 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639063348 usec, L: 0
0811 18:57:37.557541 (+ 114us) write_op.cc:270] Start()
0811 18:57:37.557579 (+ 38us) write_op.cc:276] Timestamp: P: 1786474639097605 usec, L: 0
0811 18:57:37.557727 (+ 148us) write_op.cc:270] Start()
0811 18:57:37.557748 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639103307 usec, L: 0
0811 18:57:37.557864 (+ 116us) write_op.cc:270] Start()
0811 18:57:37.557882 (+ 18us) write_op.cc:276] Timestamp: P: 1786474639127342 usec, L: 0
0811 18:57:37.558012 (+ 130us) write_op.cc:270] Start()
0811 18:57:37.558034 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639133744 usec, L: 0
0811 18:57:37.558167 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.558207 (+ 40us) write_op.cc:276] Timestamp: P: 1786474639154690 usec, L: 0
0811 18:57:37.574443 (+ 16236us) write_op.cc:270] Start()
0811 18:57:37.574499 (+ 56us) write_op.cc:276] Timestamp: P: 1786474639163269 usec, L: 0
0811 18:57:37.574785 (+ 286us) write_op.cc:270] Start()
0811 18:57:37.574809 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639186118 usec, L: 0
0811 18:57:37.574994 (+ 185us) write_op.cc:270] Start()
0811 18:57:37.575025 (+ 31us) write_op.cc:276] Timestamp: P: 1786474639192838 usec, L: 0
0811 18:57:37.575162 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.575183 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639194056 usec, L: 0
0811 18:57:37.575337 (+ 154us) write_op.cc:270] Start()
0811 18:57:37.575359 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639195224 usec, L: 0
0811 18:57:37.575497 (+ 138us) write_op.cc:270] Start()
0811 18:57:37.575518 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639196021 usec, L: 0
0811 18:57:37.579612 (+ 4094us) write_op.cc:270] Start()
0811 18:57:37.579657 (+ 45us) write_op.cc:276] Timestamp: P: 1786474639196830 usec, L: 0
0811 18:57:37.579892 (+ 235us) write_op.cc:270] Start()
0811 18:57:37.579918 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639237770 usec, L: 0
0811 18:57:37.580076 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.580103 (+ 27us) write_op.cc:276] Timestamp: P: 1786474639275400 usec, L: 0
0811 18:57:37.580291 (+ 188us) write_op.cc:270] Start()
0811 18:57:37.580321 (+ 30us) write_op.cc:276] Timestamp: P: 1786474639284916 usec, L: 0
0811 18:57:37.580505 (+ 184us) write_op.cc:270] Start()
0811 18:57:37.580531 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639301118 usec, L: 0
0811 18:57:37.580725 (+ 194us) write_op.cc:270] Start()
0811 18:57:37.580751 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639333396 usec, L: 0
0811 18:57:37.580926 (+ 175us) write_op.cc:270] Start()
0811 18:57:37.580952 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639365993 usec, L: 0
0811 18:57:37.581129 (+ 177us) write_op.cc:270] Start()
0811 18:57:37.581153 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639367848 usec, L: 0
0811 18:57:37.581308 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.581345 (+ 37us) write_op.cc:276] Timestamp: P: 1786474639384193 usec, L: 0
0811 18:57:37.581500 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.581523 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639385587 usec, L: 0
0811 18:57:37.581666 (+ 143us) write_op.cc:270] Start()
0811 18:57:37.581689 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639386926 usec, L: 0
0811 18:57:37.581845 (+ 156us) write_op.cc:270] Start()
0811 18:57:37.581867 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639387893 usec, L: 0
0811 18:57:37.582051 (+ 184us) write_op.cc:270] Start()
0811 18:57:37.582180 (+ 129us) write_op.cc:276] Timestamp: P: 1786474639420510 usec, L: 0
0811 18:57:37.582438 (+ 258us) write_op.cc:270] Start()
0811 18:57:37.582467 (+ 29us) write_op.cc:276] Timestamp: P: 1786474639422108 usec, L: 0
0811 18:57:37.582633 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.582656 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639423440 usec, L: 0
0811 18:57:37.582803 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.582824 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639457762 usec, L: 0
0811 18:57:37.582966 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.582986 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639460913 usec, L: 0
0811 18:57:37.583114 (+ 128us) write_op.cc:270] Start()
0811 18:57:37.583134 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639554903 usec, L: 0
0811 18:57:37.583301 (+ 167us) write_op.cc:270] Start()
0811 18:57:37.583322 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639590645 usec, L: 0
0811 18:57:37.583435 (+ 113us) write_op.cc:270] Start()
0811 18:57:37.583453 (+ 18us) write_op.cc:276] Timestamp: P: 1786474639592106 usec, L: 0
0811 18:57:37.583573 (+ 120us) write_op.cc:270] Start()
0811 18:57:37.583592 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639593286 usec, L: 0
0811 18:57:37.583744 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.583768 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639594797 usec, L: 0
0811 18:57:37.583947 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.583970 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639596454 usec, L: 0
0811 18:57:37.584135 (+ 165us) write_op.cc:270] Start()
0811 18:57:37.584163 (+ 28us) write_op.cc:276] Timestamp: P: 1786474639598451 usec, L: 0
0811 18:57:37.584337 (+ 174us) write_op.cc:270] Start()
0811 18:57:37.584362 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639665261 usec, L: 0
0811 18:57:37.584515 (+ 153us) write_op.cc:270] Start()
0811 18:57:37.584540 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639700714 usec, L: 0
0811 18:57:37.584713 (+ 173us) write_op.cc:270] Start()
0811 18:57:37.584738 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639702150 usec, L: 0
0811 18:57:37.584889 (+ 151us) write_op.cc:270] Start()
0811 18:57:37.584918 (+ 29us) write_op.cc:276] Timestamp: P: 1786474639703880 usec, L: 0
0811 18:57:37.585097 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.585117 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639705100 usec, L: 0
0811 18:57:37.585264 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.585287 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639760705 usec, L: 0
0811 18:57:37.585442 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.585466 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639784262 usec, L: 0
0811 18:57:37.585624 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.585648 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639786061 usec, L: 0
0811 18:57:37.585836 (+ 188us) write_op.cc:270] Start()
0811 18:57:37.585863 (+ 27us) write_op.cc:276] Timestamp: P: 1786474639812909 usec, L: 0
0811 18:57:37.586031 (+ 168us) write_op.cc:270] Start()
0811 18:57:37.586055 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639846001 usec, L: 0
0811 18:57:37.586246 (+ 191us) write_op.cc:270] Start()
0811 18:57:37.586271 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639848186 usec, L: 0
0811 18:57:37.618609 (+ 32338us) write_op.cc:270] Start()
0811 18:57:37.618660 (+ 51us) write_op.cc:276] Timestamp: P: 1786474639849712 usec, L: 0
0811 18:57:37.618910 (+ 250us) write_op.cc:270] Start()
0811 18:57:37.618940 (+ 30us) write_op.cc:276] Timestamp: P: 1786474639905300 usec, L: 0
0811 18:57:37.619109 (+ 169us) write_op.cc:270] Start()
0811 18:57:37.619135 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639923827 usec, L: 0
0811 18:57:37.619307 (+ 172us) write_op.cc:270] Start()
0811 18:57:37.619332 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639925756 usec, L: 0
0811 18:57:37.619491 (+ 159us) write_op.cc:270] Start()
0811 18:57:37.619515 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639927112 usec, L: 0
0811 18:57:37.619653 (+ 138us) write_op.cc:270] Start()
0811 18:57:37.619678 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639928390 usec, L: 0
0811 18:57:37.619812 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.619834 (+ 22us) write_op.cc:276] Timestamp: P: 1786474640021864 usec, L: 0
0811 18:57:37.619980 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.620004 (+ 24us) write_op.cc:276] Timestamp: P: 17864
I20260811 18:57:39.048919 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d08de21434edc86f0cab49f839294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.054479 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8072b2d13cbc45249f43411e9022d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.055114 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f52a745f6243ae9b1ef2dd33b06d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.079494 5622 consensus_queue.cc:1035] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:39.102599 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1060 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:39.108290 5557 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.108477 5557 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:39.109071 5557 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.111642 4169 ts_tablet_manager.cc:1905] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.116914 3678 consensus_peers.cc:487] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.117565 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1ccd56a8b7cc454ea3f2f08b59374521
I20260811 18:57:39.118507 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 78868ff1f8024163ae1d782151100cd4
I20260811 18:57:39.122993 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c4b27a9eb0994efe9f4176f74c16a62e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.124233 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" } modify_external_catalogs: true
I20260811 18:57:39.124541 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" } modify_external_catalogs: true
I20260811 18:57:39.126206 5557 raft_consensus.cc:482] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.126528 5557 raft_consensus.cc:504] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.127195 5557 leader_election.cc:288] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:39.128374 4162 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.128595 4162 ts_tablet_manager.cc:1397] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.239s user 0.004s sys 0.000s
I20260811 18:57:39.129561 4162 raft_consensus.cc:348] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.129908 4162 raft_consensus.cc:374] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.130028 4162 raft_consensus.cc:729] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:39.130053 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4b27a9eb0994efe9f4176f74c16a62e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.130455 4162 consensus_queue.cc:260] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.131199 4162 ts_tablet_manager.cc:1428] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.136062 4162 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.138197 4162 tablet_bootstrap.cc:654] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.138846 4163 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.140586 4163 tablet_bootstrap.cc:654] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.142481 3824 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:39.143096 5646 consensus_queue.cc:1035] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:39.143781 3824 leader_election.cc:341] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0cccebfc54484a9b9abffcce4e459fb5
W20260811 18:57:39.151459 3678 consensus_peers.cc:487] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.159533 4162 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.159957 4162 ts_tablet_manager.cc:1397] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.002s
I20260811 18:57:39.161032 4162 raft_consensus.cc:348] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.161381 4162 raft_consensus.cc:374] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.161502 4162 raft_consensus.cc:729] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.161850 4162 consensus_queue.cc:260] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.162703 4162 ts_tablet_manager.cc:1428] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:39.163074 4162 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.163576 4167 ts_tablet_manager.cc:1918] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.163724 4167 log.cc:1196] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:39.164062 4167 ts_tablet_manager.cc:1939] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.164932 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1066) took 3139 ms. Trace:
I20260811 18:57:39.165046 4167 rpcz_store.cc:276] 0811 18:57:36.025985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.026058 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:38.814510 (+2788452us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:38.814522 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.012264 (+197742us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.163424 (+151160us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.164909 (+ 1485us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2788206}
I20260811 18:57:39.165527 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1076) took 2978 ms. Trace:
W20260811 18:57:39.169234 3824 leader_election.cc:341] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7057b8a7bae249caaed304b5c66e6871
W20260811 18:57:39.169797 3824 leader_election.cc:341] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:39.173426 5264 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" instance_seqno: 1786474620633424) as {username='slave'} at 127.3.72.66:42153; Asking this server to re-register.
I20260811 18:57:39.174685 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f7d4c26da86b49b8a37f2289f4ef6614 (table default.Table_523277d801974e708b0f7e9c401bb36a [id=5600dcaaf83c4f978c3d47f79e01568c]) successfully deleted
I20260811 18:57:39.175380 4163 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.175519 4163 ts_tablet_manager.cc:1397] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.004s
I20260811 18:57:39.176290 4163 raft_consensus.cc:348] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.176525 4163 raft_consensus.cc:374] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.176584 4163 raft_consensus.cc:729] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.176785 4163 consensus_queue.cc:260] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:39.177826 3824 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.177935 3824 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:39.178758 3824 leader_election.cc:334] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.178867 3824 leader_election.cc:302] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.179060 5593 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.179241 5593 raft_consensus.cc:2726] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.181885 3939 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.188386 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1060 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:39.189359 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:39.189913 5617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:39.190327 5407 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.190521 5407 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.191200 5407 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.193131 4163 ts_tablet_manager.cc:1428] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.018s user 0.000s sys 0.002s
I20260811 18:57:39.165642 4167 rpcz_store.cc:276] 0811 18:57:36.187397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.187448 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:39.165394 (+2977946us) ts_tablet_manager.cc:1145] Deleting tablet 1700ee40ac0d40198c5b8a161a90b6b6
0811 18:57:39.165405 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.165510 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2977730}
I20260811 18:57:39.195429 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1080) took 3007 ms. Trace:
I20260811 18:57:39.195592 4167 rpcz_store.cc:276] 0811 18:57:36.187788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.188574 (+ 786us) service_pool.cc:227] Handling call
0811 18:57:39.195299 (+3006725us) ts_tablet_manager.cc:1145] Deleting tablet 374c94a12e374d0e82180421c80d92d0
0811 18:57:39.195311 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.195408 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3006538}
I20260811 18:57:39.196058 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1079) took 3008 ms. Trace:
I20260811 18:57:39.196226 4167 rpcz_store.cc:276] 0811 18:57:36.187681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.188633 (+ 952us) service_pool.cc:227] Handling call
0811 18:57:39.195955 (+3007322us) ts_tablet_manager.cc:1145] Deleting tablet 38a4783168554053aa953dba5e893b6b
0811 18:57:39.195965 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.196041 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22,"tablet-delete.queue_time_us":3007182}
I20260811 18:57:39.196667 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1088) took 2850 ms. Trace:
I20260811 18:57:39.196820 4163 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.196805 4167 rpcz_store.cc:276] 0811 18:57:36.346290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.346400 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:39.196551 (+2850151us) ts_tablet_manager.cc:1145] Deleting tablet 60091bd891a94cc4aac6ef47673a4ee5
0811 18:57:39.196562 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.196649 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2849864}
I20260811 18:57:39.197111 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1089) took 2849 ms. Trace:
I20260811 18:57:39.197196 4167 rpcz_store.cc:276] 0811 18:57:36.348102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.348172 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:39.197018 (+2848846us) ts_tablet_manager.cc:1145] Deleting tablet fe080f2071964da6b54f1ee5af98d918
0811 18:57:39.197027 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.197094 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2848604}
I20260811 18:57:39.197449 4167 tablet_replica.cc:330] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.197664 4167 raft_consensus.cc:2229] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.197829 4167 raft_consensus.cc:2258] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.198414 4167 ts_tablet_manager.cc:1905] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.199652 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.199797 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.200838 3678 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.203603 5266 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" instance_seqno: 1786474620110993) as {username='slave'} at 127.3.72.65:38671; Asking this server to re-register.
I20260811 18:57:39.207625 3795 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.208945 4163 tablet_bootstrap.cc:654] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.211526 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0b4fff40ba9d49828facafd431939fd4 (DEFAULT_TABLE table=default.Table_e8dcb132b5b5474b8ff081e54a55dd3f [id=58e6357b58b4481c9fad17e3aa498147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.211964 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4fff40ba9d49828facafd431939fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.212535 5266 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.210564 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.211107 3867 tablet_service.cc:1467] Processing CreateTablet for tablet e4441c724b394e8092fb19c0d2427233 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.217983 5407 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.218513 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4441c724b394e8092fb19c0d2427233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.218371 5407 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.219100 5407 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.219853 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1e51bb756dff43caace2c0626aed1443 (DEFAULT_TABLE table=default.Table_1afb343a7aa84568ad8ad65cab128123 [id=1fa01fbb7a4946da82085dab1e0a71ae]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:39.219872 3676 leader_election.cc:341] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.220014 3676 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.220459 5407 raft_consensus.cc:2726] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.220443 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.221073 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.221120 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 78868ff1f8024163ae1d782151100cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.221762 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:39.221985 3676 leader_election.cc:341] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.222465 3824 leader_election.cc:334] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.222632 3824 leader_election.cc:302] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.222916 5630 raft_consensus.cc:2726] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.223474 4169 ts_tablet_manager.cc:1918] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.226990 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e51bb756dff43caace2c0626aed1443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.234655 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 19243ab3d6744aaa87a133f1462cd2fa (table default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]) successfully deleted
I20260811 18:57:39.234936 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e3021c51525846e199f279f988248af0 (table default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]) successfully deleted
I20260811 18:57:39.235183 4169 log.cc:1196] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/45fde25bcb40493aafb083e161aa6932
I20260811 18:57:39.235440 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]) successfully deleted
I20260811 18:57:39.235797 4169 ts_tablet_manager.cc:1939] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.235877 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]) successfully deleted
I20260811 18:57:39.236066 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 27cda53b7ca54346a7a22bf26f5db990 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
W20260811 18:57:39.236322 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1700ee40ac0d40198c5b8a161a90b6b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1700ee40ac0d40198c5b8a161a90b6b6 already in progress: creating tablet
W20260811 18:57:39.236541 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 374c94a12e374d0e82180421c80d92d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374c94a12e374d0e82180421c80d92d0 already in progress: creating tablet
I20260811 18:57:39.236558 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 963) took 4264 ms. Trace:
W20260811 18:57:39.236760 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 38a4783168554053aa953dba5e893b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38a4783168554053aa953dba5e893b6b already in progress: creating tablet
W20260811 18:57:39.236961 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 60091bd891a94cc4aac6ef47673a4ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60091bd891a94cc4aac6ef47673a4ee5 already in progress: creating tablet
I20260811 18:57:39.236799 4169 rpcz_store.cc:276] 0811 18:57:34.972256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.976524 (+ 4268us) service_pool.cc:227] Handling call
0811 18:57:39.029210 (+4052686us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:39.029226 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.167064 (+137838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.223364 (+ 56300us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.236536 (+ 13172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4052422}
I20260811 18:57:39.237423 4169 tablet_replica.cc:330] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.237715 4169 raft_consensus.cc:2229] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.237977 4169 raft_consensus.cc:2258] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.239013 4169 ts_tablet_manager.cc:1905] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.243861 3678 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.244064 3678 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.244354 5407 raft_consensus.cc:2726] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.248680 4162 tablet_bootstrap.cc:654] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.248663 3824 leader_election.cc:334] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.248833 3824 leader_election.cc:302] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.249106 5630 raft_consensus.cc:2726] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:39.249518 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.253810 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.258038 3824 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.257717 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 33a79243bd874861b16f43339adf777c (DEFAULT_TABLE table=default.Table_80140471fa194b22bf8347836f95fdca [id=6de24db40cfe4fb49177dedc3d64443b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.261942 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a79243bd874861b16f43339adf777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.266049 4167 ts_tablet_manager.cc:1918] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.267678 4163 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:39.262578 3824 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7251204325dd424192a6f58e18e71c34
I20260811 18:57:39.270314 4163 ts_tablet_manager.cc:1397] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.007s sys 0.003s
I20260811 18:57:39.270476 3824 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.267521 4162 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.274148 4162 ts_tablet_manager.cc:1397] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.111s user 0.004s sys 0.005s
I20260811 18:57:39.274148 5630 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.274241 4163 raft_consensus.cc:348] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.275071 4162 raft_consensus.cc:348] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.275370 4162 raft_consensus.cc:374] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.275475 4162 raft_consensus.cc:729] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:39.275580 3824 leader_election.cc:341] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:39.275864 4162 consensus_queue.cc:260] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:39.276031 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe080f2071964da6b54f1ee5af98d918 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe080f2071964da6b54f1ee5af98d918 already in progress: creating tablet
I20260811 18:57:39.276592 4162 ts_tablet_manager.cc:1428] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.274501 4163 raft_consensus.cc:374] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.279299 4162 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.279484 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 6168d1f0fffc4fad91581d1795d721ce (DEFAULT_TABLE table=default.Table_5ef097030b2146bb9df7f7d2bffde6b6 [id=0291bde7daf64dc480710af744852acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.279906 4163 raft_consensus.cc:729] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.279994 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6168d1f0fffc4fad91581d1795d721ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.280221 4163 consensus_queue.cc:260] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.280807 4163 ts_tablet_manager.cc:1428] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:39.281193 4162 tablet_bootstrap.cc:654] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.278366 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.282240 4167 log.cc:1196] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:39.282953 4167 ts_tablet_manager.cc:1939] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.284665 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1090) took 2449 ms. Trace:
I20260811 18:57:39.286119 4163 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.286422 4167 rpcz_store.cc:276] 0811 18:57:36.835209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.835663 (+ 454us) service_pool.cc:227] Handling call
0811 18:57:39.197410 (+2361747us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:39.197418 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.246946 (+ 49528us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.265921 (+ 18975us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.284639 (+ 18718us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2359154}
I20260811 18:57:39.287200 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1093) took 2433 ms. Trace:
I20260811 18:57:39.289566 4163 tablet_bootstrap.cc:654] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.290100 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 45fde25bcb40493aafb083e161aa6932 (table default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c [id=4bc5d2d6894e40ebab164e5ffd6ac600]) successfully deleted
I20260811 18:57:39.290355 4167 rpcz_store.cc:276] 0811 18:57:36.854096 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.854798 (+ 702us) service_pool.cc:227] Handling call
0811 18:57:39.287054 (+2432256us) ts_tablet_manager.cc:1145] Deleting tablet 03c50dcffac54c7bac210eeaed340ca6
0811 18:57:39.287067 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.287181 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2432050}
I20260811 18:57:39.291034 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1100) took 2337 ms. Trace:
I20260811 18:57:39.294391 4167 rpcz_store.cc:276] 0811 18:57:36.954043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.954112 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:39.290908 (+2336796us) ts_tablet_manager.cc:1145] Deleting tablet 73c519528f75423195b6bdf6683fd91b
0811 18:57:39.290921 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.291016 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2336560}
I20260811 18:57:39.295115 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1101) took 2334 ms. Trace:
I20260811 18:57:39.295279 4167 rpcz_store.cc:276] 0811 18:57:36.960813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.960902 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:39.294975 (+2334073us) ts_tablet_manager.cc:1145] Deleting tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
0811 18:57:39.294988 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.295096 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2333853}
I20260811 18:57:39.295997 4167 tablet_replica.cc:330] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.296476 4167 raft_consensus.cc:2229] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:39.297240 4167 raft_consensus.cc:2258] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.298910 4167 ts_tablet_manager.cc:1905] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.298822 3824 leader_election.cc:334] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.298249 4162 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.303004 3824 leader_election.cc:302] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.303392 5557 raft_consensus.cc:2726] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.303527 4162 ts_tablet_manager.cc:1397] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
W20260811 18:57:39.303752 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.306247 4162 raft_consensus.cc:348] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.306645 4162 raft_consensus.cc:374] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.306774 4162 raft_consensus.cc:729] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.307089 4162 consensus_queue.cc:260] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.308158 4162 ts_tablet_manager.cc:1428] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:39.308511 4162 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:39.309760 3824 consensus_peers.cc:487] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:39.312929 4169 ts_tablet_manager.cc:1918] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.313158 4169 log.cc:1196] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:39.313560 4169 ts_tablet_manager.cc:1939] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.313927 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 964) took 4337 ms. Trace:
I20260811 18:57:39.314149 4162 tablet_bootstrap.cc:654] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.314030 4169 rpcz_store.cc:276] 0811 18:57:34.976904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.977365 (+ 461us) service_pool.cc:227] Handling call
0811 18:57:39.237371 (+4260006us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:39.237383 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.272633 (+ 35250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.311819 (+ 39186us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.313905 (+ 2086us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4259808}
I20260811 18:57:39.314517 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 969) took 4327 ms. Trace:
I20260811 18:57:39.314617 4169 rpcz_store.cc:276] 0811 18:57:34.987152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.988817 (+ 1665us) service_pool.cc:227] Handling call
0811 18:57:39.314366 (+4325549us) ts_tablet_manager.cc:1145] Deleting tablet 0bfb54175c834517b7af291c942333cd
0811 18:57:39.314381 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.314497 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4325326}
I20260811 18:57:39.314965 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 971) took 4324 ms. Trace:
I20260811 18:57:39.315052 4169 rpcz_store.cc:276] 0811 18:57:34.990014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.994866 (+ 4852us) service_pool.cc:227] Handling call
0811 18:57:39.314854 (+4319988us) ts_tablet_manager.cc:1145] Deleting tablet 1c1896700e724204a37439f134a4d988
0811 18:57:39.314865 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.314946 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319778}
I20260811 18:57:39.315390 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 972) took 4320 ms. Trace:
I20260811 18:57:39.315476 4169 rpcz_store.cc:276] 0811 18:57:34.995293 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.995495 (+ 202us) service_pool.cc:227] Handling call
0811 18:57:39.315282 (+4319787us) ts_tablet_manager.cc:1145] Deleting tablet 4dd203b7eb8643f493b4642c0396e258
0811 18:57:39.315294 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.315374 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319574}
I20260811 18:57:39.315785 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 973) took 4319 ms. Trace:
I20260811 18:57:39.315862 4169 rpcz_store.cc:276] 0811 18:57:34.995983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.996027 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:39.315696 (+4319669us) ts_tablet_manager.cc:1145] Deleting tablet afa57ed239ac4afebe517cc48d879645
0811 18:57:39.315705 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.315771 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319454}
I20260811 18:57:39.316191 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 974) took 4319 ms. Trace:
I20260811 18:57:39.316294 4169 rpcz_store.cc:276] 0811 18:57:34.996781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.996832 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:39.316063 (+4319231us) ts_tablet_manager.cc:1145] Deleting tablet b27f9dbb52f34687a33c2ebf20e8f780
0811 18:57:39.316092 (+ 29us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.316175 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319064}
I20260811 18:57:39.316615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 989) took 4248 ms. Trace:
I20260811 18:57:39.316709 4169 rpcz_store.cc:276] 0811 18:57:35.068216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.068271 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.316519 (+4248248us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:39.316530 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.316599 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4247993}
I20260811 18:57:39.317068 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 990) took 4248 ms. Trace:
I20260811 18:57:39.317162 4169 rpcz_store.cc:276] 0811 18:57:35.068664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.068719 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.316945 (+4248226us) ts_tablet_manager.cc:1145] Deleting tablet f86bfa65a8774b51bc7e7fe59442bf2f
0811 18:57:39.316956 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.317047 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4248050}
I20260811 18:57:39.317448 4169 tablet_replica.cc:330] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.317986 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (table default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]) successfully deleted
W20260811 18:57:39.318272 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bfb54175c834517b7af291c942333cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bfb54175c834517b7af291c942333cd already in progress: creating tablet
W20260811 18:57:39.318473 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1c1896700e724204a37439f134a4d988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c1896700e724204a37439f134a4d988 already in progress: creating tablet
W20260811 18:57:39.318676 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4dd203b7eb8643f493b4642c0396e258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4dd203b7eb8643f493b4642c0396e258 already in progress: creating tablet
W20260811 18:57:39.318818 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet afa57ed239ac4afebe517cc48d879645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa57ed239ac4afebe517cc48d879645 already in progress: creating tablet
W20260811 18:57:39.318957 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b27f9dbb52f34687a33c2ebf20e8f780 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b27f9dbb52f34687a33c2ebf20e8f780 already in progress: creating tablet
W20260811 18:57:39.319079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d37e81d376e14907b13746cca831a96b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d37e81d376e14907b13746cca831a96b already in progress: creating tablet
W20260811 18:57:39.319223 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f86bfa65a8774b51bc7e7fe59442bf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f86bfa65a8774b51bc7e7fe59442bf2f already in progress: creating tablet
I20260811 18:57:39.322161 5593 consensus_queue.cc:1035] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:39.325944 4167 ts_tablet_manager.cc:1918] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:39.328663 4169 raft_consensus.cc:2229] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.328896 4169 raft_consensus.cc:2258] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.333117 4169 ts_tablet_manager.cc:1905] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.328980 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 071a6513025341caaa0cac048d3caccd (DEFAULT_TABLE table=default.Table_b882f1fc510a41558369a5ced806d29a [id=d744d4b03cb64b5e8eb2b02f7387733b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.335388 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071a6513025341caaa0cac048d3caccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.339212 3723 tablet_service.cc:1467] Processing CreateTablet for tablet fa49814fb45f481a9c16e30ea947c79a (DEFAULT_TABLE table=default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb [id=22079dd74afc459081d1eb743f58c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.339651 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa49814fb45f481a9c16e30ea947c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.341440 4163 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.341683 4163 ts_tablet_manager.cc:1397] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.002s
I20260811 18:57:39.341777 4167 log.cc:1196] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3ad5f39f59b8447cb3f15b9646e4b852
W20260811 18:57:39.342128 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 3247 similar messages]
I20260811 18:57:39.326694 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 49203e259d17449984dcd528df985537 (DEFAULT_TABLE table=default.Table_d2326f1112004b75bdb9d55298314884 [id=9833505c53af4634911465bd22485158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.329835 3710 tablet_service.cc:1467] Processing CreateTablet for tablet f10006748b084a6bbd1a2f6a55cca9b9 (DEFAULT_TABLE table=default.Table_35bcb0c44a4840879a1f1d1386bf24c1 [id=955bf462395b45a3b6fea1fce8dad61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.330188 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a7946503bec9446d8deb02448eb231d5 (DEFAULT_TABLE table=default.Table_c925e9ac591346b1a5142f338c68bcc6 [id=c5d66f3cc70d498eb0db5c57bb06442d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.330600 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd (DEFAULT_TABLE table=default.Table_31dac67fa2c146b0956b5a3b25672da3 [id=4d02195d37e64519a56460e55acf9ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.331035 3722 tablet_service.cc:1467] Processing CreateTablet for tablet de822817a9634958ba0d1a3128691dba (DEFAULT_TABLE table=default.Table_5dc02b59b48d45c8bee6458f21962eeb [id=7ee28ea901454542818bc026827867b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.342577 4163 raft_consensus.cc:348] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.342784 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10006748b084a6bbd1a2f6a55cca9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.342804 4163 raft_consensus.cc:374] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.342857 4163 raft_consensus.cc:729] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.343102 4163 consensus_queue.cc:260] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.343672 4163 ts_tablet_manager.cc:1428] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:39.343948 4163 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.345706 4163 tablet_bootstrap.cc:654] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.347054 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.347261 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7946503bec9446d8deb02448eb231d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.348858 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de822817a9634958ba0d1a3128691dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.351104 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1365655a76d7454c8415102e75960152" } new_table_name: "default.Table_58a314da9a724012bc84997945c98047" modify_external_catalogs: true
I20260811 18:57:39.352777 5659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:39.353015 5659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:39.355006 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:39.335007 4162 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.355544 4162 ts_tablet_manager.cc:1397] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.000s
I20260811 18:57:39.356257 4162 raft_consensus.cc:348] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.356444 4162 raft_consensus.cc:374] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.356487 4162 raft_consensus.cc:729] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.356732 4162 consensus_queue.cc:260] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.357780 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5affb8acc7e5466ba5ed988728ba50cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.358718 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f271e4958af4bfbb65af29a053ac2d4" } new_table_name: "default.Table_44bbdf86ad5b4ba4a43a3ff0819dbeca" modify_external_catalogs: true
I20260811 18:57:39.360396 4162 ts_tablet_manager.cc:1428] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:39.360685 4162 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.362399 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49203e259d17449984dcd528df985537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.362915 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8
I20260811 18:57:39.363829 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbebdb613fa84efc9722613af36d52c8" } new_table_name: "default.Table_51582df654e7406dbc574ba8de215baf" modify_external_catalogs: true
I20260811 18:57:39.368029 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f6cd18a5e464333bbdc36290a29eaea" } new_table_name: "default.Table_69d238eee5ac443da8e2fa5a011c42c8" modify_external_catalogs: true
W20260811 18:57:39.370103 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5affb8acc7e5466ba5ed988728ba50cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.370589 4162 tablet_bootstrap.cc:654] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.370748 4167 ts_tablet_manager.cc:1939] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.372406 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.372558 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1102) took 2411 ms. Trace:
I20260811 18:57:39.375783 5641 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.380841 4167 rpcz_store.cc:276] 0811 18:57:36.961001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.982466 (+ 21465us) service_pool.cc:227] Handling call
0811 18:57:39.295948 (+2313482us) ts_tablet_manager.cc:1145] Deleting tablet 3ad5f39f59b8447cb3f15b9646e4b852
0811 18:57:39.295960 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.322798 (+ 26838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.325841 (+ 3043us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.372535 (+ 46694us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2313030}
I20260811 18:57:39.381049 5641 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.381481 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1103) took 2317 ms. Trace:
I20260811 18:57:39.381575 4167 rpcz_store.cc:276] 0811 18:57:37.064490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.068157 (+ 3667us) service_pool.cc:227] Handling call
0811 18:57:39.381317 (+2313160us) ts_tablet_manager.cc:1145] Deleting tablet 7cb3e997b4454598bdce102e92296887
0811 18:57:39.381332 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.381458 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2312937}
I20260811 18:57:39.382061 5641 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:39.382126 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1104) took 2281 ms. Trace:
I20260811 18:57:39.382205 4167 rpcz_store.cc:276] 0811 18:57:37.100957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.101349 (+ 392us) service_pool.cc:227] Handling call
0811 18:57:39.382016 (+2280667us) ts_tablet_manager.cc:1145] Deleting tablet 8f9b279c6c7344f1931b81e10248b1b6
0811 18:57:39.382029 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.382110 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2269657}
I20260811 18:57:39.383670 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.384356 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1122) took 2195 ms. Trace:
I20260811 18:57:39.384465 4167 rpcz_store.cc:276] 0811 18:57:37.188944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.188997 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:39.384247 (+2195250us) ts_tablet_manager.cc:1145] Deleting tablet ea579c33f8b14e9fa7fa389ce7f45931
0811 18:57:39.384259 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.384339 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2195065}
W20260811 18:57:39.384590 3676 leader_election.cc:341] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.388573 3678 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.389096 3678 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.389369 5641 raft_consensus.cc:2726] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.389849 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1121) took 2201 ms. Trace:
I20260811 18:57:39.390380 4167 rpcz_store.cc:276] 0811 18:57:37.188849 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.189059 (+ 210us) service_pool.cc:227] Handling call
0811 18:57:39.389728 (+2200669us) ts_tablet_manager.cc:1145] Deleting tablet 7f4e1fc5686944dfa3eb57702c810eb8
0811 18:57:39.389739 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.389833 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2200476}
I20260811 18:57:39.392184 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1132) took 2201 ms. Trace:
I20260811 18:57:39.393147 4169 ts_tablet_manager.cc:1918] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.393317 4169 log.cc:1196] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:39.393324 4167 rpcz_store.cc:276] 0811 18:57:37.190996 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.191045 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:39.392063 (+2201018us) ts_tablet_manager.cc:1145] Deleting tablet d21973bc8e514a089c2beb24a686060f
0811 18:57:39.392076 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.392166 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2200855}
I20260811 18:57:39.393705 4169 ts_tablet_manager.cc:1939] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.394024 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1133) took 2202 ms. Trace:
I20260811 18:57:39.394194 4167 rpcz_store.cc:276] 0811 18:57:37.191432 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.191478 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:39.393911 (+2202433us) ts_tablet_manager.cc:1145] Deleting tablet 2f2f5ce2b1d14a9a988cff644e196088
0811 18:57:39.393922 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.394007 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2202148}
I20260811 18:57:39.394682 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet dd80633f926942e88f54b32d9a03ea3a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.395012 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1134) took 2191 ms. Trace:
I20260811 18:57:39.395363 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9042bc231f0841a4a370be5e6251dd6e" } modify_external_catalogs: true
I20260811 18:57:39.395581 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9042bc231f0841a4a370be5e6251dd6e" } modify_external_catalogs: true
I20260811 18:57:39.395849 4167 rpcz_store.cc:276] 0811 18:57:37.204042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.204450 (+ 408us) service_pool.cc:227] Handling call
0811 18:57:39.394913 (+2190463us) ts_tablet_manager.cc:1145] Deleting tablet f2b66196716a4ba3893b6b8bfec2b8c0
0811 18:57:39.394923 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.394997 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2190268}
I20260811 18:57:39.398324 4163 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.398576 4163 ts_tablet_manager.cc:1397] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20260811 18:57:39.399188 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 993) took 4317 ms. Trace:
I20260811 18:57:39.399319 4169 rpcz_store.cc:276] 0811 18:57:35.081574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.081644 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:39.317401 (+4235757us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:39.317411 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.383686 (+ 66275us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.393017 (+ 9331us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.399162 (+ 6145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4235494}
I20260811 18:57:39.399471 4163 raft_consensus.cc:348] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.399884 4163 raft_consensus.cc:374] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.400002 4163 raft_consensus.cc:729] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.400235 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6432abd4fc5c4f9ca95971ca4795e1cf (table default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:39.400323 4163 consensus_queue.cc:260] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.400909 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1135) took 2018 ms. Trace:
I20260811 18:57:39.400945 4169 tablet_replica.cc:330] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.401054 4163 ts_tablet_manager.cc:1428] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.401407 4163 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.401429 4167 rpcz_store.cc:276] 0811 18:57:37.382084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.386399 (+ 4315us) service_pool.cc:227] Handling call
0811 18:57:39.400765 (+2014366us) ts_tablet_manager.cc:1145] Deleting tablet a3973dfdaac746a6af7a402277d2711b
0811 18:57:39.400779 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.400887 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2012295}
I20260811 18:57:39.402305 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1138) took 2015 ms. Trace:
I20260811 18:57:39.402537 4169 raft_consensus.cc:2229] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.402709 4169 raft_consensus.cc:2258] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.402866 4167 rpcz_store.cc:276] 0811 18:57:37.386929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.387035 (+ 106us) service_pool.cc:227] Handling call
0811 18:57:39.402170 (+2015135us) ts_tablet_manager.cc:1145] Deleting tablet 15dd14e2b4bd4a08b5b08f532a55291b
0811 18:57:39.402185 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.402285 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2013626}
I20260811 18:57:39.403237 4163 tablet_bootstrap.cc:654] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.403265 4169 ts_tablet_manager.cc:1905] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.407151 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1153) took 1872 ms. Trace:
I20260811 18:57:39.407578 4167 rpcz_store.cc:276] 0811 18:57:37.534574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.534640 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:39.407027 (+1872387us) ts_tablet_manager.cc:1145] Deleting tablet 75ceba7bfc8240268b16c789f07813d1
0811 18:57:39.407039 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.407133 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1872120}
I20260811 18:57:39.408361 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1152) took 1873 ms. Trace:
I20260811 18:57:39.408722 4167 rpcz_store.cc:276] 0811 18:57:37.534458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.534616 (+ 158us) service_pool.cc:227] Handling call
0811 18:57:39.408255 (+1873639us) ts_tablet_manager.cc:1145] Deleting tablet 0495c5b770b34d54afb1379a6e36204c
0811 18:57:39.408267 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.408346 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1873109}
I20260811 18:57:39.409394 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1154) took 1874 ms. Trace:
I20260811 18:57:39.411907 4169 ts_tablet_manager.cc:1918] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.412057 4169 log.cc:1196] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/58c515c153df4317b3fb427983ee15e0
I20260811 18:57:39.412398 4169 ts_tablet_manager.cc:1939] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.412434 4167 rpcz_store.cc:276] 0811 18:57:37.535057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.535100 (+ 43us) service_pool.cc:227] Handling call
0811 18:57:39.409302 (+1874202us) ts_tablet_manager.cc:1145] Deleting tablet 5a75a480f38047d880b0c1fa51403786
0811 18:57:39.409312 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.409381 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1874043}
I20260811 18:57:39.413249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 992) took 4331 ms. Trace:
I20260811 18:57:39.413360 4169 rpcz_store.cc:276] 0811 18:57:35.081452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.082144 (+ 692us) service_pool.cc:227] Handling call
0811 18:57:39.400894 (+4318750us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:39.400908 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.408103 (+ 7195us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.411800 (+ 3697us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.413207 (+ 1407us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1352,"tablet-delete.queue_time_us":4318564}
I20260811 18:57:39.413833 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 58c515c153df4317b3fb427983ee15e0 (table default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]) successfully deleted
I20260811 18:57:39.413950 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 999) took 4068 ms. Trace:
I20260811 18:57:39.414032 4169 rpcz_store.cc:276] 0811 18:57:35.345421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346431 (+ 1010us) service_pool.cc:227] Handling call
0811 18:57:39.413817 (+4067386us) ts_tablet_manager.cc:1145] Deleting tablet 36861ae786c549d687e687fe853d1801
0811 18:57:39.413832 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.413932 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4067213}
W20260811 18:57:39.414431 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 36861ae786c549d687e687fe853d1801 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36861ae786c549d687e687fe853d1801 already in progress: creating tablet
I20260811 18:57:39.414529 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1000) took 4069 ms. Trace:
I20260811 18:57:39.414605 4169 rpcz_store.cc:276] 0811 18:57:35.345533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346642 (+ 1109us) service_pool.cc:227] Handling call
0811 18:57:39.414426 (+4067784us) ts_tablet_manager.cc:1145] Deleting tablet 46505ba544104458888dbc2a37115dcc
0811 18:57:39.414437 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.414514 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4067600}
W20260811 18:57:39.414975 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 46505ba544104458888dbc2a37115dcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46505ba544104458888dbc2a37115dcc already in progress: creating tablet
I20260811 18:57:39.415006 4169 tablet_replica.cc:330] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.415218 4169 raft_consensus.cc:2229] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.415395 4169 raft_consensus.cc:2258] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.416013 4169 ts_tablet_manager.cc:1905] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.417165 5557 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.417315 5557 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.417932 5557 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.420426 3824 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.422120 4162 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.422331 4162 ts_tablet_manager.cc:1397] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.062s user 0.001s sys 0.009s
I20260811 18:57:39.424041 4162 raft_consensus.cc:348] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.424355 4162 raft_consensus.cc:374] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.424466 4162 raft_consensus.cc:729] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.424772 4162 consensus_queue.cc:260] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.425570 4162 ts_tablet_manager.cc:1428] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:39.426635 4169 ts_tablet_manager.cc:1918] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.426815 4169 log.cc:1196] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7251204325dd424192a6f58e18e71c34
I20260811 18:57:39.427145 4169 ts_tablet_manager.cc:1939] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.429563 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 996) took 4085 ms. Trace:
I20260811 18:57:39.429766 4169 rpcz_store.cc:276] 0811 18:57:35.344396 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344459 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:39.414963 (+4070504us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:39.414973 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.422982 (+ 8009us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.426530 (+ 3548us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.429542 (+ 3012us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4066895}
I20260811 18:57:39.430203 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7251204325dd424192a6f58e18e71c34 (table default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]) successfully deleted
I20260811 18:57:39.430393 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 997) took 4085 ms. Trace:
I20260811 18:57:39.430495 4169 rpcz_store.cc:276] 0811 18:57:35.344804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344956 (+ 152us) service_pool.cc:227] Handling call
0811 18:57:39.430202 (+4085246us) ts_tablet_manager.cc:1145] Deleting tablet fcb1c9787d3f4fd181f86deca28ad160
0811 18:57:39.430214 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.430331 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4082058}
W20260811 18:57:39.430943 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fcb1c9787d3f4fd181f86deca28ad160 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb1c9787d3f4fd181f86deca28ad160 already in progress: creating tablet
I20260811 18:57:39.431015 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 998) took 4086 ms. Trace:
I20260811 18:57:39.431095 4169 rpcz_store.cc:276] 0811 18:57:35.344925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344983 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:39.430894 (+4085911us) ts_tablet_manager.cc:1145] Deleting tablet 302cf69ea7c14fdaac72f6a9a9aff113
0811 18:57:39.430906 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.430998 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4082703}
W20260811 18:57:39.432065 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 302cf69ea7c14fdaac72f6a9a9aff113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302cf69ea7c14fdaac72f6a9a9aff113 already in progress: creating tablet
I20260811 18:57:39.433027 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1001) took 4087 ms. Trace:
I20260811 18:57:39.433235 4169 rpcz_store.cc:276] 0811 18:57:35.345639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.345694 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.432086 (+4086392us) ts_tablet_manager.cc:1145] Deleting tablet 18c599d945044e0088bc66c8472d2495
0811 18:57:39.432925 (+ 839us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.433011 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4083852}
W20260811 18:57:39.433657 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 18c599d945044e0088bc66c8472d2495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18c599d945044e0088bc66c8472d2495 already in progress: creating tablet
I20260811 18:57:39.433794 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 994) took 4089 ms. Trace:
I20260811 18:57:39.433892 4169 rpcz_store.cc:276] 0811 18:57:35.344089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346039 (+ 1950us) service_pool.cc:227] Handling call
0811 18:57:39.433675 (+4087636us) ts_tablet_manager.cc:1145] Deleting tablet f1342b8cd65b4c31a47b27a79c678d2b
0811 18:57:39.433688 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.433777 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085156}
W20260811 18:57:39.434206 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1342b8cd65b4c31a47b27a79c678d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1342b8cd65b4c31a47b27a79c678d2b already in progress: creating tablet
I20260811 18:57:39.434128 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.434429 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 995) took 4090 ms. Trace:
I20260811 18:57:39.434425 3746 raft_consensus.cc:2454] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:39.434523 4169 rpcz_store.cc:276] 0811 18:57:35.344233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346188 (+ 1955us) service_pool.cc:227] Handling call
0811 18:57:39.434258 (+4088070us) ts_tablet_manager.cc:1145] Deleting tablet e5f47ccc193d48efa15d60a06b148e00
0811 18:57:39.434269 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.434412 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085695}
I20260811 18:57:39.434931 3824 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.435352 5557 raft_consensus.cc:2781] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.435442 5557 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.435515 5557 raft_consensus.cc:3037] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.436877 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1155) took 1894 ms. Trace:
I20260811 18:57:39.437186 4167 rpcz_store.cc:276] 0811 18:57:37.542314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.542401 (+ 87us) service_pool.cc:227] Handling call
0811 18:57:39.413197 (+1870796us) ts_tablet_manager.cc:1145] Deleting tablet d32212c817c94fad8379462b027e7d3d
0811 18:57:39.434011 (+ 20814us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.436850 (+ 2839us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1870575}
I20260811 18:57:39.437687 4167 tablet_replica.cc:330] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.437916 4162 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.437985 4167 raft_consensus.cc:2229] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.438472 4167 raft_consensus.cc:2258] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.438382 5557 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439504 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.439819 4162 tablet_bootstrap.cc:654] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.439950 5557 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.440506 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e5f47ccc193d48efa15d60a06b148e00 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f47ccc193d48efa15d60a06b148e00 already in progress: creating tablet
I20260811 18:57:39.441092 4163 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:39.441154 3824 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.441339 4163 ts_tablet_manager.cc:1397] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260811 18:57:39.441395 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1005) took 3999 ms. Trace:
I20260811 18:57:39.441490 4169 rpcz_store.cc:276] 0811 18:57:35.442210 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.442372 (+ 162us) service_pool.cc:227] Handling call
0811 18:57:39.441021 (+3998649us) ts_tablet_manager.cc:1145] Deleting tablet 8eaafc7677344cac8d9616996c5f416c
0811 18:57:39.441036 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.441379 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3998204}
W20260811 18:57:39.441886 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8eaafc7677344cac8d9616996c5f416c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8eaafc7677344cac8d9616996c5f416c already in progress: creating tablet
I20260811 18:57:39.441993 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1008) took 3970 ms. Trace:
I20260811 18:57:39.442072 4169 rpcz_store.cc:276] 0811 18:57:35.471565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.471645 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:39.441887 (+3970242us) ts_tablet_manager.cc:1145] Deleting tablet e2fb15b72c1d49d2ac4961bc04655a29
0811 18:57:39.441902 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.441978 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969887}
W20260811 18:57:39.442476 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e2fb15b72c1d49d2ac4961bc04655a29 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2fb15b72c1d49d2ac4961bc04655a29 already in progress: creating tablet
I20260811 18:57:39.442615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1009) took 3970 ms. Trace:
I20260811 18:57:39.442703 4169 rpcz_store.cc:276] 0811 18:57:35.472263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.472331 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:39.442509 (+3970178us) ts_tablet_manager.cc:1145] Deleting tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
0811 18:57:39.442520 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.442600 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969897}
I20260811 18:57:39.443063 4169 tablet_replica.cc:330] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.443289 4169 raft_consensus.cc:2229] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:39.443328 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae already in progress: creating tablet
I20260811 18:57:39.443500 4169 raft_consensus.cc:2258] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.443924 4163 raft_consensus.cc:348] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439785 3746 raft_consensus.cc:3037] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.445123 4163 raft_consensus.cc:374] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.445192 4163 raft_consensus.cc:729] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.445382 4163 consensus_queue.cc:260] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.445986 4163 ts_tablet_manager.cc:1428] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 18:57:39.446216 4163 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.447768 4163 tablet_bootstrap.cc:654] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.448395 3746 raft_consensus.cc:2454] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:39.449087 3824 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.444056 4169 ts_tablet_manager.cc:1905] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.454528 5557 raft_consensus.cc:2781] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.454691 5557 raft_consensus.cc:686] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.454972 5557 consensus_queue.cc:237] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.468675 4163 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.469329 4163 ts_tablet_manager.cc:1397] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260811 18:57:39.471329 4163 raft_consensus.cc:348] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439337 4167 ts_tablet_manager.cc:1905] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.475864 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 133729us
I20260811 18:57:39.479883 4162 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.480068 4162 ts_tablet_manager.cc:1397] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.003s
I20260811 18:57:39.480839 4162 raft_consensus.cc:348] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.481088 4162 raft_consensus.cc:374] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.481169 4162 raft_consensus.cc:729] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.481393 4162 consensus_queue.cc:260] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.482060 4162 ts_tablet_manager.cc:1428] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:39.485606 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fa95131ce50943e3b0c6b5bdd5a30939
I20260811 18:57:39.488471 4162 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.490209 4162 tablet_bootstrap.cc:654] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.490598 5641 raft_consensus.cc:482] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.490731 5641 raft_consensus.cc:504] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.491329 5641 leader_election.cc:288] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.471886 4169 ts_tablet_manager.cc:1918] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:39.492781 4169 log.cc:1196] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:39.490664 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.492394 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfea0cd619024204b9bff1e2dd62b7b0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.493283 4169 ts_tablet_manager.cc:1939] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:39.493520 3676 leader_election.cc:341] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.495509 3678 leader_election.cc:334] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.495635 3678 leader_election.cc:302] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.496169 5641 raft_consensus.cc:2726] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.492782 4163 raft_consensus.cc:374] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.498208 4163 raft_consensus.cc:729] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.498627 4163 consensus_queue.cc:260] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.499518 4163 ts_tablet_manager.cc:1428] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.029s user 0.002s sys 0.000s
I20260811 18:57:39.502542 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f83d8784174f40e29b96eba2de396065"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.503190 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f83d8784174f40e29b96eba2de396065 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.503970 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 275ec54207504e9f8267925e2224b2ab consensus state has no leader
I20260811 18:57:39.504212 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet c944d5cfd9b3405b8151abfc9f2fd998 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.504663 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 7645a242bc4e4845ad49bfbe20b252f2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.505167 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_23655b2975534b81b076fd031ea5022a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.505704 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23655b2975534b81b076fd031ea5022a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.510550 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f65859fb42941e3a8375f65182f9ec4" } new_table_name: "default.Table_710bf71d5383421ca80e2caac75ec12f" modify_external_catalogs: true
I20260811 18:57:39.512020 5557 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.512189 5557 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.512913 5557 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.513842 5557 raft_consensus.cc:482] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.513969 5557 raft_consensus.cc:504] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.514511 5557 leader_election.cc:288] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.517330 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.517642 3746 raft_consensus.cc:2454] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:39.519101 3824 leader_election.cc:341] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: a847581412494e34b2c76e605ff11772
I20260811 18:57:39.519366 3824 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:39.520619 5557 raft_consensus.cc:2781] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.520756 5557 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.520850 5557 raft_consensus.cc:3037] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:39.523432 3824 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:39.523764 3824 leader_election.cc:334] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.523891 3824 leader_election.cc:302] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.524264 5630 raft_consensus.cc:2726] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.526326 5557 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.528319 5557 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.530441 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.530668 3746 raft_consensus.cc:3037] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.534142 3746 raft_consensus.cc:2454] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:39.534994 3824 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.535584 3824 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.537052 5557 raft_consensus.cc:2781] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.537395 5557 raft_consensus.cc:686] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.537948 5557 consensus_queue.cc:237] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:39.541558 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06519254820241f1bd4bbceb3faedca3 consensus state has no leader
I20260811 18:57:39.542052 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e" } modify_external_catalogs: true
I20260811 18:57:39.542354 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e" } modify_external_catalogs: true
W20260811 18:57:39.545305 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9fbac57c9c44a028f5b258593aff86d has no consensus state
I20260811 18:57:39.545675 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_227406eaa3694731a78479da6280faac" } modify_external_catalogs: true
I20260811 18:57:39.545852 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_227406eaa3694731a78479da6280faac" } modify_external_catalogs: true
W20260811 18:57:39.554685 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ccceeb89da34853a768df69e1298457 consensus state has no leader
I20260811 18:57:39.555058 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e142d545d8b249f3835bce3bbef9f94e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.555719 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e142d545d8b249f3835bce3bbef9f94e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.555065 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f614f2bd34c949cd94f9493a086dd8c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.557461 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f614f2bd34c949cd94f9493a086dd8c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.551277 5689 tablet.cc:1722] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:39.539875 5687 tablet.cc:1722] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:39.567662 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1010) took 4094 ms. Trace:
I20260811 18:57:39.567792 4169 rpcz_store.cc:276] 0811 18:57:35.472858 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.472911 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:39.443024 (+3970113us) ts_tablet_manager.cc:1145] Deleting tablet b1ab2eee21264b1aa5ba00528cae3843
0811 18:57:39.443034 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.457463 (+ 14429us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.471775 (+ 14312us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.567634 (+ 95859us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969890}
I20260811 18:57:39.568162 4169 tablet_replica.cc:330] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.568395 4169 raft_consensus.cc:2229] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.568625 4169 raft_consensus.cc:2258] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.569262 4169 ts_tablet_manager.cc:1905] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.569904 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet eb43127dda814a718b338906fc2a3c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.570166 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.570381 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5b7d1e32514ad48177371582e85241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.571061 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 with delete_type TABLET_DATA_DELETED (replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.577091 4162 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.577423 4162 ts_tablet_manager.cc:1397] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.022s
I20260811 18:57:39.577870 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d8f35609e86b47aa9a8ab711834da08f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.578305 4162 raft_consensus.cc:348] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.578620 4162 raft_consensus.cc:374] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.578713 4162 raft_consensus.cc:729] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.579071 4162 consensus_queue.cc:260] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.579849 4162 ts_tablet_manager.cc:1428] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:39.580175 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8f35609e86b47aa9a8ab711834da08f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.580246 4162 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.580967 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.581470 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:39.583388 4169 ts_tablet_manager.cc:1918] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.583539 4169 log.cc:1196] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:39.583766 4162 tablet_bootstrap.cc:654] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.583928 4169 ts_tablet_manager.cc:1939] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:39.584149 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:39.584466 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet e544b4b1592c4d51a2ed0d6461edc75e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.584803 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1027) took 3946 ms. Trace:
I20260811 18:57:39.584908 4169 rpcz_store.cc:276] 0811 18:57:35.637902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.638816 (+ 914us) service_pool.cc:227] Handling call
0811 18:57:39.568110 (+3929294us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:39.568124 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.576646 (+ 8522us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.583273 (+ 6627us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.584776 (+ 1503us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3929096}
I20260811 18:57:39.585822 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (table default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]) successfully deleted
I20260811 18:57:39.587891 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e894321d446d4d6da9e866788ff2b1e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.588516 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e894321d446d4d6da9e866788ff2b1e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.590564 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7a85117c40464174beed058911301729"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.591144 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a85117c40464174beed058911301729 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.595883 4163 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.599494 5641 raft_consensus.cc:482] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.599673 5641 raft_consensus.cc:504] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.600375 5641 leader_election.cc:288] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:39.605795 3678 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.598460 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0fde489a381549f394a78e98c8cade0f (DEFAULT_TABLE table=default.Table_8e26f75764214905965fa0f514cb0028 [id=3aa9faf1a865494e9e6b33917e0cd501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.598490 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 26f55fe6a9e840a4a5e7ba6bf727c779 (DEFAULT_TABLE table=default.Table_724b9320db2141f39b2511549a1aa8d4 [id=de7aea74dccc46f68bc416ad0f023c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.599496 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 625daea5324b4494996a51f0f9e9bcb7 (DEFAULT_TABLE table=default.Table_a6af8048b8f14af899345d8213953792 [id=677b3b20925544f28c2fb9ae18878c69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.600203 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1028) took 3962 ms. Trace:
I20260811 18:57:39.601573 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20872d80c9545e5822bfc82d496ce97" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.600586 3862 tablet_service.cc:1467] Processing CreateTablet for tablet eea325ca357146fcaa408d87803da349 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.607420 4169 rpcz_store.cc:276] 0811 18:57:35.638086 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.638140 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:39.600053 (+3961913us) ts_tablet_manager.cc:1145] Deleting tablet 46a15e7a0c0f4283ac5e575d3c15c9da
0811 18:57:39.600067 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.600183 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3930804}
W20260811 18:57:39.608361 3676 leader_election.cc:341] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.597818 4163 tablet_bootstrap.cc:654] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.608429 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 46a15e7a0c0f4283ac5e575d3c15c9da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46a15e7a0c0f4283ac5e575d3c15c9da already in progress: creating tablet
I20260811 18:57:39.608498 3676 leader_election.cc:302] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.609086 5630 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.609241 5630 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.610091 5630 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.613847 4162 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.613997 4162 ts_tablet_manager.cc:1397] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.001s
I20260811 18:57:39.614459 5407 raft_consensus.cc:2726] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.614734 4162 raft_consensus.cc:348] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.614943 4162 raft_consensus.cc:374] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.614985 4162 raft_consensus.cc:729] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.615283 4162 consensus_queue.cc:260] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.615935 4162 ts_tablet_manager.cc:1428] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.616335 4162 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.623620 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.623988 3746 raft_consensus.cc:2454] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:39.626658 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770" } modify_external_catalogs: true
I20260811 18:57:39.626909 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770" } modify_external_catalogs: true
I20260811 18:57:39.628968 4162 tablet_bootstrap.cc:654] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.629668 3824 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:39.629967 3824 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.629967 5557 raft_consensus.cc:2781] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.630149 5557 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.630247 5557 raft_consensus.cc:3037] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.633314 5557 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.634049 5557 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:39.630259 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.638895 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.639099 4169 tablet_replica.cc:330] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.639204 3746 raft_consensus.cc:3037] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.639333 4169 raft_consensus.cc:2229] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.639554 4169 raft_consensus.cc:2258] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.640244 4169 ts_tablet_manager.cc:1905] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.643684 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 8b7fc4a898cd4e2c9e27d6f5f964412a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.644873 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:39.645471 3548 catalog_manager.cc:6565] ReplaceTablet: tablet e544b4b1592c4d51a2ed0d6461edc75e deleted and replaced by tablet a07f682b68114c2b9cb0df27a94406ec
I20260811 18:57:39.646101 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:39.646407 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:39.646620 3549 catalog_manager.cc:5495] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.650768 4167 ts_tablet_manager.cc:1918] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.650926 4167 log.cc:1196] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e18242f762d8407db0e2f8716890feb4
I20260811 18:57:39.651362 4167 ts_tablet_manager.cc:1939] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.652309 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1156) took 2051 ms. Trace:
I20260811 18:57:39.652427 4167 rpcz_store.cc:276] 0811 18:57:37.601131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.602071 (+ 940us) service_pool.cc:227] Handling call
0811 18:57:39.437609 (+1835538us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:39.437622 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.579532 (+141910us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.650641 (+ 71109us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.652278 (+ 1637us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1835002}
I20260811 18:57:39.652935 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1168) took 1821 ms. Trace:
I20260811 18:57:39.653024 4167 rpcz_store.cc:276] 0811 18:57:37.831174 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.831269 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:39.652778 (+1821509us) ts_tablet_manager.cc:1145] Deleting tablet 3195082643264e0d90f70e1007ab9b24
0811 18:57:39.652792 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.652916 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1821255}
I20260811 18:57:39.654042 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f9cef27540ff46aebc9b6f4760ae0db4
I20260811 18:57:39.653950 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b90b4ead71940e58e93af4a9804a7d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.654624 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b90b4ead71940e58e93af4a9804a7d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.655047 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bc38e6760754449fbd6209d765cce70c
I20260811 18:57:39.655668 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ab03996c5c624e5eb17267b124e27f45
I20260811 18:57:39.655922 3555 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 602fd9a92962483ebc84ca2bcdb2bebc
I20260811 18:57:39.658238 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fde489a381549f394a78e98c8cade0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.658974 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f55fe6a9e840a4a5e7ba6bf727c779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659518 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 625daea5324b4494996a51f0f9e9bcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659967 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea325ca357146fcaa408d87803da349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659955 3746 raft_consensus.cc:2454] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:39.668270 4162 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.668627 4162 ts_tablet_manager.cc:1397] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.005s
I20260811 18:57:39.669509 4162 raft_consensus.cc:348] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.669878 4162 raft_consensus.cc:374] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.669992 4162 raft_consensus.cc:729] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.670511 4162 consensus_queue.cc:260] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.671311 4162 ts_tablet_manager.cc:1428] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:39.671655 4162 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.673393 4162 tablet_bootstrap.cc:654] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.675597 5630 raft_consensus.cc:482] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.675773 5630 raft_consensus.cc:504] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.676381 5630 leader_election.cc:288] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.676987 3824 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:39.677335 5630 raft_consensus.cc:2781] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.677465 5630 raft_consensus.cc:686] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.677752 5630 consensus_queue.cc:237] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.674301 5641 raft_consensus.cc:482] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.679077 5641 raft_consensus.cc:504] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.679734 5641 leader_election.cc:288] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.680632 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e24f8d4522466d81605ea37fa7064a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:39.681286 3676 leader_election.cc:341] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.683305 3678 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.683418 3678 leader_election.cc:302] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.683853 5641 raft_consensus.cc:2726] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.666280 4169 ts_tablet_manager.cc:1918] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:39.687491 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 929586965ce6468c82950102cfed41da (DEFAULT_TABLE table=default.Table_bce1d6623de1467198e1dfaf763f08cf [id=a85df4c40f944b598dcd718a4e9a3a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.687911 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 6688ed13d1334ed98503b21a937facd2 (DEFAULT_TABLE table=default.Table_644eac4afde147dea1b6dc8cd69dc719 [id=583ba00a08184bc38ff31f87596e3788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.688292 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929586965ce6468c82950102cfed41da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.688611 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688ed13d1334ed98503b21a937facd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.687691 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 441feddae43741678ef0f64888b5c6f3 (DEFAULT_TABLE table=default.Table_b081a3f48c9048ecb88f1bd4571f934b [id=54b7f379f89e4c009585c94a5f7e1a55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.689193 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 441feddae43741678ef0f64888b5c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.689131 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" } modify_external_catalogs: true
I20260811 18:57:39.689355 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" } modify_external_catalogs: true
I20260811 18:57:39.692121 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet c590c98e39f84ea09b26bbe8640ca93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.687454 4169 log.cc:1196] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ef47211aa8864d13a55551da47804c19
I20260811 18:57:39.686990 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 (DEFAULT_TABLE table=default.Table_9042bc231f0841a4a370be5e6251dd6e [id=044362a3a169467396a6be8959d1d07b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.687153 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet ab03996c5c624e5eb17267b124e27f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.694413 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.687045 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet bc38e6760754449fbd6209d765cce70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.687212 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e with delete_type TABLET_DATA_DELETED (replaced by tablet a07f682b68114c2b9cb0df27a94406ec) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.687336 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.697485 4169 ts_tablet_manager.cc:1939] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.697995 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1030) took 3967 ms. Trace:
I20260811 18:57:39.698141 4169 rpcz_store.cc:276] 0811 18:57:35.730672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.735506 (+ 4834us) service_pool.cc:227] Handling call
0811 18:57:39.608414 (+3872908us) ts_tablet_manager.cc:1145] Deleting tablet ef47211aa8864d13a55551da47804c19
0811 18:57:39.639030 (+ 30616us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.662936 (+ 23906us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.666176 (+ 3240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.697975 (+ 31799us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3872435}
I20260811 18:57:39.694052 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cef27540ff46aebc9b6f4760ae0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.702169 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:39.702445 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
I20260811 18:57:39.702607 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:39.702749 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e1af33fd98c6486f975368ec90493db7 (table default.Table_22bd9bbe66b64e209048c8164f116ba7 [id=7736472828a3499c8ad8026e164b9759]). Expected version 3 got 0
I20260811 18:57:39.702905 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_075181f96b7b42059289018e831e2671 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 2 got 0
I20260811 18:57:39.703045 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (table default.Table_9194817120074797a6f60bf8e24c0636 [id=1323bd99594a4382ba121cc11e79de31]). Expected version 1 got 0
I20260811 18:57:39.703183 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:39.707228 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1031) took 3969 ms. Trace:
I20260811 18:57:39.707374 4169 rpcz_store.cc:276] 0811 18:57:35.738991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.739315 (+ 324us) service_pool.cc:227] Handling call
0811 18:57:39.707060 (+3967745us) ts_tablet_manager.cc:1145] Deleting tablet 22e1a33348e34e35bab786f8966c408b
0811 18:57:39.707076 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.707195 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3967424}
I20260811 18:57:39.708340 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1032) took 3908 ms. Trace:
I20260811 18:57:39.708432 4169 rpcz_store.cc:276] 0811 18:57:35.799599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.799673 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:39.708206 (+3908533us) ts_tablet_manager.cc:1145] Deleting tablet 6f804703e20c4214b8b2a31efeaef59e
0811 18:57:39.708220 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.708320 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3907738}
I20260811 18:57:39.709143 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1033) took 3909 ms. Trace:
I20260811 18:57:39.709236 4169 rpcz_store.cc:276] 0811 18:57:35.799738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.799815 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:39.709026 (+3909211us) ts_tablet_manager.cc:1145] Deleting tablet 9b931eca7ba04c11af6571cb418f882e
0811 18:57:39.709038 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.709125 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3909008}
I20260811 18:57:39.709507 4169 tablet_replica.cc:330] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.709971 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f389d2b7ff354ac4bab5d68b9ff201c6
I20260811 18:57:39.713902 4163 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.714143 4163 ts_tablet_manager.cc:1397] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.118s user 0.013s sys 0.004s
I20260811 18:57:39.715138 4163 raft_consensus.cc:348] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.715534 4163 raft_consensus.cc:374] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.715647 4163 raft_consensus.cc:729] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.715973 4163 consensus_queue.cc:260] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.716773 4163 ts_tablet_manager.cc:1428] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.720862 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c499c7d38b65414692ba6f4d7307a7bf" } new_table_name: "default.Table_4d54139265ad4964846ab0b2a1331b90" modify_external_catalogs: true
I20260811 18:57:39.721808 4163 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.723747 4163 tablet_bootstrap.cc:654] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.727411 4162 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.727674 4162 ts_tablet_manager.cc:1397] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.005s
I20260811 18:57:39.728547 4162 raft_consensus.cc:348] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.728852 4162 raft_consensus.cc:374] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.728955 4162 raft_consensus.cc:729] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.729271 4162 consensus_queue.cc:260] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.730012 4162 ts_tablet_manager.cc:1428] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:39.730377 4162 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.730415 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a49eeecb6ae34fe28d40847ab1738944"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.731066 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a49eeecb6ae34fe28d40847ab1738944 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.732353 4162 tablet_bootstrap.cc:654] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.734509 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1170) took 1758 ms. Trace:
I20260811 18:57:39.736496 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 330801031cc74eb2a5bc2caf53a18ead (table default.Table_04ac8209b46549e3bb3f12971ab01b73 [id=54b88a6b23fe45e8b0f235baf8f7234f]). Expected version 1 got 0
I20260811 18:57:39.737267 4167 rpcz_store.cc:276] 0811 18:57:37.976345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.976640 (+ 295us) service_pool.cc:227] Handling call
0811 18:57:39.734295 (+1757655us) ts_tablet_manager.cc:1145] Deleting tablet 429eb5824f3f4b2b8da6e17ba7c69b92
0811 18:57:39.734312 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.734478 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1757421}
I20260811 18:57:39.738471 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1171) took 1762 ms. Trace:
I20260811 18:57:39.744102 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03b64ac0b80f46288fcf2b34397efcaa" } new_table_name: "default.Table_4a427b04935c4d62b0595df0085e3a98" modify_external_catalogs: true
I20260811 18:57:39.744436 4167 rpcz_store.cc:276] 0811 18:57:37.976486 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.976907 (+ 421us) service_pool.cc:227] Handling call
0811 18:57:39.738279 (+1761372us) ts_tablet_manager.cc:1145] Deleting tablet f89eebbc40234794a081734f834294ab
0811 18:57:39.738296 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.738445 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1761212}
I20260811 18:57:39.744709 4163 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.744859 4163 ts_tablet_manager.cc:1397] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260811 18:57:39.745118 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1172) took 1738 ms. Trace:
I20260811 18:57:39.745235 4167 rpcz_store.cc:276] 0811 18:57:38.007054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.007116 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:39.744937 (+1737821us) ts_tablet_manager.cc:1145] Deleting tablet a5bbd3274980470b9fe71457b49bdd62
0811 18:57:39.744951 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.745075 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1737579}
I20260811 18:57:39.745720 4163 raft_consensus.cc:348] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.746089 4163 raft_consensus.cc:374] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.746205 4163 raft_consensus.cc:729] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.746502 4163 consensus_queue.cc:260] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.747195 4163 ts_tablet_manager.cc:1428] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.747530 4163 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.749835 4163 tablet_bootstrap.cc:654] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.753669 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_faccbd3f194043708dbbd12a50f037cf" } new_table_name: "default.Table_4400f63ffc1944a1bd8a37ed1a0ed505" modify_external_catalogs: true
I20260811 18:57:39.745994 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1169) took 1769 ms. Trace:
I20260811 18:57:39.754530 4167 rpcz_store.cc:276] 0811 18:57:37.976159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.010438 (+ 34279us) service_pool.cc:227] Handling call
0811 18:57:39.745849 (+1735411us) ts_tablet_manager.cc:1145] Deleting tablet b7b38d3b9570467a9128bd7fe73acf2f
0811 18:57:39.745864 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.745953 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1734821}
I20260811 18:57:39.755209 4167 tablet_replica.cc:330] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.755512 4167 raft_consensus.cc:2229] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.755750 4167 raft_consensus.cc:2258] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.756422 4167 ts_tablet_manager.cc:1905] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.759436 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e44f4b4162648d6911b8a00b8fd7ba6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.760087 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e44f4b4162648d6911b8a00b8fd7ba6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.752969 5593 raft_consensus.cc:482] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.761679 5593 raft_consensus.cc:504] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.762133 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_30ac5178633e44aeafd707e48d7c4cba" } new_table_name: "default.Table_f94d755a69ab49dcbbc91974db72cb77" modify_external_catalogs: true
I20260811 18:57:39.766465 5593 leader_election.cc:288] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.766554 3852 tablet_service.cc:1467] Processing CreateTablet for tablet ab9fe26e8e1e4c3886af83ac3e6956f5 (DEFAULT_TABLE table=default.Table_7e6c466c96e8465c9677dcaddee8819a [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.767084 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9fe26e8e1e4c3886af83ac3e6956f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.780583 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1612fbcb897947ee843430ed61c2e50e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.781488 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1612fbcb897947ee843430ed61c2e50e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.782327 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:39.782708 3552 catalog_manager.cc:5495] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.783143 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 1 got 0
I20260811 18:57:39.786767 5641 raft_consensus.cc:482] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.786945 5641 raft_consensus.cc:504] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.787601 5641 leader_election.cc:288] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.787598 3552 catalog_manager.cc:5495] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.788034 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b0f40c58a1c4f028c865253aba93296" } new_table_name: "default.Table_573bd28ff5814150abf3d238663273d7" modify_external_catalogs: true
I20260811 18:57:39.788808 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec4bba841dd446aa3db751358ce7f32" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.788678 3552 catalog_manager.cc:5495] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.789075 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:39.789347 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:39.789495 3676 leader_election.cc:341] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.789537 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:39.789703 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
W20260811 18:57:39.790975 3678 leader_election.cc:334] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.791090 3678 leader_election.cc:302] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.791577 4163 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.791733 4163 ts_tablet_manager.cc:1397] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:39.791985 5641 raft_consensus.cc:2726] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.793007 4163 raft_consensus.cc:348] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.793342 4163 raft_consensus.cc:374] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.793468 4163 raft_consensus.cc:729] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.793833 4163 consensus_queue.cc:260] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.794184 4169 raft_consensus.cc:2229] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:39.794832 5641 raft_consensus.cc:482] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.794957 5641 raft_consensus.cc:504] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.795503 5641 leader_election.cc:288] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.796854 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 5d1b94cb25b84ae1a4c22801b1a98094 (DEFAULT_TABLE table=default.Table_04519ba2021244b6922bea0e5b79ecb6 [id=02744826873c4079a8d983c8e0d71c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.794656 4163 ts_tablet_manager.cc:1428] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:39.797297 4169 pending_rounds.cc:62] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:39.796103 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e14f8717bcd94d6d88295fd8be130ca5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.797358 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d1b94cb25b84ae1a4c22801b1a98094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.797832 4163 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:39.797828 3676 leader_election.cc:341] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.800180 3859 tablet_service.cc:1467] Processing CreateTablet for tablet b9fbac57c9c44a028f5b258593aff86d (DEFAULT_TABLE table=default.Table_51582df654e7406dbc574ba8de215baf [id=0eb328384d824024bcf7782bc4cf5b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.800694 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9fbac57c9c44a028f5b258593aff86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.800999 4163 tablet_bootstrap.cc:654] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.802484 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.802906 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b3ef682132a04b338b03a4387999d6b9 (DEFAULT_TABLE table=default.Table_4fd6bf06dd964e719ce1470aec9f60bc [id=54c1a4605328449dbb0a4928e192e271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.803292 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ef682132a04b338b03a4387999d6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.803777 4167 ts_tablet_manager.cc:1918] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.803901 4167 log.cc:1196] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/42405aec21e3405486b6c658438b5fb2
I20260811 18:57:39.804200 4167 ts_tablet_manager.cc:1939] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.804328 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_343b8634c5a946feacafd49b9aba3c62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.805027 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1174) took 1716 ms. Trace:
I20260811 18:57:39.805120 4167 rpcz_store.cc:276] 0811 18:57:38.088470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.088610 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:39.755112 (+1666502us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:39.755126 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.800687 (+ 45561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.803646 (+ 2959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.804998 (+ 1352us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1666255}
W20260811 18:57:39.808506 3678 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.808669 3678 leader_election.cc:302] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.808962 5641 raft_consensus.cc:2726] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.810125 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ef47211aa8864d13a55551da47804c19 (table default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c]) successfully deleted
W20260811 18:57:39.810689 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 22e1a33348e34e35bab786f8966c408b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22e1a33348e34e35bab786f8966c408b already in progress: creating tablet
W20260811 18:57:39.810920 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6f804703e20c4214b8b2a31efeaef59e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f804703e20c4214b8b2a31efeaef59e already in progress: creating tablet
W20260811 18:57:39.811115 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9b931eca7ba04c11af6571cb418f882e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b931eca7ba04c11af6571cb418f882e already in progress: creating tablet
W20260811 18:57:39.813042 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_343b8634c5a946feacafd49b9aba3c62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.815361 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06c8776bb47c4cb891ef13a2a1e36781 consensus state has no leader
I20260811 18:57:39.815922 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93d80fcaa1a646119a16dcb3e546b426"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.816550 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93d80fcaa1a646119a16dcb3e546b426 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.816715 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75daed89122c406caa4bd9bbaa67d9b8 consensus state has no leader
W20260811 18:57:39.817889 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b609177908d04a829b1fe744ef0f53e1 consensus state has no leader
W20260811 18:57:39.826543 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b540ae350254346a675b54118c5cfcb consensus state has no leader
W20260811 18:57:39.826543 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:39.832258 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0b6fac836f9146ab9a5700100e2bda0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.832957 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b6fac836f9146ab9a5700100e2bda0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.797354 4169 pending_rounds.cc:68] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400190985445376 op_type: NO_OP noop_request { }
I20260811 18:57:39.843808 4169 raft_consensus.cc:2866] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:39.844903 4169 raft_consensus.cc:2258] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.846997 4169 ts_tablet_manager.cc:1905] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.851079 5641 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.851255 5641 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.851864 5641 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.854700 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 407b07d906d4447dbae5fd45d44e3ef8 (DEFAULT_TABLE table=default.Table_24401a03e8d54050a74da4ed3bbb60fd [id=936d78e258b341f98d02313652a325f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.855178 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b07d906d4447dbae5fd45d44e3ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.858621 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 812cc50dcac740c5ab7633abc04c1d9e (DEFAULT_TABLE table=default.Table_3020dd34e96c426fbbca98e320405d0f [id=7f1262b7d1754c54aa332bc31a4db6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.859042 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812cc50dcac740c5ab7633abc04c1d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.863246 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 81f52a745f6243ae9b1ef2dd33b06d3a (DEFAULT_TABLE table=default.Table_fe6c2229e23048eb81c61314102eab38 [id=57de9419daff451a9ca6485e9230e621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.863695 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f52a745f6243ae9b1ef2dd33b06d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.868870 4167 tablet_replica.cc:330] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.869212 4167 raft_consensus.cc:2229] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.869386 4167 raft_consensus.cc:2258] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.869947 4167 ts_tablet_manager.cc:1905] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.858201 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8072b2d13cbc45249f43411e9022d006 (DEFAULT_TABLE table=default.Table_46d76d83b3174f5da337585b00af5f68 [id=40394961c1b348b4b67b3322da0eb2c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.871235 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8072b2d13cbc45249f43411e9022d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.875533 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51582df654e7406dbc574ba8de215baf" } modify_external_catalogs: true
I20260811 18:57:39.875599 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f7751679e1394c27bdfb569a73a14165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.875828 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51582df654e7406dbc574ba8de215baf" } modify_external_catalogs: true
W20260811 18:57:39.876264 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7751679e1394c27bdfb569a73a14165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.877401 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" } modify_external_catalogs: true
I20260811 18:57:39.877583 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" } modify_external_catalogs: true
I20260811 18:57:39.879017 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6533d74b53c54546bdd082863b38257c" } modify_external_catalogs: true
I20260811 18:57:39.879172 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6533d74b53c54546bdd082863b38257c" } modify_external_catalogs: true
I20260811 18:57:39.880829 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_194b336657844c34ac93e94c90db9444"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.881424 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_194b336657844c34ac93e94c90db9444 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.885403 3678 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.899451 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:39.901794 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7a67e56b3f641139862409e95439c6a" } modify_external_catalogs: true
I20260811 18:57:39.907416 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7a67e56b3f641139862409e95439c6a" } modify_external_catalogs: true
I20260811 18:57:39.898618 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b9fbac57c9c44a028f5b258593aff86d
I20260811 18:57:39.913606 4167 ts_tablet_manager.cc:1918] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.915272 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb4433f8104245af96d1746998f92ba1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.916165 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb4433f8104245af96d1746998f92ba1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.916716 5698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:39.916978 5698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:39.917851 5698 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:39.919184 4167 log.cc:1196] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/eb8f5b87062349c4b19ec99fa0175f24
I20260811 18:57:39.919462 5266 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.919577 4167 ts_tablet_manager.cc:1939] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.919571 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1531 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:39.920536 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1177) took 1714 ms. Trace:
I20260811 18:57:39.920644 4167 rpcz_store.cc:276] 0811 18:57:38.205694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.206098 (+ 404us) service_pool.cc:227] Handling call
0811 18:57:39.868802 (+1662704us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:39.868818 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.873068 (+ 4250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.913481 (+ 40413us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.920511 (+ 7030us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1652400}
I20260811 18:57:39.921202 4167 tablet_replica.cc:330] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.921411 4167 raft_consensus.cc:2229] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.923799 4167 raft_consensus.cc:2258] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.924372 4167 ts_tablet_manager.cc:1905] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.927892 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a" } modify_external_catalogs: true
I20260811 18:57:39.928139 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a" } modify_external_catalogs: true
I20260811 18:57:39.928849 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b5f718e0dd6347099abbd1de1d43ca73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.929493 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5f718e0dd6347099abbd1de1d43ca73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.852658 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:39.859241 3824 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.922151 5593 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:39.939088 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 79868.4us
I20260811 18:57:39.939165 5593 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.939956 5593 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.940325 5593 raft_consensus.cc:482] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.940457 5593 raft_consensus.cc:504] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.941026 5593 leader_election.cc:288] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.944519 3676 leader_election.cc:341] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.944700 3676 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.945070 5641 raft_consensus.cc:2726] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.946148 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1531 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:39.947057 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:39.953877 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 258d08de21434edc86f0cab49f839294 (DEFAULT_TABLE table=default.Table_2dffa8e09c6948c092cfdd4d888e5f81 [id=58c3e7941755444c978bd70d45db09f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.954427 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d08de21434edc86f0cab49f839294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.967521 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d65cb8d901c04d8ba2b7a40efc7fa3af
I20260811 18:57:39.970872 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 09366eccf38a4876840cc5ddddbbb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.972577 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 94857ef729eb435c97d089b8ab108cea (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.973147 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94857ef729eb435c97d089b8ab108cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.977262 4162 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.977437 4162 ts_tablet_manager.cc:1397] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.247s user 0.019s sys 0.003s
I20260811 18:57:39.978169 4162 raft_consensus.cc:348] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.978475 4162 raft_consensus.cc:374] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.978549 4162 raft_consensus.cc:729] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.978873 4162 consensus_queue.cc:260] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.987686 4162 ts_tablet_manager.cc:1428] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260811 18:57:39.988047 4162 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:39.990145 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 25266bc7788f49f8af8eb9a948e61c8d (table default.Table_f94d755a69ab49dcbbc91974db72cb77 [id=31b3d877b5da424eb8f4b3fd7086cda7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:39.991844 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 9c048122b5434ec594fbf5b2665edfab (DEFAULT_TABLE table=default.Table_0765ca548a9649e88a2ad253c392107b [id=46788ef8b34947bbbffd5f651a41a102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.994441 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c048122b5434ec594fbf5b2665edfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.996125 5641 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.996279 5641 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.996932 5641 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:39.999397 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.997678 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.004184 3887 raft_consensus.cc:2454] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:40.003396 4162 tablet_bootstrap.cc:654] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.004716 3676 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.007113 5641 raft_consensus.cc:2781] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.007191 5641 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.007253 5641 raft_consensus.cc:3037] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.010016 5641 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.012092 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.012348 3887 raft_consensus.cc:3037] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.014146 5641 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.015152 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:40.015405 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.017304 3887 raft_consensus.cc:2454] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.017828 3676 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.018114 5641 raft_consensus.cc:2781] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.018411 5641 raft_consensus.cc:686] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.018721 5641 consensus_queue.cc:237] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.018882 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db613e17da86441bb1ab37b46e730db9" } new_table_name: "default.Table_c8cfed8b43a34205a0b491d9374ba760" modify_external_catalogs: true
I20260811 18:57:40.020020 4163 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.020159 4163 ts_tablet_manager.cc:1397] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.222s user 0.025s sys 0.000s
I20260811 18:57:40.020437 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_deb475a5050c4f3dbaece0eb1e4ac618" } new_table_name: "default.Table_8be7a98fc3664e94b5af6924771fdd01" modify_external_catalogs: true
I20260811 18:57:40.020865 4163 raft_consensus.cc:348] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.021070 4163 raft_consensus.cc:374] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.021123 4163 raft_consensus.cc:729] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.021368 4163 consensus_queue.cc:260] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.021893 4163 ts_tablet_manager.cc:1428] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:40.040330 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3309d84857df4970a9891fd64c181bf5" } new_table_name: "default.Table_298b25b6ed544a9fb0118e1af056b525" modify_external_catalogs: true
I20260811 18:57:40.041119 4167 ts_tablet_manager.cc:1918] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.041289 4167 log.cc:1196] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:40.041178 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f404fe05a8674605bb8038d676443046 (DEFAULT_TABLE table=default.Table_7cd9fda507254d029c62090b18422521 [id=50dde5de39e14c068d494813f4e68a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.041622 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f404fe05a8674605bb8038d676443046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.040344 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_93b2871d4c454120a6b609ae91e096e6" } new_table_name: "default.Table_cf30a335d3ce4b438ae4b12e64576085" modify_external_catalogs: true
I20260811 18:57:40.045076 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69f270a4181547b2a2cc604a814c82c7" } new_table_name: "default.Table_a2a931a8e22943d7906481658e211322" modify_external_catalogs: true
I20260811 18:57:40.049540 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c0ad178862a14f968c9a334a1570eb82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.050242 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0ad178862a14f968c9a334a1570eb82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.050625 5407 raft_consensus.cc:482] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.050768 5407 raft_consensus.cc:504] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.051409 5407 leader_election.cc:288] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.054109 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3a9fdc3d594c1a84a5d8efa8490dfa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.053540 4162 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.054590 4162 ts_tablet_manager.cc:1397] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.000s
W20260811 18:57:40.054749 3676 leader_election.cc:341] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:40.055186 3678 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.055315 3678 leader_election.cc:302] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.055809 5641 raft_consensus.cc:2726] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.041631 4167 ts_tablet_manager.cc:1939] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.059087 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 408c18a220d94809a5c43e1bfca310c8 (DEFAULT_TABLE table=default.Table_325467f49f2648b5a83d5fc618fb302d [id=4dcf548dd1e84122a82fa8971e993312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.059489 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c18a220d94809a5c43e1bfca310c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.059932 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1178) took 1854 ms. Trace:
I20260811 18:57:40.060022 4167 rpcz_store.cc:276] 0811 18:57:38.205784 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.206247 (+ 463us) service_pool.cc:227] Handling call
0811 18:57:39.921151 (+1714904us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:39.921164 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.971565 (+ 50401us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.040992 (+ 69427us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.059909 (+ 18917us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1704664}
I20260811 18:57:40.060343 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 60f82aa67c614c6abcac348b67255904 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.060696 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f82aa67c614c6abcac348b67255904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.061349 4162 raft_consensus.cc:348] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.061558 4162 raft_consensus.cc:374] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.061611 4162 raft_consensus.cc:729] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.061825 4162 consensus_queue.cc:260] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.062698 3705 tablet_service.cc:1467] Processing CreateTablet for tablet f87a8741e8ec481ea21284abe2d6c479 (DEFAULT_TABLE table=default.Table_788572afacab482590222fc4d084ef0f [id=77992422c86e47c79d4e1aea0a66b089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.063081 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f87a8741e8ec481ea21284abe2d6c479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.066644 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d3e28a19c145403485376b057bad4c9e (DEFAULT_TABLE table=default.Table_51094e8e45fe427789a0f442148332f0 [id=7aa13fc26748491db551f61611f7c62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.066788 4167 tablet_replica.cc:330] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.066998 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e28a19c145403485376b057bad4c9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:40.068375 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7ce2b43b2134defb14d45112e4f5e5e consensus state has no leader
W20260811 18:57:40.070982 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de6245131a3a4513b88611fd4c2e031c consensus state has no leader
I20260811 18:57:40.071346 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_40846b4d992644f9ad4942d6e529abb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.071959 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40846b4d992644f9ad4942d6e529abb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.075611 4162 ts_tablet_manager.cc:1428] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260811 18:57:40.075917 4162 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.076457 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 78868ff1f8024163ae1d782151100cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.066998 4167 raft_consensus.cc:2229] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.077049 4167 raft_consensus.cc:2258] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.077598 4167 ts_tablet_manager.cc:1905] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.078061 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.078545 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 33a79243bd874861b16f43339adf777c (DEFAULT_TABLE table=default.Table_80140471fa194b22bf8347836f95fdca [id=6de24db40cfe4fb49177dedc3d64443b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.079234 3711 tablet_service.cc:1467] Processing CreateTablet for tablet e4441c724b394e8092fb19c0d2427233 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.081084 4163 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:40.082360 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 142404us
W20260811 18:57:40.083434 3824 leader_election.cc:341] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: cf6c8d542ae2489ba466ffadd518db12
W20260811 18:57:40.083798 3824 leader_election.cc:341] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0c6e0d1eeb63493bbea4cd78dd085a3c
W20260811 18:57:40.084594 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c80955bd879c4c1eb16b07ad38096259 consensus state has no leader
I20260811 18:57:40.085007 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8cc94cebd1834be58b21d1512bb24571"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.085628 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cc94cebd1834be58b21d1512bb24571 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.086040 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1922c7317c24aa984945aaaa6ce8003 consensus state has no leader
I20260811 18:57:40.086416 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_193dfc5ac3914a1f90e082521d0f84d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.086602 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4441c724b394e8092fb19c0d2427233. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:40.086917 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_193dfc5ac3914a1f90e082521d0f84d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.087446 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 112de1a11805406b9183100c6c66f720 consensus state has no leader
I20260811 18:57:40.087765 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a407d0c02054f5ca4370e8e1a1c4c71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.088552 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a407d0c02054f5ca4370e8e1a1c4c71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.090469 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a79243bd874861b16f43339adf777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.091377 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:40.094704 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1e51bb756dff43caace2c0626aed1443 (DEFAULT_TABLE table=default.Table_1afb343a7aa84568ad8ad65cab128123 [id=1fa01fbb7a4946da82085dab1e0a71ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.095726 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e51bb756dff43caace2c0626aed1443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.096283 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1fc828f73642c88171d6ac827de90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.096736 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b9fbac57c9c44a028f5b258593aff86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097119 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097491 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097890 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.098551 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.099089 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.099045 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_da206548a6a34eb78f9b4bddfc58d921"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.099651 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da206548a6a34eb78f9b4bddfc58d921 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.103118 4162 tablet_bootstrap.cc:654] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.104702 4169 ts_tablet_manager.cc:1918] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:40.104900 4169 log.cc:1196] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:40.105472 4169 ts_tablet_manager.cc:1939] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:40.105896 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1034) took 4305 ms. Trace:
I20260811 18:57:40.108448 5641 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.108603 5641 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.109584 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a306b155dd94cfaa6649e14665e9418"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.109897 5698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:57:40.110249 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a306b155dd94cfaa6649e14665e9418 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.110776 5641 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.112550 3678 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.121006 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" } modify_external_catalogs: true
I20260811 18:57:40.121230 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" } modify_external_catalogs: true
I20260811 18:57:40.123054 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93e4cfd44cab49469aa14294ac0ba348"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.123608 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93e4cfd44cab49469aa14294ac0ba348 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.125418 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" } modify_external_catalogs: true
I20260811 18:57:40.125892 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" } modify_external_catalogs: true
I20260811 18:57:40.106843 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.107146 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.111033 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.125449 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:40.130650 3676 leader_election.cc:341] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.130784 3676 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.131253 5641 raft_consensus.cc:2726] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.126508 3824 leader_election.cc:334] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.131595 3824 leader_election.cc:302] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.129730 4163 tablet_bootstrap.cc:654] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.132722 5701 raft_consensus.cc:2726] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.133397 4167 ts_tablet_manager.cc:1918] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:40.133912 3824 leader_election.cc:334] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.134052 3824 leader_election.cc:302] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.134282 5701 raft_consensus.cc:2726] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.137845 4169 rpcz_store.cc:276] 0811 18:57:35.800651 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.801165 (+ 514us) service_pool.cc:227] Handling call
0811 18:57:39.709459 (+3908294us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:39.709469 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.984358 (+274889us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.104584 (+120226us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.105875 (+ 1291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3908061}
I20260811 18:57:40.138234 4169 tablet_replica.cc:330] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.138479 4169 raft_consensus.cc:2229] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.138633 4169 raft_consensus.cc:2258] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.139181 4169 ts_tablet_manager.cc:1905] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.139436 4162 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:40.142455 3824 leader_election.cc:341] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:40.142633 4162 ts_tablet_manager.cc:1397] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.067s user 0.003s sys 0.000s
I20260811 18:57:40.143419 4162 raft_consensus.cc:348] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.143710 4162 raft_consensus.cc:374] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.143818 4162 raft_consensus.cc:729] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.143790 5701 consensus_queue.cc:1035] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.144316 4162 consensus_queue.cc:260] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.155476 4162 ts_tablet_manager.cc:1428] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:40.158365 5701 raft_consensus.cc:482] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.158557 5701 raft_consensus.cc:504] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.159142 5701 leader_election.cc:288] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:40.162611 3824 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.162938 5641 raft_consensus.cc:482] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.163151 5641 raft_consensus.cc:504] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.165786 3678 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.166359 5641 leader_election.cc:288] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.166682 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd393d9c4d24aab8a170e1329400656" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.162767 3824 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:40.168311 3676 leader_election.cc:341] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.168442 3676 leader_election.cc:302] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.168741 5701 raft_consensus.cc:2726] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.168990 5641 raft_consensus.cc:2726] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.169056 3824 consensus_peers.cc:487] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.170187 4162 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.171593 4162 tablet_bootstrap.cc:654] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.143770 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0b4fff40ba9d49828facafd431939fd4 (DEFAULT_TABLE table=default.Table_e8dcb132b5b5474b8ff081e54a55dd3f [id=58e6357b58b4481c9fad17e3aa498147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.172528 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4fff40ba9d49828facafd431939fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.173103 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160" } modify_external_catalogs: true
I20260811 18:57:40.173391 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160" } modify_external_catalogs: true
W20260811 18:57:40.173928 3824 leader_election.cc:334] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.174059 3824 leader_election.cc:302] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.174593 5701 raft_consensus.cc:2726] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.175141 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b57ed7814a1e440cba362613cbab32f8 (table default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]) successfully deleted
I20260811 18:57:40.176375 4167 log.cc:1196] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d37e81d376e14907b13746cca831a96b
I20260811 18:57:40.176546 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fdab1c3c431c4dceb08b00eabcfa1ceb
I20260811 18:57:40.176738 4167 ts_tablet_manager.cc:1939] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.177520 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1179) took 1960 ms. Trace:
I20260811 18:57:40.177613 4167 rpcz_store.cc:276] 0811 18:57:38.217330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.217540 (+ 210us) service_pool.cc:227] Handling call
0811 18:57:40.066741 (+1849201us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:40.066753 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.090384 (+ 23631us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.133291 (+ 42907us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.177497 (+ 44206us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1848922}
I20260811 18:57:40.178021 4167 tablet_replica.cc:330] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.179567 4162 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.179700 4162 ts_tablet_manager.cc:1397] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:40.180330 4162 raft_consensus.cc:348] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.180558 4162 raft_consensus.cc:374] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.180622 4162 raft_consensus.cc:729] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.180842 4162 consensus_queue.cc:260] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.181275 4162 ts_tablet_manager.cc:1428] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:57:40.181527 4162 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.183145 4162 tablet_bootstrap.cc:654] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.197353 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d57e6ac8edbc46a69270b9277778e7b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.198027 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d57e6ac8edbc46a69270b9277778e7b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.201332 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_020300b2b8b949278c35159bd43fe19d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.201882 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_020300b2b8b949278c35159bd43fe19d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.202328 5641 raft_consensus.cc:482] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.202261 5701 raft_consensus.cc:482] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.202507 5641 raft_consensus.cc:504] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.202555 5701 raft_consensus.cc:504] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.203193 5701 leader_election.cc:288] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.203408 5641 leader_election.cc:288] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.203951 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330801031cc74eb2a5bc2caf53a18ead" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.204537 3676 leader_election.cc:341] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:40.205178 3678 leader_election.cc:334] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.205323 3678 leader_election.cc:302] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.205722 5641 raft_consensus.cc:2726] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.206225 4163 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.206393 4163 ts_tablet_manager.cc:1397] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.125s user 0.006s sys 0.001s
I20260811 18:57:40.206614 4167 raft_consensus.cc:2229] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.206799 4167 raft_consensus.cc:2258] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.207193 4163 raft_consensus.cc:348] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.207474 4163 raft_consensus.cc:374] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.207546 4163 raft_consensus.cc:729] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.207779 4163 consensus_queue.cc:260] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.210656 3745 raft_consensus.cc:1261] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:57:40.211723 3824 consensus_peers.cc:487] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:40.215876 4167 ts_tablet_manager.cc:1905] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.234740 4163 ts_tablet_manager.cc:1428] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
W20260811 18:57:40.236487 3824 leader_election.cc:341] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:40.247491 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:40.248734 4163 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.249541 5701 consensus_queue.cc:1035] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:40.250447 3824 leader_election.cc:341] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:40.263342 4163 tablet_bootstrap.cc:654] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.266461 4167 ts_tablet_manager.cc:1918] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.266686 4167 log.cc:1196] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/bfea0cd619024204b9bff1e2dd62b7b0
I20260811 18:57:40.267030 4167 ts_tablet_manager.cc:1939] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.267882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1182) took 1999 ms. Trace:
I20260811 18:57:40.267992 4167 rpcz_store.cc:276] 0811 18:57:38.268190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.268658 (+ 468us) service_pool.cc:227] Handling call
0811 18:57:40.177974 (+1909316us) ts_tablet_manager.cc:1145] Deleting tablet bfea0cd619024204b9bff1e2dd62b7b0
0811 18:57:40.177984 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.247216 (+ 69232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.266314 (+ 19098us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.267859 (+ 1545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1909014}
I20260811 18:57:40.273845 4162 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.274003 4162 ts_tablet_manager.cc:1397] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.002s
I20260811 18:57:40.274814 4162 raft_consensus.cc:348] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.275038 4162 raft_consensus.cc:374] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.275096 4162 raft_consensus.cc:729] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.275307 4162 consensus_queue.cc:260] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.276403 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1183) took 2008 ms. Trace:
I20260811 18:57:40.276938 4167 rpcz_store.cc:276] 0811 18:57:38.268326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269008 (+ 682us) service_pool.cc:227] Handling call
0811 18:57:40.276240 (+2007232us) ts_tablet_manager.cc:1145] Deleting tablet 69059a0667234a408cc1c09585df63bc
0811 18:57:40.276254 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.276378 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2007040}
I20260811 18:57:40.277998 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1184) took 2009 ms. Trace:
I20260811 18:57:40.278492 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.278462 4167 rpcz_store.cc:276] 0811 18:57:38.268446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269225 (+ 779us) service_pool.cc:227] Handling call
0811 18:57:40.277860 (+2008635us) ts_tablet_manager.cc:1145] Deleting tablet 82b04e37fc264fc08e0ed9517356a17d
0811 18:57:40.277873 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.277980 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2008478}
I20260811 18:57:40.280203 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8be7a98fc3664e94b5af6924771fdd01" } new_table_name: "default.Table_0005d076764c413e92498cb07ad6920a" modify_external_catalogs: true
I20260811 18:57:40.279584 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1185) took 2011 ms. Trace:
I20260811 18:57:40.280473 4167 rpcz_store.cc:276] 0811 18:57:38.268592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269416 (+ 824us) service_pool.cc:227] Handling call
0811 18:57:40.279450 (+2010034us) ts_tablet_manager.cc:1145] Deleting tablet d11b889e9d044a049d0586382c263951
0811 18:57:40.279463 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.279564 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2009885}
I20260811 18:57:40.281639 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1181) took 2013 ms. Trace:
I20260811 18:57:40.283087 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_53bcd19697ad49b9aec577b1646c0d3e" } new_table_name: "default.Table_69847be1563a4aad8d093d935864f531" modify_external_catalogs: true
I20260811 18:57:40.285334 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.286736 4167 rpcz_store.cc:276] 0811 18:57:38.268064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269783 (+ 1719us) service_pool.cc:227] Handling call
0811 18:57:40.281515 (+2011732us) ts_tablet_manager.cc:1145] Deleting tablet e9f0e443ab84424dac307b745b496986
0811 18:57:40.281526 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.281623 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2011543}
I20260811 18:57:40.287838 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1180) took 2020 ms. Trace:
I20260811 18:57:40.288245 4167 rpcz_store.cc:276] 0811 18:57:38.267899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.295034 (+ 27135us) service_pool.cc:227] Handling call
0811 18:57:40.287672 (+1992638us) ts_tablet_manager.cc:1145] Deleting tablet 6710084f21734dea9c7ce1f43d1bf587
0811 18:57:40.287691 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.287818 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1992161}
I20260811 18:57:40.288964 4167 tablet_replica.cc:330] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.289393 4167 raft_consensus.cc:2229] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:40.293269 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7ce2b43b2134defb14d45112e4f5e5e consensus state has no leader
I20260811 18:57:40.294030 4167 raft_consensus.cc:2258] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.295076 4163 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:40.295320 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bac7ac254f4471397580dc510648dfc has no consensus state
I20260811 18:57:40.295384 4163 ts_tablet_manager.cc:1397] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.003s
I20260811 18:57:40.296180 4163 raft_consensus.cc:348] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.296411 4163 raft_consensus.cc:374] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.296463 4163 raft_consensus.cc:729] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.296635 4163 consensus_queue.cc:260] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.297252 4163 ts_tablet_manager.cc:1428] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:40.297559 4163 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.299288 4163 tablet_bootstrap.cc:654] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.300597 4167 ts_tablet_manager.cc:1905] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.307492 4162 ts_tablet_manager.cc:1428] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260811 18:57:40.309325 4167 ts_tablet_manager.cc:1918] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:40.316587 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a407d0c02054f5ca4370e8e1a1c4c71" } new_table_name: "default.Table_4821250a7fa84b68a74499b0d7a1228d" modify_external_catalogs: true
W20260811 18:57:40.317546 3824 leader_election.cc:334] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.317694 3824 leader_election.cc:302] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.317920 5630 raft_consensus.cc:2726] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.318301 3824 leader_election.cc:334] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.318526 3824 leader_election.cc:302] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.318836 5630 raft_consensus.cc:2726] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.319167 3824 consensus_peers.cc:487] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.319407 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01425dcfb75d41f6ad6349c65957ebac" } new_table_name: "default.Table_7f4309a3032b46a59a721bb32943bff8" modify_external_catalogs: true
I20260811 18:57:40.320807 5630 raft_consensus.cc:482] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.320930 5630 raft_consensus.cc:504] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.321538 5630 leader_election.cc:288] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.328706 4169 ts_tablet_manager.cc:1918] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.328878 4169 log.cc:1196] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0aee6beb2e70421083841520733087f8
I20260811 18:57:40.329257 4169 ts_tablet_manager.cc:1939] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:40.329763 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66e70ff397bd493992d2f08a30221b69 has no consensus state
I20260811 18:57:40.332235 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_075181f96b7b42059289018e831e2671" } new_table_name: "default.Table_9ec52027b1c944a69e731fed8ad2bd30" modify_external_catalogs: true
I20260811 18:57:40.332383 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1035) took 4504 ms. Trace:
I20260811 18:57:40.332473 4169 rpcz_store.cc:276] 0811 18:57:35.827679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.827838 (+ 159us) service_pool.cc:227] Handling call
0811 18:57:40.138180 (+4310342us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:40.138191 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.213178 (+ 74987us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.277840 (+ 64662us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.332361 (+ 54521us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4310134}
I20260811 18:57:40.332890 4169 tablet_replica.cc:330] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.333122 4169 raft_consensus.cc:2229] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.333304 4169 raft_consensus.cc:2258] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.333932 4169 ts_tablet_manager.cc:1905] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.340049 4167 log.cc:1196] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:40.318482 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.330789 5407 raft_consensus.cc:482] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.340639 5407 raft_consensus.cc:504] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.341315 5407 leader_election.cc:288] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.310797 4162 tablet_bootstrap.cc:492] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.342275 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a64df7c295c4d9baded1b63a9980581" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.342464 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1b92a533c58b43c69fd65a1ec335cb3a consensus state has no leader
W20260811 18:57:40.342959 3676 leader_election.cc:341] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.344071 4162 tablet_bootstrap.cc:654] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.344304 3678 leader_election.cc:334] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.344434 3678 leader_election.cc:302] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.344674 5407 raft_consensus.cc:2726] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.346910 4162 tablet_bootstrap.cc:492] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.347060 4162 ts_tablet_manager.cc:1397] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260811 18:57:40.347756 4162 raft_consensus.cc:348] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.347990 4162 raft_consensus.cc:374] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.348053 4162 raft_consensus.cc:729] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.348047 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0aee6beb2e70421083841520733087f8 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:40.348635 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c17b389153604aeeb9c985eaf661f44a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.349282 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c17b389153604aeeb9c985eaf661f44a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.349764 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet 49203e259d17449984dcd528df985537: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.353294 4167 ts_tablet_manager.cc:1939] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.353024 5642 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:40.359447 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1188) took 1888 ms. Trace:
I20260811 18:57:40.360322 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.360790 4162 consensus_queue.cc:260] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.361215 4167 rpcz_store.cc:276] 0811 18:57:38.471318 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.478361 (+ 7043us) service_pool.cc:227] Handling call
0811 18:57:40.288916 (+1810555us) ts_tablet_manager.cc:1145] Deleting tablet b4c7f500a5444bae9a015a5e903c0622
0811 18:57:40.288927 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.306248 (+ 17321us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.309217 (+ 2969us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.359417 (+ 50200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1810327}
I20260811 18:57:40.362432 4162 ts_tablet_manager.cc:1428] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260811 18:57:40.362740 4162 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.363312 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5e94a3f1b3a1418d8d5ff4ae641da751"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.363931 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e94a3f1b3a1418d8d5ff4ae641da751 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.364163 4162 tablet_bootstrap.cc:654] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.365332 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1190) took 1893 ms. Trace:
I20260811 18:57:40.365797 4167 rpcz_store.cc:276] 0811 18:57:38.471548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.482397 (+ 10849us) service_pool.cc:227] Handling call
0811 18:57:40.365190 (+1882793us) ts_tablet_manager.cc:1145] Deleting tablet 31bb11617f1548f88f534289706edb06
0811 18:57:40.365201 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.365311 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1882576}
I20260811 18:57:40.366487 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1191) took 1894 ms. Trace:
I20260811 18:57:40.366981 4167 rpcz_store.cc:276] 0811 18:57:38.471668 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.482665 (+ 10997us) service_pool.cc:227] Handling call
0811 18:57:40.366330 (+1883665us) ts_tablet_manager.cc:1145] Deleting tablet 76506ad179a9408c9e4cf8e4275ebddd
0811 18:57:40.366377 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.366469 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1883495}
I20260811 18:57:40.367514 4167 tablet_replica.cc:330] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.367584 3546 catalog_manager.cc:5495] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:40.368086 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:40.369325 4167 raft_consensus.cc:2229] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.369506 4167 raft_consensus.cc:2258] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.370088 4167 ts_tablet_manager.cc:1905] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.370855 5642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: loading cluster ID for follower catalog manager: success
W20260811 18:57:40.376416 3824 leader_election.cc:341] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:40.383720 3871 tablet_service.cc:1467] Processing CreateTablet for tablet c82456373d974f35846ade45e2398bbd (DEFAULT_TABLE table=default.Table_4b90b4ead71940e58e93af4a9804a7d8 [id=ac620652bd4b404da86f7dda6ed307d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.384250 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82456373d974f35846ade45e2398bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.387961 4163 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.388146 4163 ts_tablet_manager.cc:1397] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.006s
I20260811 18:57:40.388046 5728 tablet.cc:1722] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:40.388450 5407 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.388581 5407 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.389225 5407 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.389956 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.390203 3887 raft_consensus.cc:2454] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:40.390635 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:40.390714 3676 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:40.390654 4163 raft_consensus.cc:348] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.390980 4163 raft_consensus.cc:374] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.391045 4163 raft_consensus.cc:729] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.391081 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_388bf7f2862e42ebb5e3a779b1ade554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.391258 4163 consensus_queue.cc:260] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.391389 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.391826 4163 ts_tablet_manager.cc:1428] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 18:57:40.392109 4163 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.393184 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.393363 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:40.393865 4163 tablet_bootstrap.cc:654] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.394506 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_388bf7f2862e42ebb5e3a779b1ade554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.394969 5407 raft_consensus.cc:2781] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.395066 5407 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.395140 5407 raft_consensus.cc:3037] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.396366 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f20cec778f3248419eb734a5100ae8d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.396910 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f20cec778f3248419eb734a5100ae8d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.398056 5407 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.407722 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0e556811ad1643f1baa3bbf4edebd157
I20260811 18:57:40.409080 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.409322 3887 raft_consensus.cc:3037] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.412243 3887 raft_consensus.cc:2454] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.413586 5407 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.414808 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.415508 3676 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.416277 4167 ts_tablet_manager.cc:1918] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.416236 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b691c53749bc40a0a829753ba4c7e6aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.417223 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b691c53749bc40a0a829753ba4c7e6aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.420261 3871 tablet_service.cc:1467] Processing CreateTablet for tablet defce8dba232424ab74ebae42d6a078a (DEFAULT_TABLE table=default.Table_7a85117c40464174beed058911301729 [id=728fa9eb1e2947c8994d451b9a80eaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.420809 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defce8dba232424ab74ebae42d6a078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.416615 5407 raft_consensus.cc:2781] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.424954 4167 log.cc:1196] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8304ac7709ca4392baec4cfa78ef5b41
I20260811 18:57:40.425194 5407 raft_consensus.cc:686] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.425889 4167 ts_tablet_manager.cc:1939] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.426122 5407 consensus_queue.cc:237] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.427708 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1194) took 1149 ms. Trace:
I20260811 18:57:40.428449 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4a4afeb6069e4d3d9457fffcf5ec48f0
I20260811 18:57:40.428948 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 deleted and replaced by tablet fe48bbab3d7e4f7591883296a9996db7
I20260811 18:57:40.433809 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8edb4b4387a444abb3b70122ae70018e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.434778 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8edb4b4387a444abb3b70122ae70018e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.434768 4167 rpcz_store.cc:276] 0811 18:57:39.278179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.278257 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:40.367470 (+1089213us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:40.367481 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.410571 (+ 43090us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.416173 (+ 5602us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.427683 (+ 11510us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1086122}
I20260811 18:57:40.437476 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.437971 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.438643 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 9d09471428a5403da8b3bbf0f1597ed3 (DEFAULT_TABLE table=default.Table_e894321d446d4d6da9e866788ff2b1e7 [id=fd4705a377a94d3eb676f4dd0f37e096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.439096 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d09471428a5403da8b3bbf0f1597ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.440461 4167 tablet_replica.cc:330] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.440722 4167 raft_consensus.cc:2229] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.440891 4167 raft_consensus.cc:2258] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.441450 4167 ts_tablet_manager.cc:1905] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.442699 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5d93073a69c143ff9460248848a108d4
I20260811 18:57:40.448390 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6dab2aa50f394dabbbae787dca84b57a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.449004 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dab2aa50f394dabbbae787dca84b57a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.455476 5723 tablet.cc:1722] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:40.457772 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet ffa8a4a1c0cb4834be501bdd7d468472 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:40.458696 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 83141.9us
I20260811 18:57:40.460860 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2" } modify_external_catalogs: true
I20260811 18:57:40.461071 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2" } modify_external_catalogs: true
I20260811 18:57:40.462713 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 with delete_type TABLET_DATA_DELETED (replaced by tablet fe48bbab3d7e4f7591883296a9996db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.467928 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d93073a69c143ff9460248848a108d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.468961 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 8794dcb46ccc442e9271fc33453e4800 (DEFAULT_TABLE table=default.Table_f614f2bd34c949cd94f9493a086dd8c7 [id=9b6853170f7b4ebb9c266f96c7d99e10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.469528 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8794dcb46ccc442e9271fc33453e4800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.474979 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 56542f3afe304feb888f101961d855c7 (DEFAULT_TABLE table=default.Table_9919772109274064aee942f6a7f1220f [id=75a15771a07045039c110309e3dc0beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.475267 4163 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.475402 4163 ts_tablet_manager.cc:1397] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.010s
I20260811 18:57:40.475452 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56542f3afe304feb888f101961d855c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.476094 4163 raft_consensus.cc:348] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.476310 4163 raft_consensus.cc:374] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.476361 4163 raft_consensus.cc:729] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.477160 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 71a7eda2248c4f708bdc772ea3598dec
I20260811 18:57:40.479887 4167 ts_tablet_manager.cc:1918] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.480786 4167 log.cc:1196] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b4dc2df4c6464d71bb13fe25e0046758
I20260811 18:57:40.481312 4167 ts_tablet_manager.cc:1939] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.482920 5630 raft_consensus.cc:482] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.482786 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.483387 5630 raft_consensus.cc:504] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.483893 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.484136 5630 leader_election.cc:288] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.487383 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 71a7eda2248c4f708bdc772ea3598dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.486692 4163 consensus_queue.cc:260] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.487720 3824 leader_election.cc:334] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.487918 3824 leader_election.cc:302] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.490728 4163 ts_tablet_manager.cc:1428] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260811 18:57:40.491080 4163 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.492985 4163 tablet_bootstrap.cc:654] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.494007 4167 tablet_replica.cc:330] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.494532 4167 raft_consensus.cc:2229] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.495179 4167 raft_consensus.cc:2258] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.499025 4167 ts_tablet_manager.cc:1905] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.500532 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a49a2ac4cc894d7fb17104d0ea4fa6e7
I20260811 18:57:40.500977 3550 catalog_manager.cc:6565] ReplaceTablet: tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 deleted and replaced by tablet 79539e937f81437ab3b12b8465f12d97
I20260811 18:57:40.502851 5630 raft_consensus.cc:2726] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.509923 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4a5dc90bb3449ee94760cc397fd49f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.511152 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4a5dc90bb3449ee94760cc397fd49f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.522330 5641 raft_consensus.cc:482] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.522586 5641 raft_consensus.cc:504] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.523362 5641 leader_election.cc:288] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.526888 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 51297.5us
W20260811 18:57:40.527031 3678 leader_election.cc:334] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.527709 5641 raft_consensus.cc:482] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.527892 5641 raft_consensus.cc:504] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.528599 5641 leader_election.cc:288] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.528988 4167 ts_tablet_manager.cc:1918] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.529675 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1fc828f73642c88171d6ac827de90e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.530544 3676 leader_election.cc:341] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.530946 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a994b3f4d7d4c92946ad8d595e6ad2b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.530540 4167 log.cc:1196] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ffa8a4a1c0cb4834be501bdd7d468472
W20260811 18:57:40.531483 3676 leader_election.cc:341] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.531635 3676 leader_election.cc:302] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:40.532217 3678 leader_election.cc:334] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.532516 3678 leader_election.cc:302] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.532943 4167 ts_tablet_manager.cc:1939] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.533259 5641 raft_consensus.cc:2726] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.533584 5641 raft_consensus.cc:2726] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.544001 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 56300.6us
I20260811 18:57:40.544764 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_078a10661db44f05a3516fe2dd35d154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.547345 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_078a10661db44f05a3516fe2dd35d154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.558652 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb
I20260811 18:57:40.561429 4169 ts_tablet_manager.cc:1918] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:40.561607 4169 log.cc:1196] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e3021c51525846e199f279f988248af0
I20260811 18:57:40.562091 4169 ts_tablet_manager.cc:1939] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:40.562084 3554 catalog_manager.cc:5495] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:40.563891 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.564260 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (table default.Table_079d99e744054f2a8a2d6ac002aca4e3 [id=9c62165c055f4acea600e53873f88772]). Expected version 2 got 0
I20260811 18:57:40.564713 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:40.566551 4163 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.566728 4163 ts_tablet_manager.cc:1397] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.076s user 0.008s sys 0.011s
I20260811 18:57:40.567422 4163 raft_consensus.cc:348] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.567649 4163 raft_consensus.cc:374] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.567713 4163 raft_consensus.cc:729] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.567896 4163 consensus_queue.cc:260] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.570134 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 108372us
I20260811 18:57:40.577679 3871 tablet_service.cc:1467] Processing CreateTablet for tablet d8c6b62d2a684869b0ad31a1b25c7ebf (DEFAULT_TABLE table=default.Table_bf1e0bf055c44f45835974098ec141f8 [id=8f5e99a55df34267a0b32710772f1978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.578990 4163 ts_tablet_manager.cc:1428] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260811 18:57:40.582372 5701 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.582523 5701 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.583110 5701 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.590243 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c6b62d2a684869b0ad31a1b25c7ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.591312 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_298b25b6ed544a9fb0118e1af056b525" } new_table_name: "default.Table_df6343d393304ce6bd7043ce8b51cc50" modify_external_catalogs: true
I20260811 18:57:40.591789 4162 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.591933 4162 ts_tablet_manager.cc:1397] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.229s user 0.015s sys 0.024s
I20260811 18:57:40.592656 4162 raft_consensus.cc:348] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.592892 4162 raft_consensus.cc:374] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.592947 4162 raft_consensus.cc:729] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.593144 4162 consensus_queue.cc:260] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.593567 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6936ad30157d4323baa32612df13a29d" } new_table_name: "default.Table_70318db01b604cd79403f7bf85270fce" modify_external_catalogs: true
I20260811 18:57:40.593712 4162 ts_tablet_manager.cc:1428] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:40.594033 4162 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.601235 4162 tablet_bootstrap.cc:654] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.603937 5734 raft_consensus.cc:482] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.604216 5734 raft_consensus.cc:504] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.605386 4162 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.605578 4162 ts_tablet_manager.cc:1397] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260811 18:57:40.606931 4162 raft_consensus.cc:348] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.607196 4162 raft_consensus.cc:374] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.607272 4162 raft_consensus.cc:729] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.607484 4162 consensus_queue.cc:260] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.608083 4162 ts_tablet_manager.cc:1428] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:40.608394 4162 tablet_bootstrap.cc:492] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:40.609248 3678 leader_election.cc:334] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.609771 5734 leader_election.cc:288] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.590822 4163 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.607118 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1e244addf5462499fcadaa3332cba1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.610257 4162 tablet_bootstrap.cc:654] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.610711 3676 leader_election.cc:341] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.610831 3676 leader_election.cc:302] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.611446 5735 tablet.cc:1722] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:40.613119 4163 tablet_bootstrap.cc:654] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.616297 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f94d755a69ab49dcbbc91974db72cb77" } new_table_name: "default.Table_2a545f0339f848b6a536ac32accb3d2b" modify_external_catalogs: true
I20260811 18:57:40.616786 5733 raft_consensus.cc:2726] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.618453 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_25c8439e034242f7bc3d1c546188fc8e" } new_table_name: "default.Table_f9902d67710648848c10bb18357257a6" modify_external_catalogs: true
I20260811 18:57:40.628548 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 72f11748cf984942ba99bda4ad6605ab
I20260811 18:57:40.629180 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet cf6c8d542ae2489ba466ffadd518db12 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:40.629812 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_59d30e9c40554addab5dc9b7c1d11432"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.630414 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59d30e9c40554addab5dc9b7c1d11432 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.646085 4162 tablet_bootstrap.cc:492] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.646397 4162 ts_tablet_manager.cc:1397] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.003s
I20260811 18:57:40.647333 4162 raft_consensus.cc:348] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.647660 4162 raft_consensus.cc:374] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.647805 4162 raft_consensus.cc:729] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.648126 4162 consensus_queue.cc:260] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.648916 4162 ts_tablet_manager.cc:1428] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:40.649854 4162 tablet_bootstrap.cc:492] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.651999 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28" } new_table_name: "default.Table_7fcad849494e46c1bcef25d34ef0cd78" modify_external_catalogs: true
W20260811 18:57:40.664992 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 81393.2us
W20260811 18:57:40.665505 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4c7f500a5444bae9a015a5e903c0622. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.666559 5733 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.666831 5733 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.669538 4162 tablet_bootstrap.cc:654] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.669590 5733 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.673803 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.674124 3887 raft_consensus.cc:2454] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:40.674773 3676 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:40.675064 5733 raft_consensus.cc:2781] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.675205 5733 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.675346 5733 raft_consensus.cc:3037] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:40.677346 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.680563 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6" } modify_external_catalogs: true
I20260811 18:57:40.680809 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6" } modify_external_catalogs: true
I20260811 18:57:40.689117 5593 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.689321 5593 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.689882 5593 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.690541 5630 consensus_queue.cc:1035] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:40.694589 3824 leader_election.cc:341] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 6b4de110bd3348f0b0dae21c9f2b6a19
W20260811 18:57:40.696766 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1922c7317c24aa984945aaaa6ce8003 consensus state has no leader
I20260811 18:57:40.695129 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1036) took 4814 ms. Trace:
I20260811 18:57:40.699504 4169 rpcz_store.cc:276] 0811 18:57:35.880779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.881029 (+ 250us) service_pool.cc:227] Handling call
0811 18:57:40.332842 (+4451813us) ts_tablet_manager.cc:1145] Deleting tablet e3021c51525846e199f279f988248af0
0811 18:57:40.332854 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.448406 (+115552us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.561256 (+112850us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.695085 (+133829us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4451518}
I20260811 18:57:40.699913 4169 tablet_replica.cc:330] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.703034 4163 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.703213 4163 ts_tablet_manager.cc:1397] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.112s user 0.003s sys 0.006s
I20260811 18:57:40.704037 4163 raft_consensus.cc:348] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.704291 4163 raft_consensus.cc:374] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.704363 4163 raft_consensus.cc:729] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.704571 4163 consensus_queue.cc:260] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.708248 4163 ts_tablet_manager.cc:1428] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:40.708621 4163 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:40.719354 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:40.719807 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38" } modify_external_catalogs: true
I20260811 18:57:40.720011 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38" } modify_external_catalogs: true
W20260811 18:57:40.721551 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae75d6652c704e1995966c6dd4c06307 consensus state has no leader
W20260811 18:57:40.723438 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9948c33ed58b4160932be82ef9615ba5 consensus state has no leader
I20260811 18:57:40.724624 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c925e9ac591346b1a5142f338c68bcc6" } modify_external_catalogs: true
I20260811 18:57:40.724819 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c925e9ac591346b1a5142f338c68bcc6" } modify_external_catalogs: true
I20260811 18:57:40.726410 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_22ffcd40375b45aab1616fa1fb6f030d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.727078 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22ffcd40375b45aab1616fa1fb6f030d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.729166 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b50efab3c782447cad8e45b1fa51aa1f
I20260811 18:57:40.734637 4169 raft_consensus.cc:2229] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.734966 4169 raft_consensus.cc:2258] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.735913 4169 ts_tablet_manager.cc:1905] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.752440 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d] alter complete (version 1)
I20260811 18:57:40.688943 5733 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.721388 4162 tablet_bootstrap.cc:492] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.768213 4162 ts_tablet_manager.cc:1397] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.118s user 0.009s sys 0.004s
I20260811 18:57:40.769117 4162 raft_consensus.cc:348] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.769200 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.769349 4162 raft_consensus.cc:374] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.769404 4162 raft_consensus.cc:729] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.769444 3887 raft_consensus.cc:3037] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.769652 4162 consensus_queue.cc:260] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.723304 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:40.770314 3746 raft_consensus.cc:2454] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:40.772637 3887 raft_consensus.cc:2454] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.705665 5734 raft_consensus.cc:482] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.773872 5734 raft_consensus.cc:504] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.775307 5734 leader_election.cc:288] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.760870 5407 raft_consensus.cc:482] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.778431 5407 raft_consensus.cc:504] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.779078 5407 leader_election.cc:288] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.765305 5641 raft_consensus.cc:482] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.764583 5728 tablet.cc:1722] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:40.779400 5641 raft_consensus.cc:504] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.780015 5641 leader_election.cc:288] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.778023 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.780483 3678 leader_election.cc:334] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.783810 3678 leader_election.cc:334] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.784041 3678 leader_election.cc:334] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.784823 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0116997a1c64e658045c319f1c45861" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.784932 3676 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.785287 5734 raft_consensus.cc:2781] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.785346 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe24bb6edb944008ff9cc73faa7374c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.785663 3676 leader_election.cc:341] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.785794 3676 leader_election.cc:302] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.786187 5641 raft_consensus.cc:2726] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.785653 5734 raft_consensus.cc:686] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.786892 5734 consensus_queue.cc:237] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.789413 3676 leader_election.cc:341] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.789541 3676 leader_election.cc:302] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.789599 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf7720e31d84ae2ab592da1e7a3132a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.789956 5641 raft_consensus.cc:2726] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.790189 5733 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.790298 3676 leader_election.cc:341] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.790405 3676 leader_election.cc:302] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:40.797078 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 222712us
I20260811 18:57:40.804664 4163 tablet_bootstrap.cc:654] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.806788 5641 raft_consensus.cc:2726] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.815836 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_525108ad39d242bc818b5bb86a5c0103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.816668 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_525108ad39d242bc818b5bb86a5c0103 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.818450 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_beb35f174ef644f5994dd0282f0d4ee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.818984 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_beb35f174ef644f5994dd0282f0d4ee3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.820214 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65f6fe5195b443fcbd5f077f1a147107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.820669 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65f6fe5195b443fcbd5f077f1a147107 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.821846 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_58f2aeab56c6446184cfeddcb1134314" } modify_external_catalogs: true
I20260811 18:57:40.822029 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_58f2aeab56c6446184cfeddcb1134314" } modify_external_catalogs: true
I20260811 18:57:40.825174 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf1c037a37504b3fad9612f5a734dd33
I20260811 18:57:40.816634 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 79539e937f81437ab3b12b8465f12d97) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.826810 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a7946503bec9446d8deb02448eb231d5
I20260811 18:57:40.832841 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.833756 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a54921b815f2408c96bdf18e243041d5 (DEFAULT_TABLE table=default.Table_23655b2975534b81b076fd031ea5022a [id=e040f0cd609644928bb9690a053fa426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.834780 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.840480 3859 tablet_service.cc:1467] Processing CreateTablet for tablet f891108d44c44296934924941981b568 (DEFAULT_TABLE table=default.Table_4b752724648a499980ad71fd596a731f [id=78f3a4a7d6ad41278417b8eef697d0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.845278 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0bac7ac254f4471397580dc510648dfc (DEFAULT_TABLE table=default.Table_69847be1563a4aad8d093d935864f531 [id=6a7e6156c0c247f68bcdb76ed1eb6365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.849972 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 756062a54d144339b91f8c68d9edceab (DEFAULT_TABLE table=default.Table_e142d545d8b249f3835bce3bbef9f94e [id=51b0aa68c2e749768c886732760d4f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.868304 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54921b815f2408c96bdf18e243041d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.868907 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f891108d44c44296934924941981b568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.869251 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bac7ac254f4471397580dc510648dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.869573 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756062a54d144339b91f8c68d9edceab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.878120 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3aa1c735a06c46e9833fdb958cac274d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.878929 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3aa1c735a06c46e9833fdb958cac274d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.881009 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f3b91b31a884708a27160ebfca75cab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.881682 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f3b91b31a884708a27160ebfca75cab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.886219 5733 consensus_queue.cc:1035] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.892804 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" } modify_external_catalogs: true
W20260811 18:57:40.912899 3678 consensus_peers.cc:487] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.919169 3862 tablet_service.cc:1467] Processing CreateTablet for tablet ce6a80313c854c17913db64aeab9acf3 (DEFAULT_TABLE table=default.Table_c4b27a9eb0994efe9f4176f74c16a62e [id=6e29d4572cd64f8cb70940f4119296e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.919744 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6a80313c854c17913db64aeab9acf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.935029 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b50efab3c782447cad8e45b1fa51aa1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.935449 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a07f682b68114c2b9cb0df27a94406ec (DEFAULT_TABLE table=default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.936100 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07f682b68114c2b9cb0df27a94406ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.943498 4162 ts_tablet_manager.cc:1428] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.175s user 0.004s sys 0.007s
I20260811 18:57:40.950555 4163 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:40.950781 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 92904.8us
I20260811 18:57:40.958736 4162 tablet_bootstrap.cc:492] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.967087 4163 ts_tablet_manager.cc:1397] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.259s user 0.005s sys 0.000s
I20260811 18:57:40.962435 5742 raft_consensus.cc:482] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.968542 5742 raft_consensus.cc:504] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.968181 5630 raft_consensus.cc:482] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.970994 5630 raft_consensus.cc:504] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.971894 5630 leader_election.cc:288] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:40.972911 3678 leader_election.cc:334] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.974181 4163 raft_consensus.cc:348] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.979110 5742 leader_election.cc:288] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.986301 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e3021c51525846e199f279f988248af0 (table default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]) successfully deleted
I20260811 18:57:40.998678 4163 raft_consensus.cc:374] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.998947 4163 raft_consensus.cc:729] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.999325 4163 consensus_queue.cc:260] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.976478 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0540e4e43ea34fbfbc66b6035c60684d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.978869 3859 tablet_service.cc:1467] Processing CreateTablet for tablet bba6561b8c4b4c728db18446a9a91f30 (DEFAULT_TABLE table=default.Table_5affb8acc7e5466ba5ed988728ba50cb [id=0a26d099a1324a9aa8005962a8d9bd79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.000206 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba6561b8c4b4c728db18446a9a91f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.982708 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet bf1c037a37504b3fad9612f5a734dd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:41.003258 3676 leader_election.cc:341] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.003432 3676 leader_election.cc:302] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.009931 5742 raft_consensus.cc:2726] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.983560 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 997be6173a194bbe91c6c0b9046d1a0e (DEFAULT_TABLE table=default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae [id=7586915510704916afd712ab81f9f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.984809 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet a7946503bec9446d8deb02448eb231d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.985358 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 01fe8e896b2246fabd2b6fae396bf085 (DEFAULT_TABLE table=default.Table_f83d8784174f40e29b96eba2de396065 [id=fdb879cbb1cb4f498f94d14048b87e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.010891 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01fe8e896b2246fabd2b6fae396bf085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.985924 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 81e467ea40364946b7c0bc4b5592996b (DEFAULT_TABLE table=default.Table_2486985deae44613a5f8d342c8236784 [id=74d8ef880ca24497ace800c16e041dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.014761 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e467ea40364946b7c0bc4b5592996b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.017249 3361 master-stress-test.cc:578] Killing test thread 1/25
W20260811 18:57:41.024854 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae02c9b0b066474894546829aed304e3 (table default.Table_76b86d4f079c45b1a7f34a195fc97dfb [id=137cef1d67054dc6be05fcef9b22c375], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:40.986475 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f11748cf984942ba99bda4ad6605ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.975323 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 86f215c21d184dd6ba828fa31286cce1 (DEFAULT_TABLE table=default.Table_d8f35609e86b47aa9a8ab711834da08f [id=72dfc9b4df824dcd92241af7ab88c68e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.029963 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f215c21d184dd6ba828fa31286cce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.048406 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 997be6173a194bbe91c6c0b9046d1a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.067209 4162 tablet_bootstrap.cc:654] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.069007 4163 ts_tablet_manager.cc:1428] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.096s user 0.002s sys 0.000s
I20260811 18:57:41.069476 4163 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.071363 4163 tablet_bootstrap.cc:654] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.050889 3824 leader_election.cc:334] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.072559 3824 leader_election.cc:302] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.072957 5630 raft_consensus.cc:2726] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.086077 4169 ts_tablet_manager.cc:1918] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.086373 4169 log.cc:1196] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:41.086845 4169 ts_tablet_manager.cc:1939] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.086078 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88ee0abdcd9943568a07d73017ff598a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.087409 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88ee0abdcd9943568a07d73017ff598a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.087821 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" } modify_external_catalogs: true
W20260811 18:57:41.119937 3824 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:41.120177 3824 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.120416 5630 raft_consensus.cc:2781] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.120493 5630 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.120563 5630 raft_consensus.cc:3037] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.123310 5630 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.123991 5630 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.128211 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b149c08055014e08aad5ad5be387047b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.128815 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b149c08055014e08aad5ad5be387047b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.129477 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (table default.Table_6c8484897dd046ab8499d1069356cdcd [id=221d836949c341a09235288dc5802137]). Expected version 3 got 0
I20260811 18:57:41.129726 3546 catalog_manager.cc:5495] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:41.144066 5630 raft_consensus.cc:482] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.144272 5630 raft_consensus.cc:504] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.144912 5630 leader_election.cc:288] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.148226 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c66e99a00ba4953b1faed44821e14ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.148929 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c66e99a00ba4953b1faed44821e14ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.150770 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aadee0d920fc4be5aa7f965bc9db93e4" } new_table_name: "default.Table_1a248177f5ce4689a09bdb2d292adae1" modify_external_catalogs: true
I20260811 18:57:41.151306 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 76e71af5ee8b463eb158ac9ae76ff252
I20260811 18:57:41.152717 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1612fbcb897947ee843430ed61c2e50e" } new_table_name: "default.Table_cef814687d014504b5a7a569342e4bb4" modify_external_catalogs: true
I20260811 18:57:41.154788 5630 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.154959 5630 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.155576 5630 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.169507 4162 tablet_bootstrap.cc:492] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.170500 4162 ts_tablet_manager.cc:1397] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.212s user 0.004s sys 0.000s
I20260811 18:57:41.171536 4162 raft_consensus.cc:348] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.172752 4162 raft_consensus.cc:374] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.173117 4162 raft_consensus.cc:729] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.158317 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1037) took 5220 ms. Trace:
I20260811 18:57:41.175781 4169 rpcz_store.cc:276] 0811 18:57:35.938786 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.939004 (+ 218us) service_pool.cc:227] Handling call
0811 18:57:40.699843 (+4760839us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:40.699855 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.932192 (+232337us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.085957 (+153765us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.158287 (+ 72330us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70432,"tablet-delete.queue_time_us":4760561}
I20260811 18:57:41.178586 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1049) took 5044 ms. Trace:
I20260811 18:57:41.180433 4169 rpcz_store.cc:276] 0811 18:57:36.134037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.134647 (+ 610us) service_pool.cc:227] Handling call
0811 18:57:41.178416 (+5043769us) ts_tablet_manager.cc:1145] Deleting tablet 1700ee40ac0d40198c5b8a161a90b6b6
0811 18:57:41.178433 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.178562 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5043504}
I20260811 18:57:41.182070 3361 master-stress-test.cc:580] Killed test thread 1/25
I20260811 18:57:41.182070 5742 raft_consensus.cc:482] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.182224 3361 master-stress-test.cc:578] Killing test thread 2/25
I20260811 18:57:41.182368 3361 master-stress-test.cc:580] Killed test thread 2/25
I20260811 18:57:41.182456 3361 master-stress-test.cc:578] Killing test thread 3/25
I20260811 18:57:41.182585 5742 raft_consensus.cc:504] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.184267 5742 leader_election.cc:288] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.183360 4162 consensus_queue.cc:260] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.186609 3678 leader_election.cc:334] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.175094 5630 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.179673 5593 consensus_queue.cc:1035] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.187844 5630 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.188462 5630 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.180065 3824 leader_election.cc:341] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: e18242f762d8407db0e2f8716890feb4
W20260811 18:57:41.189237 3824 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.191891 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0fde489a381549f394a78e98c8cade0f (DEFAULT_TABLE table=default.Table_8e26f75764214905965fa0f514cb0028 [id=3aa9faf1a865494e9e6b33917e0cd501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.192067 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 26f55fe6a9e840a4a5e7ba6bf727c779 (DEFAULT_TABLE table=default.Table_724b9320db2141f39b2511549a1aa8d4 [id=de7aea74dccc46f68bc416ad0f023c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.192525 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fde489a381549f394a78e98c8cade0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.192932 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 625daea5324b4494996a51f0f9e9bcb7 (DEFAULT_TABLE table=default.Table_a6af8048b8f14af899345d8213953792 [id=677b3b20925544f28c2fb9ae18878c69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.195566 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 625daea5324b4494996a51f0f9e9bcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.196892 3361 master-stress-test.cc:580] Killed test thread 3/25
I20260811 18:57:41.197008 3361 master-stress-test.cc:578] Killing test thread 4/25
I20260811 18:57:41.199318 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f55fe6a9e840a4a5e7ba6bf727c779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.207298 4162 ts_tablet_manager.cc:1428] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.036s user 0.001s sys 0.001s
I20260811 18:57:41.210968 3718 tablet_service.cc:1467] Processing CreateTablet for tablet eea325ca357146fcaa408d87803da349 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.215816 3713 tablet_service.cc:1467] Processing CreateTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 (DEFAULT_TABLE table=default.Table_9042bc231f0841a4a370be5e6251dd6e [id=044362a3a169467396a6be8959d1d07b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.216890 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 929586965ce6468c82950102cfed41da (DEFAULT_TABLE table=default.Table_bce1d6623de1467198e1dfaf763f08cf [id=a85df4c40f944b598dcd718a4e9a3a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.217549 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e with delete_type TABLET_DATA_DELETED (replaced by tablet a07f682b68114c2b9cb0df27a94406ec) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.218809 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ae96e10a2ed4407c9d42a359d82d389e
I20260811 18:57:41.221726 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bef031fd83e44076810e019e0038f70f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.222424 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bef031fd83e44076810e019e0038f70f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.223732 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a65337aa3b94589a51dd41812870aca" } new_table_name: "default.Table_73a91eab61b94f63a4d5123a9e446693" modify_external_catalogs: true
I20260811 18:57:41.224882 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.225129 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c12987db235f4f4fb949580479275ec2" } modify_external_catalogs: true
I20260811 18:57:41.225312 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c12987db235f4f4fb949580479275ec2" } modify_external_catalogs: true
I20260811 18:57:41.225932 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet bc38e6760754449fbd6209d765cce70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.226781 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet ab03996c5c624e5eb17267b124e27f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.227322 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61e3f65a0ee948fab69aa4add687f2bc" } new_table_name: "default.Table_c8f1686d290b4930adcdda63173c77bc" modify_external_catalogs: true
I20260811 18:57:41.227797 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 441feddae43741678ef0f64888b5c6f3 (DEFAULT_TABLE table=default.Table_b081a3f48c9048ecb88f1bd4571f934b [id=54b7f379f89e4c009585c94a5f7e1a55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.228272 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 6688ed13d1334ed98503b21a937facd2 (DEFAULT_TABLE table=default.Table_644eac4afde147dea1b6dc8cd69dc719 [id=583ba00a08184bc38ff31f87596e3788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.228773 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_078a10661db44f05a3516fe2dd35d154" } new_table_name: "default.Table_d82e6143f45747ddafae8a1e2651b9a0" modify_external_catalogs: true
I20260811 18:57:41.229921 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 5d1b94cb25b84ae1a4c22801b1a98094 (DEFAULT_TABLE table=default.Table_04519ba2021244b6922bea0e5b79ecb6 [id=02744826873c4079a8d983c8e0d71c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.229983 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ab9fe26e8e1e4c3886af83ac3e6956f5 (DEFAULT_TABLE table=default.Table_7e6c466c96e8465c9677dcaddee8819a [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.183887 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b203df4124364737839bf4c1ae072988" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.230515 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b9fbac57c9c44a028f5b258593aff86d (DEFAULT_TABLE table=default.Table_51582df654e7406dbc574ba8de215baf [id=0eb328384d824024bcf7782bc4cf5b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.230947 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.231005 3676 leader_election.cc:341] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.231161 3676 leader_election.cc:302] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.231601 3824 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.231633 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.232051 3824 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.232498 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b3ef682132a04b338b03a4387999d6b9 (DEFAULT_TABLE table=default.Table_4fd6bf06dd964e719ce1470aec9f60bc [id=54c1a4605328449dbb0a4928e192e271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.232781 5630 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.232841 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.233352 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.233830 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.167547 4163 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.235615 4163 ts_tablet_manager.cc:1397] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.166s user 0.004s sys 0.000s
I20260811 18:57:41.236449 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.236867 4163 raft_consensus.cc:348] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.237188 4163 raft_consensus.cc:374] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.237319 4163 raft_consensus.cc:729] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.237613 4163 consensus_queue.cc:260] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.238698 4163 ts_tablet_manager.cc:1428] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:41.239214 5742 raft_consensus.cc:2726] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.190683 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1051) took 5055 ms. Trace:
I20260811 18:57:41.206441 5593 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.239630 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1fc828f73642c88171d6ac827de90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.239591 5593 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.240204 5593 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.240304 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b9fbac57c9c44a028f5b258593aff86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.240463 4169 rpcz_store.cc:276] 0811 18:57:36.134871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.134917 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:41.190523 (+5055606us) ts_tablet_manager.cc:1145] Deleting tablet 38a4783168554053aa953dba5e893b6b
0811 18:57:41.190538 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.190664 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5055422}
I20260811 18:57:41.242857 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.243453 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.245362 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 58bc967770de4ed486b638fb6b174db8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.245787 5642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: acquiring CA information for follower catalog manager: success
I20260811 18:57:41.247128 3361 master-stress-test.cc:580] Killed test thread 4/25
I20260811 18:57:41.247304 3361 master-stress-test.cc:578] Killing test thread 5/25
I20260811 18:57:41.247270 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.247396 3361 master-stress-test.cc:580] Killed test thread 5/25
I20260811 18:57:41.247473 3361 master-stress-test.cc:578] Killing test thread 6/25
I20260811 18:57:41.249136 4163 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.251129 4163 tablet_bootstrap.cc:654] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.252120 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 consensus state has no leader
I20260811 18:57:41.252399 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 54fa32f8fa1144c4bd9dcd00a550001a from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:41.254979 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c575ba3060a140759484bea1462ffd73 has no consensus state
I20260811 18:57:41.255266 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 1f9621bbd0f94223a5d36bc01487f615 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.255551 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.255795 3740 raft_consensus.cc:3037] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.255826 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 968fa8b8912b40a8bdba77db1d39c5c0 (table default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]) successfully deleted
I20260811 18:57:41.257097 3739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.257476 3739 raft_consensus.cc:2454] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:41.260984 3740 raft_consensus.cc:2454] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.264325 3546 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b] alter complete (version 3)
W20260811 18:57:41.275591 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1700ee40ac0d40198c5b8a161a90b6b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1700ee40ac0d40198c5b8a161a90b6b6 already in progress: creating tablet
W20260811 18:57:41.275913 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 38a4783168554053aa953dba5e893b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38a4783168554053aa953dba5e893b6b already in progress: creating tablet
I20260811 18:57:41.277542 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet eea325ca357146fcaa408d87803da349 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.278021 3361 master-stress-test.cc:580] Killed test thread 6/25
I20260811 18:57:41.278196 3361 master-stress-test.cc:578] Killing test thread 7/25
I20260811 18:57:41.278307 3361 master-stress-test.cc:580] Killed test thread 7/25
I20260811 18:57:41.278424 3361 master-stress-test.cc:578] Killing test thread 8/25
I20260811 18:57:41.278527 3361 master-stress-test.cc:580] Killed test thread 8/25
I20260811 18:57:41.278635 3361 master-stress-test.cc:578] Killing test thread 9/25
I20260811 18:57:41.278730 3361 master-stress-test.cc:580] Killed test thread 9/25
I20260811 18:57:41.278808 3361 master-stress-test.cc:578] Killing test thread 10/25
I20260811 18:57:41.278889 3361 master-stress-test.cc:580] Killed test thread 10/25
I20260811 18:57:41.278972 3361 master-stress-test.cc:578] Killing test thread 11/25
W20260811 18:57:41.282564 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a62cdd333ad8494897ae8ec5afd5af74 consensus state has no leader
I20260811 18:57:41.283800 3361 master-stress-test.cc:580] Killed test thread 11/25
I20260811 18:57:41.283941 3361 master-stress-test.cc:578] Killing test thread 12/25
I20260811 18:57:41.284019 3361 master-stress-test.cc:580] Killed test thread 12/25
I20260811 18:57:41.284081 3361 master-stress-test.cc:578] Killing test thread 13/25
I20260811 18:57:41.284183 3361 master-stress-test.cc:580] Killed test thread 13/25
I20260811 18:57:41.284260 3361 master-stress-test.cc:578] Killing test thread 14/25
I20260811 18:57:41.253858 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1052) took 5109 ms. Trace:
I20260811 18:57:41.284504 4169 rpcz_store.cc:276] 0811 18:57:36.144056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.308615 (+164559us) service_pool.cc:227] Handling call
0811 18:57:41.240865 (+4932250us) ts_tablet_manager.cc:1145] Deleting tablet 374c94a12e374d0e82180421c80d92d0
0811 18:57:41.253686 (+ 12821us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.253831 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4931905}
I20260811 18:57:41.285113 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1062) took 4935 ms. Trace:
I20260811 18:57:41.285225 4169 rpcz_store.cc:276] 0811 18:57:36.349751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.349811 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:41.284939 (+4935128us) ts_tablet_manager.cc:1145] Deleting tablet 60091bd891a94cc4aac6ef47673a4ee5
0811 18:57:41.284958 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.285090 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4932595}
I20260811 18:57:41.285635 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1063) took 4935 ms. Trace:
I20260811 18:57:41.285710 4169 rpcz_store.cc:276] 0811 18:57:36.349867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.351599 (+ 1732us) service_pool.cc:227] Handling call
0811 18:57:41.285496 (+4933897us) ts_tablet_manager.cc:1145] Deleting tablet fe080f2071964da6b54f1ee5af98d918
0811 18:57:41.285510 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.285619 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4930726}
I20260811 18:57:41.285996 4169 tablet_replica.cc:330] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.286212 4169 raft_consensus.cc:2229] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.258644 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.274540 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ae96e10a2ed4407c9d42a359d82d389e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.286577 4169 raft_consensus.cc:2258] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.287227 4169 ts_tablet_manager.cc:1905] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.291980 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:41.292444 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:41.292739 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:41.293010 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:41.293577 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:41.294140 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 452ee450778e4d33adfa7e2745b5d970 (table default.Table_4d6c4d770dc949e78c1707ddbab71740 [id=48f45fcc06e0418185841260a06cb98c]) successfully deleted
I20260811 18:57:41.295084 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dfb6ac49f1d4403dad38a3125264088a
W20260811 18:57:41.296679 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 03c50dcffac54c7bac210eeaed340ca6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03c50dcffac54c7bac210eeaed340ca6 already in progress: creating tablet
W20260811 18:57:41.297286 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 73c519528f75423195b6bdf6683fd91b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c519528f75423195b6bdf6683fd91b already in progress: creating tablet
W20260811 18:57:41.297538 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b already in progress: creating tablet
W20260811 18:57:41.282248 3824 leader_election.cc:334] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.298501 3824 leader_election.cc:302] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.298516 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 3ad5f39f59b8447cb3f15b9646e4b852 (table default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]) successfully deleted
W20260811 18:57:41.298753 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7cb3e997b4454598bdce102e92296887 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7cb3e997b4454598bdce102e92296887 already in progress: creating tablet
I20260811 18:57:41.298817 5593 raft_consensus.cc:2726] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.299037 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8f9b279c6c7344f1931b81e10248b1b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f9b279c6c7344f1931b81e10248b1b6 already in progress: creating tablet
W20260811 18:57:41.299230 3824 consensus_peers.cc:487] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:41.299353 3824 consensus_peers.cc:487] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:41.299360 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ea579c33f8b14e9fa7fa389ce7f45931 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea579c33f8b14e9fa7fa389ce7f45931 already in progress: creating tablet
W20260811 18:57:41.299546 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7f4e1fc5686944dfa3eb57702c810eb8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f4e1fc5686944dfa3eb57702c810eb8 already in progress: creating tablet
W20260811 18:57:41.299759 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:41.300526 3824 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.300828 5593 raft_consensus.cc:2781] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.300910 5593 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.300974 5593 raft_consensus.cc:3037] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.300978 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet c44eee516d634d9a986107b55410dfc3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.303869 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea325ca357146fcaa408d87803da349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.304107 5593 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.304764 5593 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.306891 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929586965ce6468c82950102cfed41da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307292 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 441feddae43741678ef0f64888b5c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307593 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688ed13d1334ed98503b21a937facd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307842 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9fe26e8e1e4c3886af83ac3e6956f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.308038 5757 tablet_replica.cc:330] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.308315 5757 raft_consensus.cc:2229] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:41.308714 5757 raft_consensus.cc:2258] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.309324 5757 ts_tablet_manager.cc:1905] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.308059 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d1b94cb25b84ae1a4c22801b1a98094. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.310448 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45e040f8120a4edfba89c1a8d55deb4a has no consensus state
I20260811 18:57:41.314510 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ef682132a04b338b03a4387999d6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.317816 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cef27540ff46aebc9b6f4760ae0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.318210 4169 ts_tablet_manager.cc:1918] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.322516 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9fbac57c9c44a028f5b258593aff86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.321542 5742 consensus_queue.cc:1035] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.324677 3361 master-stress-test.cc:580] Killed test thread 14/25
I20260811 18:57:41.324790 3361 master-stress-test.cc:578] Killing test thread 15/25
I20260811 18:57:41.324860 3361 master-stress-test.cc:580] Killed test thread 15/25
I20260811 18:57:41.324921 3361 master-stress-test.cc:578] Killing test thread 16/25
I20260811 18:57:41.324988 3361 master-stress-test.cc:580] Killed test thread 16/25
I20260811 18:57:41.325062 3361 master-stress-test.cc:578] Killing test thread 17/25
I20260811 18:57:41.325133 3361 master-stress-test.cc:580] Killed test thread 17/25
I20260811 18:57:41.325194 3361 master-stress-test.cc:578] Killing test thread 18/25
I20260811 18:57:41.325253 3361 master-stress-test.cc:580] Killed test thread 18/25
I20260811 18:57:41.325316 3361 master-stress-test.cc:578] Killing test thread 19/25
I20260811 18:57:41.325389 3361 master-stress-test.cc:580] Killed test thread 19/25
I20260811 18:57:41.325449 3361 master-stress-test.cc:578] Killing test thread 20/25
I20260811 18:57:41.325511 3361 master-stress-test.cc:580] Killed test thread 20/25
I20260811 18:57:41.325570 3361 master-stress-test.cc:578] Killing test thread 21/25
I20260811 18:57:41.325632 3361 master-stress-test.cc:580] Killed test thread 21/25
I20260811 18:57:41.325693 3361 master-stress-test.cc:578] Killing test thread 22/25
I20260811 18:57:41.327144 4162 tablet_bootstrap.cc:492] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.327888 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 2 got 0
I20260811 18:57:41.337205 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 58bc967770de4ed486b638fb6b174db8
I20260811 18:57:41.337733 3553 catalog_manager.cc:6565] ReplaceTablet: tablet 58bc967770de4ed486b638fb6b174db8 deleted and replaced by tablet f336fdd55965432f83b2d1c436fdf1fa
I20260811 18:57:41.343441 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eea325ca357146fcaa408d87803da349
I20260811 18:57:41.343839 3549 catalog_manager.cc:6565] ReplaceTablet: tablet eea325ca357146fcaa408d87803da349 deleted and replaced by tablet a4e0b84bdb134a459d2d9afd046326c2
I20260811 18:57:41.344518 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c44eee516d634d9a986107b55410dfc3
I20260811 18:57:41.345045 3548 catalog_manager.cc:6565] ReplaceTablet: tablet c44eee516d634d9a986107b55410dfc3 deleted and replaced by tablet 5faabb319fc84f24ab548a707be284f1
I20260811 18:57:41.345803 3361 master-stress-test.cc:580] Killed test thread 22/25
I20260811 18:57:41.345944 3361 master-stress-test.cc:578] Killing test thread 23/25
I20260811 18:57:41.341475 4162 tablet_bootstrap.cc:654] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.299768 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d21973bc8e514a089c2beb24a686060f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d21973bc8e514a089c2beb24a686060f already in progress: creating tablet
W20260811 18:57:41.348656 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 48896us
W20260811 18:57:41.348932 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2f2f5ce2b1d14a9a988cff644e196088 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f2f5ce2b1d14a9a988cff644e196088 already in progress: creating tablet
W20260811 18:57:41.349256 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f2b66196716a4ba3893b6b8bfec2b8c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2b66196716a4ba3893b6b8bfec2b8c0 already in progress: creating tablet
W20260811 18:57:41.349512 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a3973dfdaac746a6af7a402277d2711b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3973dfdaac746a6af7a402277d2711b already in progress: creating tablet
W20260811 18:57:41.349730 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 15dd14e2b4bd4a08b5b08f532a55291b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15dd14e2b4bd4a08b5b08f532a55291b already in progress: creating tablet
W20260811 18:57:41.349941 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 75ceba7bfc8240268b16c789f07813d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75ceba7bfc8240268b16c789f07813d1 already in progress: creating tablet
W20260811 18:57:41.350165 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0495c5b770b34d54afb1379a6e36204c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0495c5b770b34d54afb1379a6e36204c already in progress: creating tablet
W20260811 18:57:41.350411 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5a75a480f38047d880b0c1fa51403786 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a75a480f38047d880b0c1fa51403786 already in progress: creating tablet
W20260811 18:57:41.350641 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d32212c817c94fad8379462b027e7d3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32212c817c94fad8379462b027e7d3d already in progress: creating tablet
I20260811 18:57:41.350899 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e18242f762d8407db0e2f8716890feb4 (table default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]) successfully deleted
W20260811 18:57:41.351626 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3195082643264e0d90f70e1007ab9b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3195082643264e0d90f70e1007ab9b24 already in progress: creating tablet
W20260811 18:57:41.352082 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 429eb5824f3f4b2b8da6e17ba7c69b92 already in progress: creating tablet
W20260811 18:57:41.352625 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f89eebbc40234794a081734f834294ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89eebbc40234794a081734f834294ab already in progress: creating tablet
W20260811 18:57:41.353009 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a5bbd3274980470b9fe71457b49bdd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5bbd3274980470b9fe71457b49bdd62 already in progress: creating tablet
W20260811 18:57:41.353428 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b7b38d3b9570467a9128bd7fe73acf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b38d3b9570467a9128bd7fe73acf2f already in progress: creating tablet
I20260811 18:57:41.353557 3361 master-stress-test.cc:580] Killed test thread 23/25
I20260811 18:57:41.353699 3361 master-stress-test.cc:578] Killing test thread 24/25
I20260811 18:57:41.353888 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 42405aec21e3405486b6c658438b5fb2 (table default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]) successfully deleted
I20260811 18:57:41.343853 5757 ts_tablet_manager.cc:1918] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:41.354452 5757 log.cc:1196] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:41.354527 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet eb8f5b87062349c4b19ec99fa0175f24 (table default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]) successfully deleted
I20260811 18:57:41.354931 5757 ts_tablet_manager.cc:1939] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.355628 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f75dc515d7f3473b8e43f20255a70e3e (table default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]) successfully deleted
I20260811 18:57:41.357508 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d37e81d376e14907b13746cca831a96b (table default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]) successfully deleted
I20260811 18:57:41.359300 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet bfea0cd619024204b9bff1e2dd62b7b0 (table default.Table_911b59bd767843d39233f5c65932fb5d [id=17fdd095e10d41fe8033aad4d1034af5]) successfully deleted
I20260811 18:57:41.359905 5757 tablet_replica.cc:330] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:41.360028 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 69059a0667234a408cc1c09585df63bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69059a0667234a408cc1c09585df63bc already in progress: creating tablet
W20260811 18:57:41.360679 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 82b04e37fc264fc08e0ed9517356a17d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82b04e37fc264fc08e0ed9517356a17d already in progress: creating tablet
W20260811 18:57:41.361255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d11b889e9d044a049d0586382c263951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d11b889e9d044a049d0586382c263951 already in progress: creating tablet
W20260811 18:57:41.361763 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e9f0e443ab84424dac307b745b496986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9f0e443ab84424dac307b745b496986 already in progress: creating tablet
W20260811 18:57:41.362231 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6710084f21734dea9c7ce1f43d1bf587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6710084f21734dea9c7ce1f43d1bf587 already in progress: creating tablet
I20260811 18:57:41.362717 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]) successfully deleted
I20260811 18:57:41.362977 5757 raft_consensus.cc:2229] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:41.363250 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 31bb11617f1548f88f534289706edb06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31bb11617f1548f88f534289706edb06 already in progress: creating tablet
I20260811 18:57:41.363519 5757 raft_consensus.cc:2258] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:41.363708 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 76506ad179a9408c9e4cf8e4275ebddd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76506ad179a9408c9e4cf8e4275ebddd already in progress: creating tablet
I20260811 18:57:41.364215 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8304ac7709ca4392baec4cfa78ef5b41 (table default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]) successfully deleted
I20260811 18:57:41.364270 5757 ts_tablet_manager.cc:1905] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:41.364722 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eb43127dda814a718b338906fc2a3c47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb43127dda814a718b338906fc2a3c47 already in progress: creating tablet
I20260811 18:57:41.364912 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b4dc2df4c6464d71bb13fe25e0046758 (table default.Table_d84e9ae317424a768d54ab23dc68d514 [id=73a97b63e5db407a94ff739f295cc859]) successfully deleted
W20260811 18:57:41.365109 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ee5b7d1e32514ad48177371582e85241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5b7d1e32514ad48177371582e85241 already in progress: creating tablet
W20260811 18:57:41.365285 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4451cc4d7c5940cdb41c021ae486a5a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4451cc4d7c5940cdb41c021ae486a5a9 already in progress: creating tablet
I20260811 18:57:41.365433 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ffa8a4a1c0cb4834be501bdd7d468472 (table default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]) successfully deleted
W20260811 18:57:41.365607 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c590c98e39f84ea09b26bbe8640ca93d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c590c98e39f84ea09b26bbe8640ca93d already in progress: creating tablet
W20260811 18:57:41.365782 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cb54beb34a654fbdb9c7632901bb0b9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb54beb34a654fbdb9c7632901bb0b9e already in progress: creating tablet
W20260811 18:57:41.367524 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 09366eccf38a4876840cc5ddddbbb103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09366eccf38a4876840cc5ddddbbb103 already in progress: creating tablet
W20260811 18:57:41.367985 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 78868ff1f8024163ae1d782151100cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78868ff1f8024163ae1d782151100cd4 already in progress: creating tablet
W20260811 18:57:41.368688 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1ccd56a8b7cc454ea3f2f08b59374521 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ccd56a8b7cc454ea3f2f08b59374521 already in progress: creating tablet
W20260811 18:57:41.369079 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 already in progress: creating tablet
W20260811 18:57:41.369798 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fa95131ce50943e3b0c6b5bdd5a30939 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa95131ce50943e3b0c6b5bdd5a30939 already in progress: creating tablet
I20260811 18:57:41.372710 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]) successfully deleted
W20260811 18:57:41.373425 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f9cef27540ff46aebc9b6f4760ae0db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9cef27540ff46aebc9b6f4760ae0db4 already in progress: creating tablet
W20260811 18:57:41.374121 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bc38e6760754449fbd6209d765cce70c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc38e6760754449fbd6209d765cce70c already in progress: creating tablet
W20260811 18:57:41.374475 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ab03996c5c624e5eb17267b124e27f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab03996c5c624e5eb17267b124e27f45 already in progress: creating tablet
I20260811 18:57:41.378588 3713 tablet_service.cc:1467] Processing CreateTablet for tablet c82456373d974f35846ade45e2398bbd (DEFAULT_TABLE table=default.Table_4b90b4ead71940e58e93af4a9804a7d8 [id=ac620652bd4b404da86f7dda6ed307d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.379074 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82456373d974f35846ade45e2398bbd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.383009 3678 consensus_peers.cc:487] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:41.392637 5407 raft_consensus.cc:482] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.392809 5407 raft_consensus.cc:504] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.393420 5407 leader_election.cc:288] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.394935 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
W20260811 18:57:41.395627 3678 leader_election.cc:334] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.333110 4163 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.402712 4163 ts_tablet_manager.cc:1397] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.154s user 0.004s sys 0.000s
I20260811 18:57:41.402885 5757 ts_tablet_manager.cc:1918] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.403004 5757 log.cc:1196] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/38e24f8d4522466d81605ea37fa7064a
I20260811 18:57:41.403311 5757 ts_tablet_manager.cc:1939] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.403658 4163 raft_consensus.cc:348] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.403956 4163 raft_consensus.cc:374] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.404022 4163 raft_consensus.cc:729] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.404301 4163 consensus_queue.cc:260] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.405100 4163 ts_tablet_manager.cc:1428] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:41.405491 4163 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:41.319404 3824 leader_election.cc:341] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 19243ab3d6744aaa87a133f1462cd2fa
W20260811 18:57:41.407188 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 87794.3us
W20260811 18:57:41.407471 3824 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:41.409142 3824 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.409271 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:41.421993 3713 tablet_service.cc:1467] Processing CreateTablet for tablet defce8dba232424ab74ebae42d6a078a (DEFAULT_TABLE table=default.Table_7a85117c40464174beed058911301729 [id=728fa9eb1e2947c8994d451b9a80eaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.423180 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.423386 4162 tablet_bootstrap.cc:492] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.423517 4162 ts_tablet_manager.cc:1397] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.096s user 0.011s sys 0.002s
I20260811 18:57:41.423591 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 9d09471428a5403da8b3bbf0f1597ed3 (DEFAULT_TABLE table=default.Table_e894321d446d4d6da9e866788ff2b1e7 [id=fd4705a377a94d3eb676f4dd0f37e096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.423964 5593 raft_consensus.cc:2781] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.424111 5593 raft_consensus.cc:686] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.424294 4162 raft_consensus.cc:348] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.424357 5593 consensus_queue.cc:237] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.424746 4162 raft_consensus.cc:374] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.424861 4162 raft_consensus.cc:729] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.425129 4162 consensus_queue.cc:260] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.425740 4162 ts_tablet_manager.cc:1428] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:41.417482 4163 tablet_bootstrap.cc:654] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.394966 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b2ca385374abeb709fccbdf6fa0ba" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.427229 3676 leader_election.cc:341] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.427353 3676 leader_election.cc:302] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.427747 5407 raft_consensus.cc:2726] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.420962 5764 tablet.cc:1717] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Skipping requested alter to schema version 3, tablet already version 3
I20260811 18:57:41.428687 5764 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37348 (request call id 1241) took 1144 ms. Trace:
I20260811 18:57:41.428810 5764 rpcz_store.cc:276] 0811 18:57:40.283830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.286527 (+ 2697us) service_pool.cc:227] Handling call
0811 18:57:41.428669 (+1142142us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:40.286967 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:40.287071 (+ 104us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:40.803344 (+516273us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:40.803354 (+ 10us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:40.803405 (+ 51us) alter_schema_op.cc:126] START. Timestamp: 1786474660803359
0811 18:57:40.803408 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:40.803666 (+ 258us) log.cc:842] Serialized 173 byte log entry
0811 18:57:41.408587 (+604921us) op_driver.cc:444] REPLICATION: finished
0811 18:57:41.420918 (+ 12331us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:41.428355 (+ 7437us) log.cc:842] Serialized 94 byte log entry
0811 18:57:41.428614 (+ 259us) alter_schema_op.cc:71] Released schema lock
0811 18:57:41.428630 (+ 16us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12235,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":18090,"prepare.run_wall_time_us":517205,"raft.queue_time_us":107,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":5,"replication_time_us":605148,"thread_start_us":12637,"threads_started":1}]]}
I20260811 18:57:41.430613 4162 tablet_bootstrap.cc:492] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:41.431324 3824 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:41.431938 5757 tablet_replica.cc:330] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.432231 5757 raft_consensus.cc:2229] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.432801 5757 raft_consensus.cc:2258] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.434134 5757 ts_tablet_manager.cc:1905] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.437953 5764 tablet.cc:1722] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.446206 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.446537 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defce8dba232424ab74ebae42d6a078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.446800 4163 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.446842 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb6ac49f1d4403dad38a3125264088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447261 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 58bc967770de4ed486b638fb6b174db8 with delete_type TABLET_DATA_DELETED (replaced by tablet f336fdd55965432f83b2d1c436fdf1fa) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447288 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d09471428a5403da8b3bbf0f1597ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.448920 4162 tablet_bootstrap.cc:654] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.449748 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.450361 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet eea325ca357146fcaa408d87803da349 with delete_type TABLET_DATA_DELETED (replaced by tablet a4e0b84bdb134a459d2d9afd046326c2) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.450780 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c44eee516d634d9a986107b55410dfc3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5faabb319fc84f24ab548a707be284f1) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.451131 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447108 4163 ts_tablet_manager.cc:1397] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.006s
I20260811 18:57:41.453941 4163 raft_consensus.cc:348] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.460562 5757 ts_tablet_manager.cc:1918] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.463590 4163 raft_consensus.cc:374] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.463793 4163 raft_consensus.cc:729] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.464085 4163 consensus_queue.cc:260] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.465029 4163 ts_tablet_manager.cc:1428] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260811 18:57:41.465390 4163 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.462316 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 with delete_type TABLET_DATA_DELETED (replaced by tablet fe48bbab3d7e4f7591883296a9996db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.466720 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 8794dcb46ccc442e9271fc33453e4800 (DEFAULT_TABLE table=default.Table_f614f2bd34c949cd94f9493a086dd8c7 [id=9b6853170f7b4ebb9c266f96c7d99e10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.466835 5757 log.cc:1196] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9ec4bba841dd446aa3db751358ce7f32
I20260811 18:57:41.467163 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8794dcb46ccc442e9271fc33453e4800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.467248 5757 ts_tablet_manager.cc:1939] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.467324 4163 tablet_bootstrap.cc:654] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.464483 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d93073a69c143ff9460248848a108d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.318331 4169 log.cc:1196] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:41.470433 4169 ts_tablet_manager.cc:1939] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.471045 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1064) took 5119 ms. Trace:
I20260811 18:57:41.471242 4169 rpcz_store.cc:276] 0811 18:57:36.351368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.353947 (+ 2579us) service_pool.cc:227] Handling call
0811 18:57:41.285949 (+4932002us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:41.285960 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.315087 (+ 29127us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.318114 (+ 3027us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.471018 (+152904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4930927}
I20260811 18:57:41.471879 4169 tablet_replica.cc:330] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.472165 4169 raft_consensus.cc:2229] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.472385 4169 raft_consensus.cc:2258] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.473008 4169 ts_tablet_manager.cc:1905] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.474051 4162 tablet_bootstrap.cc:492] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.474267 4162 ts_tablet_manager.cc:1397] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.005s
W20260811 18:57:41.474706 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 374c94a12e374d0e82180421c80d92d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374c94a12e374d0e82180421c80d92d0 already in progress: creating tablet
W20260811 18:57:41.475030 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 60091bd891a94cc4aac6ef47673a4ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60091bd891a94cc4aac6ef47673a4ee5 already in progress: creating tablet
I20260811 18:57:41.475085 4162 raft_consensus.cc:348] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.475293 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fe080f2071964da6b54f1ee5af98d918 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe080f2071964da6b54f1ee5af98d918 already in progress: creating tablet
I20260811 18:57:41.475260 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.475318 4162 raft_consensus.cc:374] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.475420 4162 raft_consensus.cc:729] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.475430 3741 raft_consensus.cc:3037] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.478925 3741 raft_consensus.cc:2454] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:41.478966 3824 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.479336 3824 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.479665 3824 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.479736 4163 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.479938 4163 ts_tablet_manager.cc:1397] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:41.480773 4163 raft_consensus.cc:348] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481122 4163 raft_consensus.cc:374] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.481178 4163 raft_consensus.cc:729] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.475675 4162 consensus_queue.cc:260] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481369 4163 consensus_queue.cc:260] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481925 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 56542f3afe304feb888f101961d855c7 (DEFAULT_TABLE table=default.Table_9919772109274064aee942f6a7f1220f [id=75a15771a07045039c110309e3dc0beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.482301 4162 ts_tablet_manager.cc:1428] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:41.482473 4163 ts_tablet_manager.cc:1428] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 18:57:41.482906 4163 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.482622 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56542f3afe304feb888f101961d855c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.483597 4162 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.485419 4162 tablet_bootstrap.cc:654] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.486012 4163 tablet_bootstrap.cc:654] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.485966 5757 tablet_replica.cc:330] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.492501 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d8c6b62d2a684869b0ad31a1b25c7ebf (DEFAULT_TABLE table=default.Table_bf1e0bf055c44f45835974098ec141f8 [id=8f5e99a55df34267a0b32710772f1978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.492642 5757 raft_consensus.cc:2229] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.492810 5757 raft_consensus.cc:2258] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.492962 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c6b62d2a684869b0ad31a1b25c7ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.493182 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a54921b815f2408c96bdf18e243041d5 (DEFAULT_TABLE table=default.Table_23655b2975534b81b076fd031ea5022a [id=e040f0cd609644928bb9690a053fa426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.493433 5757 ts_tablet_manager.cc:1905] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.493544 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54921b815f2408c96bdf18e243041d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.503337 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 71a7eda2248c4f708bdc772ea3598dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.508503 4162 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.508844 4162 ts_tablet_manager.cc:1397] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20260811 18:57:41.509920 5407 raft_consensus.cc:482] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.510450 5407 raft_consensus.cc:504] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.511068 5407 leader_election.cc:288] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.512342 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 79539e937f81437ab3b12b8465f12d97) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.512969 3678 leader_election.cc:334] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.513145 4162 raft_consensus.cc:348] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.513484 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4059e5907b1e488ab9e1d04e563f96cd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.513301 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.516253 4162 raft_consensus.cc:374] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:41.514181 3676 leader_election.cc:341] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.516459 3676 leader_election.cc:302] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.516867 5407 raft_consensus.cc:2726] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.513548 5772 tablet.cc:1717] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Skipping requested alter to schema version 3, tablet already version 3
I20260811 18:57:41.517684 5772 tablet.cc:1722] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.518299 4162 raft_consensus.cc:729] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.521845 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f891108d44c44296934924941981b568 (DEFAULT_TABLE table=default.Table_4b752724648a499980ad71fd596a731f [id=78f3a4a7d6ad41278417b8eef697d0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.522296 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f891108d44c44296934924941981b568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.522225 4162 consensus_queue.cc:260] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.523173 4163 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.523398 4163 ts_tablet_manager.cc:1397] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.004s
I20260811 18:57:41.524255 4163 raft_consensus.cc:348] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.524959 4162 ts_tablet_manager.cc:1428] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260811 18:57:41.527632 4162 tablet_bootstrap.cc:492] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.533804 4162 tablet_bootstrap.cc:654] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.527287 5757 ts_tablet_manager.cc:1918] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.537039 5757 log.cc:1196] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b20872d80c9545e5822bfc82d496ce97
I20260811 18:57:41.537508 5757 ts_tablet_manager.cc:1939] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.539764 5757 tablet_replica.cc:330] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.540766 5757 raft_consensus.cc:2229] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.541009 5757 raft_consensus.cc:2258] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.541628 5757 ts_tablet_manager.cc:1905] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.550498 4163 raft_consensus.cc:374] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.550733 4163 raft_consensus.cc:729] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.551133 4163 consensus_queue.cc:260] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.551921 4163 ts_tablet_manager.cc:1428] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260811 18:57:41.557602 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 0bac7ac254f4471397580dc510648dfc (DEFAULT_TABLE table=default.Table_69847be1563a4aad8d093d935864f531 [id=6a7e6156c0c247f68bcdb76ed1eb6365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.558785 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 756062a54d144339b91f8c68d9edceab (DEFAULT_TABLE table=default.Table_e142d545d8b249f3835bce3bbef9f94e [id=51b0aa68c2e749768c886732760d4f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.560889 3706 tablet_service.cc:1467] Processing CreateTablet for tablet ce6a80313c854c17913db64aeab9acf3 (DEFAULT_TABLE table=default.Table_c4b27a9eb0994efe9f4176f74c16a62e [id=6e29d4572cd64f8cb70940f4119296e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.528234 5593 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.528033 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48698.8us
I20260811 18:57:41.559700 5767 raft_consensus.cc:482] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.564857 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f11748cf984942ba99bda4ad6605ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.566761 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b50efab3c782447cad8e45b1fa51aa1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.567451 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 997be6173a194bbe91c6c0b9046d1a0e (DEFAULT_TABLE table=default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae [id=7586915510704916afd712ab81f9f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.567642 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a07f682b68114c2b9cb0df27a94406ec (DEFAULT_TABLE table=default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.568830 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 86f215c21d184dd6ba828fa31286cce1 (DEFAULT_TABLE table=default.Table_d8f35609e86b47aa9a8ab711834da08f [id=72dfc9b4df824dcd92241af7ab88c68e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.569764 3708 tablet_service.cc:1467] Processing CreateTablet for tablet bba6561b8c4b4c728db18446a9a91f30 (DEFAULT_TABLE table=default.Table_5affb8acc7e5466ba5ed988728ba50cb [id=0a26d099a1324a9aa8005962a8d9bd79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.570088 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 01fe8e896b2246fabd2b6fae396bf085 (DEFAULT_TABLE table=default.Table_f83d8784174f40e29b96eba2de396065 [id=fdb879cbb1cb4f498f94d14048b87e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.570789 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.571409 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.563108 5767 raft_consensus.cc:504] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.571957 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756062a54d144339b91f8c68d9edceab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572158 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba6561b8c4b4c728db18446a9a91f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572296 5767 leader_election.cc:288] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.562652 4163 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.572569 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01fe8e896b2246fabd2b6fae396bf085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572752 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 997be6173a194bbe91c6c0b9046d1a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.573048 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6a80313c854c17913db64aeab9acf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.573335 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet bf1c037a37504b3fad9612f5a734dd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.573665 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07f682b68114c2b9cb0df27a94406ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.563242 3824 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.574751 3824 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:41.574914 3824 raft_consensus.cc:2670] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:41.575492 3824 leader_election.cc:334] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.575702 3824 leader_election.cc:302] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.575917 5767 raft_consensus.cc:2726] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.576198 3824 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.565064 5593 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
I20260811 18:57:41.579388 5593 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.579718 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet fc3f120758c6405dafd2ea0bf54c81d8 (table default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]) successfully deleted
I20260811 18:57:41.580191 5593 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.580600 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bac7ac254f4471397580dc510648dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.581697 3824 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.581996 5767 raft_consensus.cc:2781] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.582140 5767 raft_consensus.cc:686] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.582521 5767 consensus_queue.cc:237] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.584715 5630 raft_consensus.cc:482] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.584887 5630 raft_consensus.cc:504] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.585530 5630 leader_election.cc:288] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.585644 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a7946503bec9446d8deb02448eb231d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.586561 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f215c21d184dd6ba828fa31286cce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.586903 5407 raft_consensus.cc:482] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.587036 5407 raft_consensus.cc:504] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.587675 5407 leader_election.cc:288] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.589289 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 81e467ea40364946b7c0bc4b5592996b (DEFAULT_TABLE table=default.Table_2486985deae44613a5f8d342c8236784 [id=74d8ef880ca24497ace800c16e041dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.589397 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f287040b879642448367abe2b1c73cd9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.589980 5407 raft_consensus.cc:482] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.590207 5407 raft_consensus.cc:504] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.590909 5407 leader_election.cc:288] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.592418 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e467ea40364946b7c0bc4b5592996b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.592371 3678 leader_election.cc:334] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.593921 3676 leader_election.cc:341] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.594069 3676 leader_election.cc:302] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.594930 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7da0868b114052ba421801b76fb33c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.595603 3678 leader_election.cc:334] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.596216 3676 leader_election.cc:341] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.596346 3676 leader_election.cc:302] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.596746 5733 raft_consensus.cc:2726] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.596832 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.596907 5733 raft_consensus.cc:2726] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.599030 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.599365 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.600651 4162 tablet_bootstrap.cc:492] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.600859 4162 ts_tablet_manager.cc:1397] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.073s user 0.009s sys 0.004s
I20260811 18:57:41.601840 4162 raft_consensus.cc:348] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.602147 4162 raft_consensus.cc:374] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.602272 4162 raft_consensus.cc:729] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.602602 4162 consensus_queue.cc:260] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.604127 4162 ts_tablet_manager.cc:1428] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:41.608428 4162 tablet_bootstrap.cc:492] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.596354 5757 ts_tablet_manager.cc:1918] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.608973 5757 log.cc:1196] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e14f8717bcd94d6d88295fd8be130ca5
I20260811 18:57:41.609474 5757 ts_tablet_manager.cc:1939] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.610661 5757 tablet_replica.cc:330] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.610857 4162 tablet_bootstrap.cc:654] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.611763 5757 raft_consensus.cc:2229] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.613667 5757 raft_consensus.cc:2258] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.614406 5757 ts_tablet_manager.cc:1905] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.619944 4163 tablet_bootstrap.cc:654] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.625466 5757 ts_tablet_manager.cc:1918] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.626644 4162 tablet_bootstrap.cc:492] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.626956 4162 ts_tablet_manager.cc:1397] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.005s
I20260811 18:57:41.627923 4162 raft_consensus.cc:348] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.628242 4162 raft_consensus.cc:374] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.628377 4162 raft_consensus.cc:729] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.628706 4162 consensus_queue.cc:260] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.629570 4162 ts_tablet_manager.cc:1428] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:41.629791 5757 log.cc:1196] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ebf7720e31d84ae2ab592da1e7a3132a
I20260811 18:57:41.634119 5757 ts_tablet_manager.cc:1939] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.630162 4162 tablet_bootstrap.cc:492] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.636055 4169 ts_tablet_manager.cc:1918] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.636422 4169 log.cc:1196] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:41.636829 4169 ts_tablet_manager.cc:1939] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.637773 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1066) took 5277 ms. Trace:
I20260811 18:57:41.637902 4169 rpcz_store.cc:276] 0811 18:57:36.360446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.360888 (+ 442us) service_pool.cc:227] Handling call
0811 18:57:41.471827 (+5110939us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:41.471839 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.563939 (+ 92100us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.635913 (+ 71974us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.637746 (+ 1833us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5110508}
I20260811 18:57:41.637863 4162 tablet_bootstrap.cc:654] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.638942 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:41.639079 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:41.643203 5757 tablet_replica.cc:330] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.643502 5757 raft_consensus.cc:2229] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.644155 5757 raft_consensus.cc:2258] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.644888 5757 ts_tablet_manager.cc:1905] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.650216 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.652412 5757 ts_tablet_manager.cc:1918] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:41.653496 3824 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.657958 5757 log.cc:1196] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:41.658739 5757 ts_tablet_manager.cc:1939] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.662767 5757 tablet_replica.cc:330] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.663261 5757 raft_consensus.cc:2229] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.663753 5757 raft_consensus.cc:2258] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.664572 5757 ts_tablet_manager.cc:1905] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.668593 5764 tablet.cc:1722] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:41.668864 3824 leader_election.cc:341] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0bfb54175c834517b7af291c942333cd
W20260811 18:57:41.672037 3824 leader_election.cc:341] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 3ad5f39f59b8447cb3f15b9646e4b852
W20260811 18:57:41.672259 3824 leader_election.cc:341] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:41.672793 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1067) took 5312 ms. Trace:
I20260811 18:57:41.672971 4169 rpcz_store.cc:276] 0811 18:57:36.360557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.360672 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:41.638275 (+5277603us) ts_tablet_manager.cc:1145] Deleting tablet 03c50dcffac54c7bac210eeaed340ca6
0811 18:57:41.638287 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.672763 (+ 34476us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5276748}
I20260811 18:57:41.673472 4169 tablet_replica.cc:330] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.673786 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 48d32bb4eca047e4ba786c04b8bebf8b (table default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]) successfully deleted
I20260811 18:57:41.677443 4162 tablet_bootstrap.cc:492] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.677727 4162 ts_tablet_manager.cc:1397] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.013s
I20260811 18:57:41.681708 4162 raft_consensus.cc:348] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.682080 4162 raft_consensus.cc:374] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.682212 4162 raft_consensus.cc:729] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.682577 4162 consensus_queue.cc:260] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.683171 5733 consensus_queue.cc:1035] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.683633 4162 ts_tablet_manager.cc:1428] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:41.684695 5757 ts_tablet_manager.cc:1918] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.684859 5757 log.cc:1196] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:41.685398 5757 ts_tablet_manager.cc:1939] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.686548 4162 tablet_bootstrap.cc:492] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.691677 4162 tablet_bootstrap.cc:654] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.695935 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b40da8cb7324da9804e6fdafd8e8266: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.697108 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet e902efbb774244d18e3ba165bfb7c695 on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:41.698949 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 71df3738a5cf4b12820dda131ced62e1 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.706097 5642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:41.707721 5757 tablet_replica.cc:330] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.708169 5757 raft_consensus.cc:2229] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.708483 5757 raft_consensus.cc:2258] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.709322 5757 ts_tablet_manager.cc:1905] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.716753 4162 tablet_bootstrap.cc:492] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.717022 4162 ts_tablet_manager.cc:1397] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.011s
I20260811 18:57:41.718211 5757 ts_tablet_manager.cc:1918] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.718171 4162 raft_consensus.cc:348] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.718668 4162 raft_consensus.cc:374] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.718820 4162 raft_consensus.cc:729] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.719205 4162 consensus_queue.cc:260] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.718501 5757 log.cc:1196] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a0116997a1c64e658045c319f1c45861
I20260811 18:57:41.720047 5757 ts_tablet_manager.cc:1939] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.721091 4162 ts_tablet_manager.cc:1428] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:41.676192 4163 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.723898 4163 ts_tablet_manager.cc:1397] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.161s user 0.005s sys 0.001s
W20260811 18:57:41.681638 3824 leader_election.cc:334] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.711344 5593 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.724107 3824 leader_election.cc:302] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.724172 5593 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.724764 5593 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.724917 5593 raft_consensus.cc:2726] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.724944 4163 raft_consensus.cc:348] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.725169 4163 raft_consensus.cc:374] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.725267 4163 raft_consensus.cc:729] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:41.725320 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 03c50dcffac54c7bac210eeaed340ca6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03c50dcffac54c7bac210eeaed340ca6 already in progress: creating tablet
I20260811 18:57:41.725471 4163 consensus_queue.cc:260] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.726074 4163 ts_tablet_manager.cc:1428] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:41.726708 4169 raft_consensus.cc:2229] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.726949 4169 raft_consensus.cc:2258] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.727558 4163 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.727655 4169 ts_tablet_manager.cc:1905] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:41.729018 3824 leader_election.cc:334] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.729147 3824 leader_election.cc:302] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:41.729455 3824 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.729892 5630 raft_consensus.cc:2726] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.730867 5630 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:41.731369 3678 consensus_peers.cc:487] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:41.729553 3824 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.731026 5630 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.732227 5593 raft_consensus.cc:2726] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:57:41.732331 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:41.732542 5630 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.734661 3824 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.735526 3824 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.737623 5772 tablet.cc:1722] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:41.747799 3824 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 059b325df1db4fde9d4b26b559121316
I20260811 18:57:41.751370 3824 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.751833 5593 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.752022 3824 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:41.752140 3824 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.752360 5593 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.749441 4163 tablet_bootstrap.cc:654] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.753261 4169 ts_tablet_manager.cc:1918] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.753474 4169 log.cc:1196] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:41.753713 5764 tablet.cc:1722] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.754590 5757 tablet_replica.cc:330] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.754855 4162 tablet_bootstrap.cc:492] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.754905 5757 raft_consensus.cc:2229] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.755072 5757 raft_consensus.cc:2258] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.756453 5772 tablet.cc:1722] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.757334 4162 tablet_bootstrap.cc:654] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.776816 5757 ts_tablet_manager.cc:1905] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.777021 5630 raft_consensus.cc:482] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.777308 5630 raft_consensus.cc:504] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.778072 5630 leader_election.cc:288] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.778043 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet ae96e10a2ed4407c9d42a359d82d389e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.780735 3824 leader_election.cc:341] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 42405aec21e3405486b6c658438b5fb2
W20260811 18:57:41.781010 3824 leader_election.cc:334] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.781152 3824 leader_election.cc:302] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.781476 5630 raft_consensus.cc:2726] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.782042 4169 ts_tablet_manager.cc:1939] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.783241 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1074) took 4853 ms. Trace:
I20260811 18:57:41.785106 5757 ts_tablet_manager.cc:1918] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.786736 5630 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.790458 5630 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.791162 5630 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.785815 4163 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.791441 4163 ts_tablet_manager.cc:1397] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.004s
I20260811 18:57:41.792364 4163 raft_consensus.cc:348] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.792595 4163 raft_consensus.cc:374] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.792680 4163 raft_consensus.cc:729] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.792919 4163 consensus_queue.cc:260] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.798014 4163 ts_tablet_manager.cc:1428] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:41.803516 4163 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.806011 4169 rpcz_store.cc:276] 0811 18:57:36.929690 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.929791 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:41.673408 (+4743617us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:41.673426 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.749781 (+ 76355us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.753148 (+ 3367us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.783217 (+ 30069us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4743406}
I20260811 18:57:41.807451 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1075) took 4877 ms. Trace:
W20260811 18:57:41.807802 3824 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.813000 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d00d943b778741b4aeec076ba46a9c40 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:41.785601 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb6ac49f1d4403dad38a3125264088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.786628 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 58bc967770de4ed486b638fb6b174db8 with delete_type TABLET_DATA_DELETED (replaced by tablet f336fdd55965432f83b2d1c436fdf1fa) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.790522 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet eea325ca357146fcaa408d87803da349 with delete_type TABLET_DATA_DELETED (replaced by tablet a4e0b84bdb134a459d2d9afd046326c2) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.792104 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet c44eee516d634d9a986107b55410dfc3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5faabb319fc84f24ab548a707be284f1) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.793721 4162 tablet_bootstrap.cc:492] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.815361 4162 ts_tablet_manager.cc:1397] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.011s
I20260811 18:57:41.817764 4162 raft_consensus.cc:348] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.795825 5742 raft_consensus.cc:482] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.818511 5742 raft_consensus.cc:504] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.820163 5742 leader_election.cc:288] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.800673 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.820714 3741 raft_consensus.cc:2454] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:41.813970 5734 raft_consensus.cc:482] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.810992 4169 rpcz_store.cc:276] 0811 18:57:36.930284 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.930385 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:41.807302 (+4876917us) ts_tablet_manager.cc:1145] Deleting tablet 73c519528f75423195b6bdf6683fd91b
0811 18:57:41.807316 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.807427 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4876752}
I20260811 18:57:41.824239 5734 raft_consensus.cc:504] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.824937 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1076) took 4894 ms. Trace:
I20260811 18:57:41.822724 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b6e8839e24ec9a69854349bc3ecf8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.825999 5734 leader_election.cc:288] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.819497 4163 tablet_bootstrap.cc:654] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.828738 3676 leader_election.cc:341] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.829514 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d81e93739f44d9e99b6ebc647e1d0e4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.833011 3676 leader_election.cc:341] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.825589 3824 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.833701 5630 raft_consensus.cc:2781] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.833935 5630 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.834107 5630 raft_consensus.cc:3037] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.818282 4162 raft_consensus.cc:374] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.835040 4162 raft_consensus.cc:729] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:41.835537 3678 leader_election.cc:334] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.835774 3678 leader_election.cc:302] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.835626 4162 consensus_queue.cc:260] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.836347 3678 leader_election.cc:334] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.836956 4162 ts_tablet_manager.cc:1428] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260811 18:57:41.837004 5742 raft_consensus.cc:2726] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.825129 4169 rpcz_store.cc:276] 0811 18:57:36.930847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.930910 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:41.824577 (+4893667us) ts_tablet_manager.cc:1145] Deleting tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
0811 18:57:41.824611 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.824847 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4893448}
I20260811 18:57:41.837867 4169 tablet_replica.cc:330] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.838551 4169 raft_consensus.cc:2229] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.838914 4169 raft_consensus.cc:2258] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.840070 4169 ts_tablet_manager.cc:1905] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.841265 5630 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.842909 5630 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.844063 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 73c519528f75423195b6bdf6683fd91b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c519528f75423195b6bdf6683fd91b already in progress: creating tablet
I20260811 18:57:41.836550 3678 leader_election.cc:302] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.845455 5734 raft_consensus.cc:2726] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.847059 4162 tablet_bootstrap.cc:492] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:41.847831 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b already in progress: creating tablet
I20260811 18:57:41.850260 5757 log.cc:1196] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8d1e244addf5462499fcadaa3332cba1
I20260811 18:57:41.857270 5792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:41.857770 5757 ts_tablet_manager.cc:1939] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.861752 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.862213 3741 raft_consensus.cc:3037] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.867229 5757 tablet_replica.cc:330] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.867848 5757 raft_consensus.cc:2229] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:41.869150 3824 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.868666 3741 raft_consensus.cc:2454] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.873762 5757 raft_consensus.cc:2258] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.875325 5757 ts_tablet_manager.cc:1905] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.876322 3824 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.877255 5630 raft_consensus.cc:2781] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.875913 4162 tablet_bootstrap.cc:654] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.878873 5630 raft_consensus.cc:686] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.879503 5630 consensus_queue.cc:237] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.881269 5734 raft_consensus.cc:482] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.881587 5734 raft_consensus.cc:504] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.886453 5734 leader_election.cc:288] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.859776 5792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:41.889168 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdbcfcd69a24d3f8a0f8383ba4c88e8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.889850 5792 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:41.892112 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1649 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:41.892885 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1649 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
W20260811 18:57:41.893803 3676 leader_election.cc:341] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.895349 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:41.899791 5757 ts_tablet_manager.cc:1918] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.900266 5757 log.cc:1196] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4059e5907b1e488ab9e1d04e563f96cd
I20260811 18:57:41.901154 5757 ts_tablet_manager.cc:1939] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.903774 5757 tablet_replica.cc:330] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.904450 5757 raft_consensus.cc:2229] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.904956 5757 raft_consensus.cc:2258] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.906299 5757 ts_tablet_manager.cc:1905] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.906534 4163 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.910946 4163 ts_tablet_manager.cc:1397] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.108s user 0.012s sys 0.001s
I20260811 18:57:41.912084 4163 raft_consensus.cc:348] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.912403 4163 raft_consensus.cc:374] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.912534 4163 raft_consensus.cc:729] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.912855 4163 consensus_queue.cc:260] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.914481 3678 leader_election.cc:334] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.914649 3678 leader_election.cc:302] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.915095 5734 raft_consensus.cc:2726] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.916675 5757 ts_tablet_manager.cc:1918] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.916870 5757 log.cc:1196] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0540e4e43ea34fbfbc66b6035c60684d
I20260811 18:57:41.917327 5757 ts_tablet_manager.cc:1939] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.918735 5757 tablet_replica.cc:330] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.919049 5757 raft_consensus.cc:2229] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.919276 5757 raft_consensus.cc:2258] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.919891 5757 ts_tablet_manager.cc:1905] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.922226 4169 ts_tablet_manager.cc:1918] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:41.922937 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 38e24f8d4522466d81605ea37fa7064a (table default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]) successfully deleted
W20260811 18:57:41.923532 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f389d2b7ff354ac4bab5d68b9ff201c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f389d2b7ff354ac4bab5d68b9ff201c6 already in progress: creating tablet
I20260811 18:57:41.923965 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9ec4bba841dd446aa3db751358ce7f32 (table default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]) successfully deleted
I20260811 18:57:41.924584 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b20872d80c9545e5822bfc82d496ce97 (table default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]) successfully deleted
I20260811 18:57:41.925983 4169 log.cc:1196] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3ad5f39f59b8447cb3f15b9646e4b852
I20260811 18:57:41.926030 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e14f8717bcd94d6d88295fd8be130ca5 (table default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]) successfully deleted
I20260811 18:57:41.926251 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ebf7720e31d84ae2ab592da1e7a3132a (table default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]) successfully deleted
I20260811 18:57:41.926577 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cd1fc828f73642c88171d6ac827de90e (table default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]) successfully deleted
I20260811 18:57:41.923022 4163 ts_tablet_manager.cc:1428] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
W20260811 18:57:41.926824 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b9fbac57c9c44a028f5b258593aff86d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fbac57c9c44a028f5b258593aff86d already in progress: creating tablet
W20260811 18:57:41.927117 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d65cb8d901c04d8ba2b7a40efc7fa3af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65cb8d901c04d8ba2b7a40efc7fa3af already in progress: creating tablet
I20260811 18:57:41.927325 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4a64df7c295c4d9baded1b63a9980581 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
I20260811 18:57:41.927362 4163 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:41.927538 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fdab1c3c431c4dceb08b00eabcfa1ceb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdab1c3c431c4dceb08b00eabcfa1ceb already in progress: creating tablet
I20260811 18:57:41.927563 5757 ts_tablet_manager.cc:1918] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.927701 5757 log.cc:1196] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b203df4124364737839bf4c1ae072988
W20260811 18:57:41.927735 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0e556811ad1643f1baa3bbf4edebd157 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e556811ad1643f1baa3bbf4edebd157 already in progress: creating tablet
W20260811 18:57:41.927912 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 already in progress: creating tablet
W20260811 18:57:41.928095 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5d93073a69c143ff9460248848a108d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d93073a69c143ff9460248848a108d4 already in progress: creating tablet
W20260811 18:57:41.928321 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 71a7eda2248c4f708bdc772ea3598dec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71a7eda2248c4f708bdc772ea3598dec already in progress: creating tablet
I20260811 18:57:41.928366 5757 ts_tablet_manager.cc:1939] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:41.928504 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 already in progress: creating tablet
W20260811 18:57:41.928748 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb already in progress: creating tablet
W20260811 18:57:41.928969 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 72f11748cf984942ba99bda4ad6605ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72f11748cf984942ba99bda4ad6605ab already in progress: creating tablet
I20260811 18:57:41.929996 4163 tablet_bootstrap.cc:654] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.930253 5792 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:41.931074 4169 ts_tablet_manager.cc:1939] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.932530 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1077) took 5000 ms. Trace:
I20260811 18:57:41.935891 4162 tablet_bootstrap.cc:492] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.936053 4162 ts_tablet_manager.cc:1397] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.089s user 0.004s sys 0.005s
I20260811 18:57:41.937196 4162 raft_consensus.cc:348] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.937472 4162 raft_consensus.cc:374] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.937528 4162 raft_consensus.cc:729] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.937758 4162 consensus_queue.cc:260] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.938290 5767 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.938534 5767 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.939334 5767 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.941035 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.941291 3741 raft_consensus.cc:2454] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:41.941962 3824 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.942267 4162 ts_tablet_manager.cc:1428] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:41.942299 3824 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.942608 5767 raft_consensus.cc:2781] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.942734 5767 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.942862 4162 tablet_bootstrap.cc:492] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.942867 5767 raft_consensus.cc:3037] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.945864 5767 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.946624 5767 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.947527 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.947748 3741 raft_consensus.cc:3037] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.951356 3741 raft_consensus.cc:2454] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.955464 4162 tablet_bootstrap.cc:654] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.957298 3824 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.957984 5593 raft_consensus.cc:2781] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.958180 5593 raft_consensus.cc:686] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.958525 5593 consensus_queue.cc:237] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:41.987200 3824 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.987805 4169 rpcz_store.cc:276] 0811 18:57:36.932302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.932362 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:41.837736 (+4905374us) ts_tablet_manager.cc:1145] Deleting tablet 3ad5f39f59b8447cb3f15b9646e4b852
0811 18:57:41.837779 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.918785 (+ 81006us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.922092 (+ 3307us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.932487 (+ 10395us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4905177}
I20260811 18:57:41.989490 3548 catalog_manager.cc:5495] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:41.990007 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:41.996435 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6ddd2d6468454840818302a8fd9bfb88 (DEFAULT_TABLE table=default.Table_59d30e9c40554addab5dc9b7c1d11432 [id=785355398c254031b55ce65fb12e21c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.997359 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 45e040f8120a4edfba89c1a8d55deb4a (DEFAULT_TABLE table=default.Table_d82e6143f45747ddafae8a1e2651b9a0 [id=d8d665af78f448268e114e574587dcfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.997776 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ddd2d6468454840818302a8fd9bfb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.997923 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 9be08aff2b324689a34748f897c04f50 (DEFAULT_TABLE table=default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0 [id=90c633f7d0b44c30823b6e07b231d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.998319 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be08aff2b324689a34748f897c04f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.998610 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ebead0adb7184690b117d775949f5973 (DEFAULT_TABLE table=default.Table_6dab2aa50f394dabbbae787dca84b57a [id=07a603a975f144dd85497bb60920e9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.002857 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebead0adb7184690b117d775949f5973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.996711 5630 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.004124 5630 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.004845 5630 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.998790 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2afb2162af4ca8a03481c7df0d8766 (DEFAULT_TABLE table=default.Table_8edb4b4387a444abb3b70122ae70018e [id=f90f0baeb30349a39dab268a3a70e0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.006809 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.007109 3741 raft_consensus.cc:2454] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.007829 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2afb2162af4ca8a03481c7df0d8766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.009574 3824 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:42.009948 3824 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.010403 5630 raft_consensus.cc:2781] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.010541 5630 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.010896 5630 raft_consensus.cc:3037] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.010967 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e040f8120a4edfba89c1a8d55deb4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.011516 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1078) took 5078 ms. Trace:
I20260811 18:57:42.012696 4163 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.012868 4163 ts_tablet_manager.cc:1397] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.086s user 0.000s sys 0.004s
I20260811 18:57:42.013777 4163 raft_consensus.cc:348] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.014036 4163 raft_consensus.cc:374] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.014166 4163 raft_consensus.cc:729] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.014482 4163 consensus_queue.cc:260] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.016506 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 3ad5f39f59b8447cb3f15b9646e4b852 (table default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]) successfully deleted
I20260811 18:57:42.011641 4169 rpcz_store.cc:276] 0811 18:57:36.932770 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.932827 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:42.011300 (+5078473us) ts_tablet_manager.cc:1145] Deleting tablet 7cb3e997b4454598bdce102e92296887
0811 18:57:42.011315 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.011432 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5078318}
I20260811 18:57:42.017252 4169 tablet_replica.cc:330] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:42.018010 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7cb3e997b4454598bdce102e92296887 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7cb3e997b4454598bdce102e92296887 already in progress: creating tablet
I20260811 18:57:42.018389 4169 raft_consensus.cc:2229] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.021562 5734 raft_consensus.cc:482] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.021782 5734 raft_consensus.cc:504] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.022630 5734 leader_election.cc:288] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:42.023797 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bc6cce2c19746219b8b0fe54b13dcd6" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:42.024442 3676 leader_election.cc:341] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:42.025674 3678 leader_election.cc:334] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.025825 3678 leader_election.cc:302] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.026249 5734 raft_consensus.cc:2726] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:42.031700 3678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 90 similar messages]
I20260811 18:57:42.041286 4169 raft_consensus.cc:2258] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.042639 4169 ts_tablet_manager.cc:1905] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.044024 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 80d3f09aada44148b45897e4a7e30bd1 (DEFAULT_TABLE table=default.Table_93e4cfd44cab49469aa14294ac0ba348 [id=dec80fce066e407d85eaab6390d0c75d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.044951 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 66e70ff397bd493992d2f08a30221b69 (DEFAULT_TABLE table=default.Table_4821250a7fa84b68a74499b0d7a1228d [id=8cac8dcdf645477b926cf3a72a64dd61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.043995 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ff03310a9aca474d8148818b2126be67 (DEFAULT_TABLE table=default.Table_b691c53749bc40a0a829753ba4c7e6aa [id=3742d08328974ae3979f3b27f4758e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.045428 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e70ff397bd493992d2f08a30221b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.045640 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff03310a9aca474d8148818b2126be67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.045908 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b609177908d04a829b1fe744ef0f53e1 (table default.Table_4a427b04935c4d62b0595df0085e3a98 [id=4845fddf6562470ebb7275b67fad076e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b609177908d04a829b1fe744ef0f53e1 consensus state has no leader
I20260811 18:57:42.046160 5734 raft_consensus.cc:482] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.046322 5734 raft_consensus.cc:504] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.046995 5734 leader_election.cc:288] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:42.048182 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114397bea09444479df113552c22e154" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:42.048621 5768 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.048810 5768 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.049443 5768 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.049881 3676 leader_election.cc:341] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:42.041631 5630 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.050799 5630 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.051064 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80d3f09aada44148b45897e4a7e30bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.053282 4163 ts_tablet_manager.cc:1428] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.032s user 0.003s sys 0.000s
W20260811 18:57:42.057549 3678 leader_election.cc:334] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.057719 3678 leader_election.cc:302] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.058495 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.058544 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 0e928aaaf32d4ba1ac24a4df0e05f2f0 (DEFAULT_TABLE table=default.Table_020300b2b8b949278c35159bd43fe19d [id=5726999b7dcf4236b82fc9a7d097cfa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.058768 3741 raft_consensus.cc:2454] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.059039 5734 raft_consensus.cc:2726] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:42.059295 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e928aaaf32d4ba1ac24a4df0e05f2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.062686 3851 tablet_service.cc:1467] Processing CreateTablet for tablet dac5803fac024ec8aeaafee8dad0ce7f (DEFAULT_TABLE table=default.Table_c17b389153604aeeb9c985eaf661f44a [id=40aa393b93fd4946b648765f3c864b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.044934 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd80c582f9043d79e6bedf2a500c7ec (DEFAULT_TABLE table=default.Table_5e94a3f1b3a1418d8d5ff4ae641da751 [id=2c12b1a39437497f982e0b90b1b86a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.065439 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 659c2937c672482f9ea7b4e524d9847b (DEFAULT_TABLE table=default.Table_d57e6ac8edbc46a69270b9277778e7b2 [id=a9fd0364a9664280a268e512d946f4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.067013 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.067135 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1f9621bbd0f94223a5d36bc01487f615
I20260811 18:57:42.067258 3741 raft_consensus.cc:3037] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.067654 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 1f9621bbd0f94223a5d36bc01487f615 deleted and replaced by tablet 332d3d9ddb224a4697809b19b622749d
I20260811 18:57:42.070266 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (table default.Table_6c8484897dd046ab8499d1069356cdcd [id=221d836949c341a09235288dc5802137]). Expected version 3 got 0
I20260811 18:57:42.070567 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:42.070832 3741 raft_consensus.cc:2454] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.070950 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:42.071117 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 2 got 0
I20260811 18:57:42.071400 4163 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.073709 4163 tablet_bootstrap.cc:654] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.078104 3824 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.078639 5768 raft_consensus.cc:2781] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.078806 5768 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.078936 5768 raft_consensus.cc:3037] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.081851 3361 master-stress-test.cc:580] Killed test thread 24/25
I20260811 18:57:42.081970 3361 master-stress-test.cc:578] Killing test thread 25/25
I20260811 18:57:42.082070 3361 master-stress-test.cc:580] Killed test thread 25/25
I20260811 18:57:42.081974 5768 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.082311 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3659
I20260811 18:57:42.083123 5768 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.086959 5768 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.087255 5768 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.088202 5768 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.083201 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 with delete_type TABLET_DATA_DELETED (replaced by tablet 332d3d9ddb224a4697809b19b622749d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.091307 3824 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.097259 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dac5803fac024ec8aeaafee8dad0ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.097354 5630 raft_consensus.cc:2781] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.097558 5630 raft_consensus.cc:686] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.097978 5630 consensus_queue.cc:237] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.098394 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd80c582f9043d79e6bedf2a500c7ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.099023 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 659c2937c672482f9ea7b4e524d9847b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.103523 3860 tablet_service.cc:1467] Processing CreateTablet for tablet cc188485180e4fbe995f1b089f9ee8ae (DEFAULT_TABLE table=default.Table_193dfc5ac3914a1f90e082521d0f84d0 [id=4c67d86ae83848199da5466ca7d56b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.104426 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc188485180e4fbe995f1b089f9ee8ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.105520 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 4b65d92574764d508077e9e3c47d56a5 (DEFAULT_TABLE table=default.Table_8cc94cebd1834be58b21d1512bb24571 [id=f397dfcc6d4f409ab0ed55231ef67674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.106014 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b65d92574764d508077e9e3c47d56a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.116591 3824 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.117599 3824 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.124920 4163 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.125136 4163 ts_tablet_manager.cc:1397] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.004s
I20260811 18:57:42.126008 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.126269 3741 raft_consensus.cc:3037] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.129477 3741 raft_consensus.cc:2454] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.130330 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.130579 3741 raft_consensus.cc:2454] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.140987 5630 raft_consensus.cc:482] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.141268 5630 raft_consensus.cc:504] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.143018 5630 leader_election.cc:288] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.148064 4162 tablet_bootstrap.cc:492] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:42.148218 4162 ts_tablet_manager.cc:1397] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.206s user 0.082s sys 0.058s
I20260811 18:57:42.149070 4162 raft_consensus.cc:348] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.149307 4162 raft_consensus.cc:374] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.149355 4162 raft_consensus.cc:729] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.149556 4162 consensus_queue.cc:260] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:42.149853 3824 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.150040 3824 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.150319 3824 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.150887 5768 raft_consensus.cc:2781] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.151069 5768 raft_consensus.cc:686] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.151322 3548 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1] alter complete (version 2)
I20260811 18:57:42.151355 5768 consensus_queue.cc:237] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.161259 4162 ts_tablet_manager.cc:1428] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:42.161396 4169 ts_tablet_manager.cc:1918] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.161595 4169 log.cc:1196] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:42.162027 4169 ts_tablet_manager.cc:1939] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.162871 4163 raft_consensus.cc:348] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.163239 4163 raft_consensus.cc:374] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.164191 3824 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:42.164911 3824 leader_election.cc:341] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: ffa8a4a1c0cb4834be501bdd7d468472
I20260811 18:57:42.165160 5630 raft_consensus.cc:2781] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.167223 5630 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.167438 5630 raft_consensus.cc:3037] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.171573 4162 tablet_bootstrap.cc:492] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:42.176218 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 930b6e8839e24ec9a69854349bc3ecf8 (table default.Table_9db7061f3798449280da2e921e2181bf [id=d7d11af2eee54ab880a15c525b7ad0a9]). Expected version 4 got 0
I20260811 18:57:42.176656 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 114397bea09444479df113552c22e154 (table default.Table_14587bf2389240a1b10454cad65b748a [id=cfba5a388f8f406d8b2a9ae1fc397585]). Expected version 1 got 0
I20260811 18:57:42.177124 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:42.180516 4162 tablet_bootstrap.cc:654] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.197398 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187] alter complete (version 2)
I20260811 18:57:42.210290 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba] alter complete (version 2)
I20260811 18:57:42.210906 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 9600b135935b443daee15cc864ba0032 (DEFAULT_TABLE table=default.Table_40846b4d992644f9ad4942d6e529abb1 [id=36f2ed90d7544a70850e1298a9349d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.212190 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9600b135935b443daee15cc864ba0032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.214226 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1079) took 5281 ms. Trace:
I20260811 18:57:42.214442 4169 rpcz_store.cc:276] 0811 18:57:36.933231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.933289 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.017107 (+5083818us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:42.017121 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.110498 (+ 93377us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.161230 (+ 50732us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.214205 (+ 52975us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22558,"tablet-delete.queue_time_us":5083672}
I20260811 18:57:42.215448 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1080) took 5281 ms. Trace:
I20260811 18:57:42.215619 4169 rpcz_store.cc:276] 0811 18:57:36.933669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.933723 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:42.214943 (+5281220us) ts_tablet_manager.cc:1145] Deleting tablet 8f9b279c6c7344f1931b81e10248b1b6
0811 18:57:42.214956 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.215427 (+ 471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5281059}
I20260811 18:57:42.218675 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5852830ee0544f38aa6ed8c8c2e250a6 (DEFAULT_TABLE table=default.Table_c0ad178862a14f968c9a334a1570eb82 [id=7ce0ced05e7d477fb96f5eda68ffbbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.219705 5630 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.220398 5630 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.221377 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 79539e937f81437ab3b12b8465f12d97 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.221804 4162 tablet_bootstrap.cc:492] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:42.221946 4162 ts_tablet_manager.cc:1397] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.050s user 0.014s sys 0.005s
I20260811 18:57:42.222703 4162 raft_consensus.cc:348] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.222930 4162 raft_consensus.cc:374] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.222978 4162 raft_consensus.cc:729] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.223167 4162 consensus_queue.cc:260] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.225644 3863 tablet_service.cc:1467] Processing CreateTablet for tablet fe48bbab3d7e4f7591883296a9996db7 (DEFAULT_TABLE table=default.Table_e98a0fba26cb4171bf8eb772249c3e47 [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.227525 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 68a7051bc539468aafba30fbaf0669d4 (table default.Table_821a790ed161401bb9758d71a0760916 [id=4a5d04ba5f3942a3942909ebd2bf5e20]) successfully deleted
W20260811 18:57:42.227823 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8f9b279c6c7344f1931b81e10248b1b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f9b279c6c7344f1931b81e10248b1b6 already in progress: creating tablet
I20260811 18:57:42.227766 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc68b2b47ca4426a204df0dea3801f0 (DEFAULT_TABLE table=default.Table_f7751679e1394c27bdfb569a73a14165 [id=65adccc0a8a2492abf41483dcc44bba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.231402 4162 ts_tablet_manager.cc:1428] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:42.232446 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c575ba3060a140759484bea1462ffd73 (DEFAULT_TABLE table=default.Table_cef814687d014504b5a7a569342e4bb4 [id=4d3139f9e2ac44c182ca8bbd99ef2871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.232699 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.232969 3741 raft_consensus.cc:3037] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.235656 3741 raft_consensus.cc:2454] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.237428 4162 tablet_bootstrap.cc:492] T 4e0747ef88574c68ae074732cb7095b9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:42.239317 4162 tablet_bootstrap.cc:654] T 4e0747ef88574c68ae074732cb7095b9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.241400 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 735d4fc7df7f4157ae7df534ab4833ce (DEFAULT_TABLE table=default.Table_fb4433f8104245af96d1746998f92ba1 [id=7e347067e6d049bca74fe91eff329a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.245698 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 69aa5092d0b94654a64f033cf7ae5ee1 (DEFAULT_TABLE table=default.Table_da206548a6a34eb78f9b4bddfc58d921 [id=9af5a612c8df473db99778c812035966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.246838 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5387f5c3f2af4b009fad01c936639e1a (DEFAULT_TABLE table=default.Table_0b6fac836f9146ab9a5700100e2bda0f [id=693fe88a8ea9492b986b8e1d42bc9c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.248762 3548 catalog_manager.cc:5495] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:42.249311 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e1af33fd98c6486f975368ec90493db7 (table default.Table_22bd9bbe66b64e209048c8164f116ba7 [id=7736472828a3499c8ad8026e164b9759]). Expected version 3 got 0
I20260811 18:57:42.249837 3788 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:42.250227 3824 leader_election.cc:334] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.250541 3824 leader_election.cc:302] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:42.250973 5630 raft_consensus.cc:2726] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:42.250836 3548 catalog_manager.cc:5495] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.251888 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (table default.Table_c71b8760db044bad851b4bcd819bf75f [id=7a869d9623bc4e07938b2d26688723a5]). Expected version 2 got 0
I20260811 18:57:42.252735 3548 catalog_manager.cc:5495] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.253535 3824 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.253786 5768 raft_consensus.cc:2781] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.253930 5768 raft_consensus.cc:686] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.253564 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 8dc07c28ccfd48ee9c33e49442c6047b (table default.Table_710bf71d5383421ca80e2caac75ec12f [id=cabc6cc20754488eb152c0d4f686dce0]). Expected version 3 got 0
I20260811 18:57:42.254235 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_9ec52027b1c944a69e731fed8ad2bd30 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 3 got 0
I20260811 18:57:42.254145 5768 consensus_queue.cc:237] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.254590 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (table default.Table_9194817120074797a6f60bf8e24c0636 [id=1323bd99594a4382ba121cc11e79de31]). Expected version 1 got 0
I20260811 18:57:42.254866 3548 catalog_manager.cc:5495] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.255254 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:42.255451 3548 catalog_manager.cc:5495] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.255843 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet a811d4488d0e4541808a5e1aa44d663b (table default.Table_fb8dd203bf3941fbacfd7dad1418e566 [id=a71b71af19a44182aea7e742062558ca]). Expected version 1 got 0
I20260811 18:57:42.260427 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 735d4fc7df7f4157ae7df534ab4833ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.269956 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe48bbab3d7e4f7591883296a9996db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.270610 5593 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.272871 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79539e937f81437ab3b12b8465f12d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.272790 5593 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.273403 5593 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.273610 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5387f5c3f2af4b009fad01c936639e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.276957 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69aa5092d0b94654a64f033cf7ae5ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.283500 4169 tablet_replica.cc:330] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.284436 4169 raft_consensus.cc:2229] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.283878 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc68b2b47ca4426a204df0dea3801f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.298121 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5852830ee0544f38aa6ed8c8c2e250a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.300282 3824 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.300601 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c575ba3060a140759484bea1462ffd73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.302580 4169 raft_consensus.cc:2258] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.306934 4169 ts_tablet_manager.cc:1905] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.332989 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 403943us
W20260811 18:57:42.333355 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b50efab3c782447cad8e45b1fa51aa1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b50efab3c782447cad8e45b1fa51aa1f already in progress: creating tablet
I20260811 18:57:42.333583 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a0116997a1c64e658045c319f1c45861 (table default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]) successfully deleted
I20260811 18:57:42.333770 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8d1e244addf5462499fcadaa3332cba1 (table default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]) successfully deleted
W20260811 18:57:42.333938 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf1c037a37504b3fad9612f5a734dd33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf1c037a37504b3fad9612f5a734dd33 already in progress: creating tablet
W20260811 18:57:42.315110 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.316095 5630 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.339572 5630 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.340413 5630 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.342175 3824 consensus_peers.cc:487] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.346025 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc2a90633b840c5be8d8dab46ad6768 (DEFAULT_TABLE table=default.Table_a4a5dc90bb3449ee94760cc397fd49f3 [id=bb87391ac9a3420daff021b5d24e3c54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.346065 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 586b96fbf0344a85a884263e7316a147 (DEFAULT_TABLE table=default.Table_2e44f4b4162648d6911b8a00b8fd7ba6 [id=358ae5b14b774ccf993af80443f75b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.346555 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc2a90633b840c5be8d8dab46ad6768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.346781 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586b96fbf0344a85a884263e7316a147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.350378 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 531bb240b3af460e832429cc8397a8e2 (DEFAULT_TABLE table=default.Table_343b8634c5a946feacafd49b9aba3c62 [id=dfbfcbc7dfe64bd997058149f61eeb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.350792 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531bb240b3af460e832429cc8397a8e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.360031 4169 ts_tablet_manager.cc:1918] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.360268 4169 log.cc:1196] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/059b325df1db4fde9d4b26b559121316
I20260811 18:57:42.360715 4169 ts_tablet_manager.cc:1939] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.361078 3547 catalog_manager.cc:5495] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.361639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1081) took 5129 ms. Trace:
I20260811 18:57:42.361701 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_9ec52027b1c944a69e731fed8ad2bd30 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 3 got 0
I20260811 18:57:42.361757 4169 rpcz_store.cc:276] 0811 18:57:37.232437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.259911 (+ 27474us) service_pool.cc:227] Handling call
0811 18:57:42.216037 (+4956126us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:42.216049 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.336481 (+120432us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.344806 (+ 8325us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.361617 (+ 16811us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4955906}
I20260811 18:57:42.362254 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1098) took 4929 ms. Trace:
I20260811 18:57:42.362370 4169 rpcz_store.cc:276] 0811 18:57:37.433068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.433495 (+ 427us) service_pool.cc:227] Handling call
0811 18:57:42.362082 (+4928587us) ts_tablet_manager.cc:1145] Deleting tablet 7f4e1fc5686944dfa3eb57702c810eb8
0811 18:57:42.362098 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.362234 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4928349}
I20260811 18:57:42.362743 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1099) took 4929 ms. Trace:
I20260811 18:57:42.362828 4169 rpcz_store.cc:276] 0811 18:57:37.433142 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.436742 (+ 3600us) service_pool.cc:227] Handling call
0811 18:57:42.362613 (+4925871us) ts_tablet_manager.cc:1145] Deleting tablet ea579c33f8b14e9fa7fa389ce7f45931
0811 18:57:42.362624 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.362727 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4925680}
I20260811 18:57:42.363101 4169 tablet_replica.cc:330] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.363328 4169 raft_consensus.cc:2229] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.363497 4169 raft_consensus.cc:2258] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.364070 4169 ts_tablet_manager.cc:1905] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.373888 4169 ts_tablet_manager.cc:1918] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.374074 4169 log.cc:1196] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2c0f6df7a291456d80538e2b76417873
I20260811 18:57:42.374487 4169 ts_tablet_manager.cc:1939] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.375367 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1100) took 4938 ms. Trace:
I20260811 18:57:42.375476 4169 rpcz_store.cc:276] 0811 18:57:37.437115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.443580 (+ 6465us) service_pool.cc:227] Handling call
0811 18:57:42.363055 (+4919475us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:42.363066 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.370581 (+ 7515us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.373766 (+ 3185us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.375340 (+ 1574us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4919240}
I20260811 18:57:42.375934 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1112) took 4890 ms. Trace:
I20260811 18:57:42.376030 4169 rpcz_store.cc:276] 0811 18:57:37.485733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.485810 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:42.375782 (+4889972us) ts_tablet_manager.cc:1145] Deleting tablet 2f2f5ce2b1d14a9a988cff644e196088
0811 18:57:42.375795 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.375914 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4889709}
I20260811 18:57:42.376411 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1111) took 4890 ms. Trace:
W20260811 18:57:42.378671 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a7946503bec9446d8deb02448eb231d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7946503bec9446d8deb02448eb231d5 already in progress: creating tablet
I20260811 18:57:42.379032 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4059e5907b1e488ab9e1d04e563f96cd (table default.Table_e5769978bc1148a7a53c0df338266d1e [id=07bc4a05b4fb49a6883635270832afe9]) successfully deleted
I20260811 18:57:42.379062 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 24c27902a7ba4289b794f6f136897272 (DEFAULT_TABLE table=default.Table_8a306b155dd94cfaa6649e14665e9418 [id=063614ea03ba403890d5ab712daf00ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.379302 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 54ee342f3b2b40a7a194b16b07f7d525 (DEFAULT_TABLE table=default.Table_93d80fcaa1a646119a16dcb3e546b426 [id=156f7d5e97f2469d8b286865f6812105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.379281 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0540e4e43ea34fbfbc66b6035c60684d (table default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]) successfully deleted
I20260811 18:57:42.379534 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b203df4124364737839bf4c1ae072988 (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
I20260811 18:57:42.379645 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24c27902a7ba4289b794f6f136897272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.379897 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ee342f3b2b40a7a194b16b07f7d525. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.379972 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 76e71af5ee8b463eb158ac9ae76ff252 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76e71af5ee8b463eb158ac9ae76ff252 already in progress: creating tablet
W20260811 18:57:42.380159 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ae96e10a2ed4407c9d42a359d82d389e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae96e10a2ed4407c9d42a359d82d389e already in progress: creating tablet
W20260811 18:57:42.380337 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dfb6ac49f1d4403dad38a3125264088a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dfb6ac49f1d4403dad38a3125264088a already in progress: creating tablet
W20260811 18:57:42.380568 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eea325ca357146fcaa408d87803da349 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eea325ca357146fcaa408d87803da349 already in progress: creating tablet
W20260811 18:57:42.380719 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 58bc967770de4ed486b638fb6b174db8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58bc967770de4ed486b638fb6b174db8 already in progress: creating tablet
W20260811 18:57:42.380851 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c44eee516d634d9a986107b55410dfc3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c44eee516d634d9a986107b55410dfc3 already in progress: creating tablet
I20260811 18:57:42.379063 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6bb0873b4c44cca31c1b1942501358 (DEFAULT_TABLE table=default.Table_388bf7f2862e42ebb5e3a779b1ade554 [id=0598c1530ed24c9fa1f29ceb8b3f46ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.388510 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 059b325df1db4fde9d4b26b559121316 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
W20260811 18:57:42.388821 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7f4e1fc5686944dfa3eb57702c810eb8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f4e1fc5686944dfa3eb57702c810eb8 already in progress: creating tablet
W20260811 18:57:42.389026 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ea579c33f8b14e9fa7fa389ce7f45931 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea579c33f8b14e9fa7fa389ce7f45931 already in progress: creating tablet
I20260811 18:57:42.387162 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 1012e838b09043bba433f742a3084d61 (DEFAULT_TABLE table=default.Table_b5f718e0dd6347099abbd1de1d43ca73 [id=8808c040289440708d0175c9ac3ebe16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.386301 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6bb0873b4c44cca31c1b1942501358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.389346 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2c0f6df7a291456d80538e2b76417873 (table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]) successfully deleted
W20260811 18:57:42.389532 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2f2f5ce2b1d14a9a988cff644e196088 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f2f5ce2b1d14a9a988cff644e196088 already in progress: creating tablet
I20260811 18:57:42.389653 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1012e838b09043bba433f742a3084d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.388396 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 163e9cc026d0435a835c55cc760ec5f8 (DEFAULT_TABLE table=default.Table_f20cec778f3248419eb734a5100ae8d1 [id=dbafd5502cb944dba9b18f4e5d863044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.394877 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 163e9cc026d0435a835c55cc760ec5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.388991 3851 tablet_service.cc:1467] Processing CreateTablet for tablet e506527756a344498bc63b31815a2d2d (DEFAULT_TABLE table=default.Table_a49eeecb6ae34fe28d40847ab1738944 [id=6e5a954d5dcb4ad1b5564959496a3f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.396992 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e506527756a344498bc63b31815a2d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.376502 4169 rpcz_store.cc:276] 0811 18:57:37.485617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486402 (+ 785us) service_pool.cc:227] Handling call
0811 18:57:42.376304 (+4889902us) ts_tablet_manager.cc:1145] Deleting tablet d21973bc8e514a089c2beb24a686060f
0811 18:57:42.376314 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.376394 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4889742}
I20260811 18:57:42.401420 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1113) took 4915 ms. Trace:
I20260811 18:57:42.401561 4169 rpcz_store.cc:276] 0811 18:57:37.486415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486469 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:42.401281 (+4914812us) ts_tablet_manager.cc:1145] Deleting tablet f2b66196716a4ba3893b6b8bfec2b8c0
0811 18:57:42.401294 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.401399 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4914598}
I20260811 18:57:42.401998 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1114) took 4915 ms. Trace:
I20260811 18:57:42.402137 4169 rpcz_store.cc:276] 0811 18:57:37.486832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486875 (+ 43us) service_pool.cc:227] Handling call
0811 18:57:42.401891 (+4915016us) ts_tablet_manager.cc:1145] Deleting tablet a3973dfdaac746a6af7a402277d2711b
0811 18:57:42.401902 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.401982 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4914831}
I20260811 18:57:42.402559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1117) took 4913 ms. Trace:
I20260811 18:57:42.402688 4169 rpcz_store.cc:276] 0811 18:57:37.488700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.488796 (+ 96us) service_pool.cc:227] Handling call
0811 18:57:42.402458 (+4913662us) ts_tablet_manager.cc:1145] Deleting tablet 15dd14e2b4bd4a08b5b08f532a55291b
0811 18:57:42.402467 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.402542 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4913471}
I20260811 18:57:42.403088 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1131) took 4911 ms. Trace:
I20260811 18:57:42.403215 4169 rpcz_store.cc:276] 0811 18:57:37.491695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.780636 (+288941us) service_pool.cc:227] Handling call
0811 18:57:42.402956 (+4622320us) ts_tablet_manager.cc:1145] Deleting tablet 0495c5b770b34d54afb1379a6e36204c
0811 18:57:42.402966 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.403054 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4608038}
I20260811 18:57:42.403607 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1132) took 4911 ms. Trace:
I20260811 18:57:42.403728 4169 rpcz_store.cc:276] 0811 18:57:37.491768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.784392 (+292624us) service_pool.cc:227] Handling call
0811 18:57:42.403498 (+4619106us) ts_tablet_manager.cc:1145] Deleting tablet 75ceba7bfc8240268b16c789f07813d1
0811 18:57:42.403508 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.403589 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4585944}
I20260811 18:57:42.404168 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1133) took 4912 ms. Trace:
I20260811 18:57:42.404301 4169 rpcz_store.cc:276] 0811 18:57:37.491840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.785280 (+293440us) service_pool.cc:227] Handling call
0811 18:57:42.404003 (+4618723us) ts_tablet_manager.cc:1145] Deleting tablet 5a75a480f38047d880b0c1fa51403786
0811 18:57:42.404024 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.404146 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":405,"tablet-delete.queue_time_us":4586447}
I20260811 18:57:42.404727 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1134) took 4912 ms. Trace:
I20260811 18:57:42.404852 4169 rpcz_store.cc:276] 0811 18:57:37.491905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.787870 (+295965us) service_pool.cc:227] Handling call
0811 18:57:42.404603 (+4616733us) ts_tablet_manager.cc:1145] Deleting tablet d32212c817c94fad8379462b027e7d3d
0811 18:57:42.404616 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.404703 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4572213}
I20260811 18:57:42.405264 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1147) took 4523 ms. Trace:
I20260811 18:57:42.405386 4169 rpcz_store.cc:276] 0811 18:57:37.881847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882287 (+ 440us) service_pool.cc:227] Handling call
0811 18:57:42.405132 (+4522845us) ts_tablet_manager.cc:1145] Deleting tablet 3195082643264e0d90f70e1007ab9b24
0811 18:57:42.405143 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.405239 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4483795}
I20260811 18:57:42.405694 4169 tablet_replica.cc:330] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.405890 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.405962 4169 raft_consensus.cc:2229] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.406193 4169 raft_consensus.cc:2258] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.406847 4169 ts_tablet_manager.cc:1905] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.411499 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d21973bc8e514a089c2beb24a686060f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d21973bc8e514a089c2beb24a686060f already in progress: creating tablet
W20260811 18:57:42.411775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f2b66196716a4ba3893b6b8bfec2b8c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2b66196716a4ba3893b6b8bfec2b8c0 already in progress: creating tablet
I20260811 18:57:42.409686 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c400808b5240455d8bfd8980b943ad01 (DEFAULT_TABLE table=default.Table_194b336657844c34ac93e94c90db9444 [id=62d6b6510702407b92cfe27215761113]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:42.411963 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a3973dfdaac746a6af7a402277d2711b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3973dfdaac746a6af7a402277d2711b already in progress: creating tablet
W20260811 18:57:42.412132 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 15dd14e2b4bd4a08b5b08f532a55291b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15dd14e2b4bd4a08b5b08f532a55291b already in progress: creating tablet
I20260811 18:57:42.412293 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c400808b5240455d8bfd8980b943ad01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.412309 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0495c5b770b34d54afb1379a6e36204c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0495c5b770b34d54afb1379a6e36204c already in progress: creating tablet
W20260811 18:57:42.412457 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 75ceba7bfc8240268b16c789f07813d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75ceba7bfc8240268b16c789f07813d1 already in progress: creating tablet
W20260811 18:57:42.412621 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5a75a480f38047d880b0c1fa51403786 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a75a480f38047d880b0c1fa51403786 already in progress: creating tablet
W20260811 18:57:42.412798 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d32212c817c94fad8379462b027e7d3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32212c817c94fad8379462b027e7d3d already in progress: creating tablet
W20260811 18:57:42.412941 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3195082643264e0d90f70e1007ab9b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3195082643264e0d90f70e1007ab9b24 already in progress: creating tablet
I20260811 18:57:42.417354 4169 ts_tablet_manager.cc:1918] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:42.417930 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.430086 3824 consensus_peers.cc:487] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.430169 4169 log.cc:1196] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5d495354c0994fb4a74c376078e41a26
I20260811 18:57:42.430819 4169 ts_tablet_manager.cc:1939] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.431466 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1148) took 4548 ms. Trace:
I20260811 18:57:42.431705 4169 rpcz_store.cc:276] 0811 18:57:37.882582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882630 (+ 48us) service_pool.cc:227] Handling call
0811 18:57:42.405649 (+4523019us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:42.405658 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.414140 (+ 8482us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.417263 (+ 3123us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.431444 (+ 14181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4484232}
I20260811 18:57:42.432246 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5d495354c0994fb4a74c376078e41a26 (table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]) successfully deleted
I20260811 18:57:42.432313 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1149) took 4549 ms. Trace:
I20260811 18:57:42.432406 4169 rpcz_store.cc:276] 0811 18:57:37.882860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882904 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:42.432163 (+4549259us) ts_tablet_manager.cc:1145] Deleting tablet b7b38d3b9570467a9128bd7fe73acf2f
0811 18:57:42.432179 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.432293 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4510691}
I20260811 18:57:42.432998 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1150) took 4549 ms. Trace:
W20260811 18:57:42.433050 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b7b38d3b9570467a9128bd7fe73acf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b38d3b9570467a9128bd7fe73acf2f already in progress: creating tablet
I20260811 18:57:42.433156 4169 rpcz_store.cc:276] 0811 18:57:37.883098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883139 (+ 41us) service_pool.cc:227] Handling call
0811 18:57:42.432882 (+4549743us) ts_tablet_manager.cc:1145] Deleting tablet 429eb5824f3f4b2b8da6e17ba7c69b92
0811 18:57:42.432894 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.432981 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4511362}
W20260811 18:57:42.433568 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 429eb5824f3f4b2b8da6e17ba7c69b92 already in progress: creating tablet
I20260811 18:57:42.433597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1151) took 4550 ms. Trace:
I20260811 18:57:42.433681 4169 rpcz_store.cc:276] 0811 18:57:37.883320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883387 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:42.433487 (+4550100us) ts_tablet_manager.cc:1145] Deleting tablet f89eebbc40234794a081734f834294ab
0811 18:57:42.433500 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.433579 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4511913}
W20260811 18:57:42.434123 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f89eebbc40234794a081734f834294ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89eebbc40234794a081734f834294ab already in progress: creating tablet
I20260811 18:57:42.434137 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1152) took 4550 ms. Trace:
I20260811 18:57:42.434218 4169 rpcz_store.cc:276] 0811 18:57:37.883429 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883568 (+ 139us) service_pool.cc:227] Handling call
0811 18:57:42.434020 (+4550452us) ts_tablet_manager.cc:1145] Deleting tablet a5bbd3274980470b9fe71457b49bdd62
0811 18:57:42.434031 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.434120 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4512371}
I20260811 18:57:42.434834 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1159) took 4259 ms. Trace:
W20260811 18:57:42.434845 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a5bbd3274980470b9fe71457b49bdd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5bbd3274980470b9fe71457b49bdd62 already in progress: creating tablet
I20260811 18:57:42.438097 4169 rpcz_store.cc:276] 0811 18:57:38.175718 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.176437 (+ 719us) service_pool.cc:227] Handling call
0811 18:57:42.434696 (+4258259us) ts_tablet_manager.cc:1145] Deleting tablet bfea0cd619024204b9bff1e2dd62b7b0
0811 18:57:42.434707 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.434817 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4257989}
W20260811 18:57:42.439379 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bfea0cd619024204b9bff1e2dd62b7b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfea0cd619024204b9bff1e2dd62b7b0 already in progress: creating tablet
I20260811 18:57:42.439507 4169 tablet_replica.cc:330] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.439751 4169 raft_consensus.cc:2229] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:42.439913 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet c6852aaef1d5486b87e5462eae532719 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.439944 4169 raft_consensus.cc:2258] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.440572 4169 ts_tablet_manager.cc:1905] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.429311 4163 raft_consensus.cc:729] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.450635 4163 consensus_queue.cc:260] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.454237 4163 ts_tablet_manager.cc:1428] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.329s user 0.022s sys 0.000s
I20260811 18:57:42.454581 4163 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.456033 4163 tablet_bootstrap.cc:654] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.457226 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.472922 4169 ts_tablet_manager.cc:1918] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.478875 4169 log.cc:1196] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:42.481096 4169 ts_tablet_manager.cc:1939] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.483975 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1156) took 4311 ms. Trace:
I20260811 18:57:42.486773 4169 rpcz_store.cc:276] 0811 18:57:38.172816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.173076 (+ 260us) service_pool.cc:227] Handling call
0811 18:57:42.439454 (+4266378us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:42.439466 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.460835 (+ 21369us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.472806 (+ 11971us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.483946 (+ 11140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4253002}
I20260811 18:57:42.488235 4163 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.488386 4163 ts_tablet_manager.cc:1397] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:42.488818 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ce94d37efd2245bf8cc71fdd90bd9735 (table default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]) successfully deleted
I20260811 18:57:42.489097 4163 raft_consensus.cc:348] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.489432 4163 raft_consensus.cc:374] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.489519 4163 raft_consensus.cc:729] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.489806 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1157) took 4316 ms. Trace:
I20260811 18:57:42.491706 4169 rpcz_store.cc:276] 0811 18:57:38.173320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.175314 (+ 1994us) service_pool.cc:227] Handling call
0811 18:57:42.489363 (+4314049us) ts_tablet_manager.cc:1145] Deleting tablet 6710084f21734dea9c7ce1f43d1bf587
0811 18:57:42.489381 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.489768 (+ 387us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4302808}
W20260811 18:57:42.492507 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6710084f21734dea9c7ce1f43d1bf587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6710084f21734dea9c7ce1f43d1bf587 already in progress: creating tablet
I20260811 18:57:42.493196 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1158) took 4317 ms. Trace:
I20260811 18:57:42.489856 4163 consensus_queue.cc:260] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.495868 4169 rpcz_store.cc:276] 0811 18:57:38.175603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.175667 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:42.492987 (+4317320us) ts_tablet_manager.cc:1145] Deleting tablet e9f0e443ab84424dac307b745b496986
0811 18:57:42.493005 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.493167 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4306388}
I20260811 18:57:42.496347 4163 ts_tablet_manager.cc:1428] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
W20260811 18:57:42.496583 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e9f0e443ab84424dac307b745b496986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9f0e443ab84424dac307b745b496986 already in progress: creating tablet
I20260811 18:57:42.497149 4163 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.498606 4163 tablet_bootstrap.cc:654] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.509779 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1160) took 4332 ms. Trace:
I20260811 18:57:42.510025 4169 rpcz_store.cc:276] 0811 18:57:38.176860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.176904 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:42.496984 (+4320080us) ts_tablet_manager.cc:1145] Deleting tablet 69059a0667234a408cc1c09585df63bc
0811 18:57:42.509632 (+ 12648us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.509755 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4308070}
I20260811 18:57:42.510468 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:42.510674 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 69059a0667234a408cc1c09585df63bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69059a0667234a408cc1c09585df63bc already in progress: creating tablet
I20260811 18:57:42.511119 4169 tablet_replica.cc:330] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.511474 4169 raft_consensus.cc:2229] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.511721 4169 raft_consensus.cc:2258] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.512411 4169 ts_tablet_manager.cc:1905] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.522080 4163 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.522253 4163 ts_tablet_manager.cc:1397] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.005s
I20260811 18:57:42.523033 4163 raft_consensus.cc:348] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.523257 4163 raft_consensus.cc:374] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.523303 4163 raft_consensus.cc:729] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.523475 4163 consensus_queue.cc:260] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.524003 4163 ts_tablet_manager.cc:1428] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.524250 4163 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.525916 4163 tablet_bootstrap.cc:654] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.543344 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.548653 4163 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.548823 4163 ts_tablet_manager.cc:1397] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.000s
I20260811 18:57:42.549618 4163 raft_consensus.cc:348] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.549867 4163 raft_consensus.cc:374] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.549921 4163 raft_consensus.cc:729] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.550186 4163 consensus_queue.cc:260] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.559468 4163 ts_tablet_manager.cc:1428] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:42.565944 4163 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.568301 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.571682 4169 ts_tablet_manager.cc:1918] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.571831 4169 log.cc:1196] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:42.578820 4169 ts_tablet_manager.cc:1939] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.579334 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1161) took 4390 ms. Trace:
I20260811 18:57:42.579443 4169 rpcz_store.cc:276] 0811 18:57:38.189107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.190677 (+ 1570us) service_pool.cc:227] Handling call
0811 18:57:42.511068 (+4320391us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:42.511079 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.515635 (+ 4556us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.571553 (+ 55918us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.579313 (+ 7760us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4320192}
I20260811 18:57:42.580016 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c99ceca57cb245f4a3a6cb4b36f1306d (table default.Table_8a27cf06565547a39a8eee2420c9a31b [id=0a4e4826c2cf4250a34e7e26eece8e3a]) successfully deleted
I20260811 18:57:42.580374 4169 tablet_replica.cc:330] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.580644 4169 raft_consensus.cc:2229] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.577244 4163 tablet_bootstrap.cc:654] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.580829 4169 raft_consensus.cc:2258] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.582808 4169 ts_tablet_manager.cc:1905] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.591742 4169 ts_tablet_manager.cc:1918] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.591898 4169 log.cc:1196] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:42.592206 4169 ts_tablet_manager.cc:1939] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.593073 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1162) took 4398 ms. Trace:
I20260811 18:57:42.593185 4169 rpcz_store.cc:276] 0811 18:57:38.195030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.195193 (+ 163us) service_pool.cc:227] Handling call
0811 18:57:42.579867 (+4384674us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:42.579879 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.588407 (+ 8528us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.591620 (+ 3213us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.593045 (+ 1425us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4384474}
I20260811 18:57:42.593729 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7b404e99e9414c67b3cd5aff12203a53 (table default.Table_d8397de033b74ce9accaa4085f59d7e6 [id=bebb3efbbc56416597abdf4a0b3952ed]) successfully deleted
I20260811 18:57:42.597200 4163 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.597352 4163 ts_tablet_manager.cc:1397] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.000s
I20260811 18:57:42.598083 4163 raft_consensus.cc:348] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.598321 4163 raft_consensus.cc:374] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.604180 4163 raft_consensus.cc:729] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.604547 4163 consensus_queue.cc:260] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.605719 4163 ts_tablet_manager.cc:1428] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260811 18:57:42.606237 4163 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.607964 4163 tablet_bootstrap.cc:654] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.612079 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1164) took 4410 ms. Trace:
I20260811 18:57:42.615387 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.615346 4169 rpcz_store.cc:276] 0811 18:57:38.201782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.202210 (+ 428us) service_pool.cc:227] Handling call
0811 18:57:42.596378 (+4394168us) ts_tablet_manager.cc:1145] Deleting tablet d11b889e9d044a049d0586382c263951
0811 18:57:42.596387 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.612049 (+ 15662us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4385918}
W20260811 18:57:42.616004 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d11b889e9d044a049d0586382c263951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d11b889e9d044a049d0586382c263951 already in progress: creating tablet
I20260811 18:57:42.616715 4169 tablet_replica.cc:330] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.616995 4169 raft_consensus.cc:2229] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.617293 4169 raft_consensus.cc:2258] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.618155 4169 ts_tablet_manager.cc:1905] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.620146 5630 raft_consensus.cc:482] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.620289 5630 raft_consensus.cc:504] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.620970 5630 leader_election.cc:288] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.623260 3824 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.648000 4169 ts_tablet_manager.cc:1918] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.649758 4169 log.cc:1196] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:42.650782 4169 ts_tablet_manager.cc:1939] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.651908 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1165) took 4450 ms. Trace:
I20260811 18:57:42.653110 4169 rpcz_store.cc:276] 0811 18:57:38.201889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.214685 (+ 12796us) service_pool.cc:227] Handling call
0811 18:57:42.616395 (+4401710us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:42.616662 (+ 267us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.637932 (+ 21270us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.647876 (+ 9944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.651872 (+ 3996us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4401458}
I20260811 18:57:42.654022 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0cccebfc54484a9b9abffcce4e459fb5 (table default.Table_d88f576cc3ca4c6eadc6e071aacc15b2 [id=5fc5ade3f38c438d9b501127ac866b88]) successfully deleted
I20260811 18:57:42.654731 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1163) took 4453 ms. Trace:
I20260811 18:57:42.655313 4169 rpcz_store.cc:276] 0811 18:57:38.201675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.214985 (+ 13310us) service_pool.cc:227] Handling call
0811 18:57:42.654536 (+4439551us) ts_tablet_manager.cc:1145] Deleting tablet 82b04e37fc264fc08e0ed9517356a17d
0811 18:57:42.654553 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.654705 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4439360}
W20260811 18:57:42.656122 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 82b04e37fc264fc08e0ed9517356a17d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82b04e37fc264fc08e0ed9517356a17d already in progress: creating tablet
I20260811 18:57:42.656631 4169 tablet_replica.cc:330] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.657080 4169 raft_consensus.cc:2229] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.657549 4169 raft_consensus.cc:2258] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.659209 4169 ts_tablet_manager.cc:1905] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.664034 4163 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.664291 4163 ts_tablet_manager.cc:1397] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.058s user 0.014s sys 0.004s
I20260811 18:57:42.665441 4163 raft_consensus.cc:348] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.665882 4163 raft_consensus.cc:374] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.666947 4163 raft_consensus.cc:729] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.667384 4163 consensus_queue.cc:260] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.683480 5630 raft_consensus.cc:482] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.683712 5630 raft_consensus.cc:504] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.684626 5630 leader_election.cc:288] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.687387 3824 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.688678 4163 ts_tablet_manager.cc:1428] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
W20260811 18:57:42.692883 3824 consensus_peers.cc:487] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.701792 4163 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.704968 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:42.707319 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 220ea85575904e0ea8fdf7799c9ddeed (DEFAULT_TABLE table=default.Table_bef031fd83e44076810e019e0038f70f [id=e216b4160f2a4e40988252fe28960cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.707803 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220ea85575904e0ea8fdf7799c9ddeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.709299 4163 tablet_bootstrap.cc:654] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.711007 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 88f35da2dc8a43fc86f9a12958e3be1b (DEFAULT_TABLE table=default.Table_b149c08055014e08aad5ad5be387047b [id=a2113616a0b445549c700f7e1c707191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711114 3863 tablet_service.cc:1467] Processing CreateTablet for tablet ac974915a88b41c59d7613d4a69f62c2 (DEFAULT_TABLE table=default.Table_65f6fe5195b443fcbd5f077f1a147107 [id=f93e0ff8e13f48739183e1dc3fc3a66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711277 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 73880e575a4845658fe1766aa9715c9f (DEFAULT_TABLE table=default.Table_beb35f174ef644f5994dd0282f0d4ee3 [id=82c9ae34f67544de859f6c7ffa7e7dd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711385 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f35da2dc8a43fc86f9a12958e3be1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.711540 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac974915a88b41c59d7613d4a69f62c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.711758 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f0b07f91bb794ad88326db69d154c173 (DEFAULT_TABLE table=default.Table_525108ad39d242bc818b5bb86a5c0103 [id=1b6f82af31cb46bfb9a295a323479be9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.712071 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b07f91bb794ad88326db69d154c173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.713598 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73880e575a4845658fe1766aa9715c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.714630 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 8300d38bad3244eb9bc4cc20bf4eb4d2 (DEFAULT_TABLE table=default.Table_6f3b91b31a884708a27160ebfca75cab [id=c83e220d78e54a8e8291795ba64f94f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.714967 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8300d38bad3244eb9bc4cc20bf4eb4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.715736 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0efae9cb481a435c8a4511fd6d1fe415 (DEFAULT_TABLE table=default.Table_88ee0abdcd9943568a07d73017ff598a [id=06b70587cd6d417a8ef0b3ec4ecc7c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.716068 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efae9cb481a435c8a4511fd6d1fe415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.718983 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 730a2aed5a1344b387398114f6105d3f (DEFAULT_TABLE table=default.Table_3aa1c735a06c46e9833fdb958cac274d [id=0916d009f52243bba02b715768a244f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.719501 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6d41c0dd71408e89231ff56fde6e29 (DEFAULT_TABLE table=default.Table_22ffcd40375b45aab1616fa1fb6f030d [id=73fc4e78208d4494b0c0e9a1150c0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.719944 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 332d3d9ddb224a4697809b19b622749d (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.720898 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f336fdd55965432f83b2d1c436fdf1fa (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:42.727080 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet 7fdacc021a974bd981dc30e4915378a3: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.730588 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6d41c0dd71408e89231ff56fde6e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.731827 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 730a2aed5a1344b387398114f6105d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.709857 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 523be25072014425b867e97d429334e8 (DEFAULT_TABLE table=default.Table_9c66e99a00ba4953b1faed44821e14ba [id=edb2d64fea5e43a094186bea93ee7ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.734205 3870 tablet_service.cc:1467] Processing CreateTablet for tablet a4e0b84bdb134a459d2d9afd046326c2 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.734386 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 5faabb319fc84f24ab548a707be284f1 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.745803 4163 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.745987 4163 ts_tablet_manager.cc:1397] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.000s
W20260811 18:57:42.750468 3824 consensus_peers.cc:487] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.753917 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332d3d9ddb224a4697809b19b622749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754148 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523be25072014425b867e97d429334e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754312 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e0b84bdb134a459d2d9afd046326c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754679 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f336fdd55965432f83b2d1c436fdf1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.758476 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5faabb319fc84f24ab548a707be284f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.768261 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:42.779369 4169 ts_tablet_manager.cc:1918] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.779520 4169 log.cc:1196] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:42.779861 4169 ts_tablet_manager.cc:1939] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.780691 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1166) took 4461 ms. Trace:
I20260811 18:57:42.780788 4169 rpcz_store.cc:276] 0811 18:57:38.319222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.319294 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:42.656572 (+4337278us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:42.656586 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.704236 (+ 47650us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.779254 (+ 75018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.780666 (+ 1412us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4337010}
I20260811 18:57:42.781215 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8cce97bfdb1d41e685e091c8c5cc6946 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
W20260811 18:57:42.785063 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:42.790506 4169 tablet_replica.cc:330] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.791412 4169 raft_consensus.cc:2229] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.791795 4169 raft_consensus.cc:2258] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.792847 4169 ts_tablet_manager.cc:1905] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.801352 3824 consensus_peers.cc:487] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.803807 5593 raft_consensus.cc:482] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.803963 5593 raft_consensus.cc:504] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.804564 5593 leader_election.cc:288] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.806273 3824 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.807173 4169 ts_tablet_manager.cc:1918] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.807461 4169 log.cc:1196] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7057b8a7bae249caaed304b5c66e6871
I20260811 18:57:42.808166 4169 ts_tablet_manager.cc:1939] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.809547 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1167) took 4481 ms. Trace:
I20260811 18:57:42.809767 4169 rpcz_store.cc:276] 0811 18:57:38.328355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.334770 (+ 6415us) service_pool.cc:227] Handling call
0811 18:57:42.781082 (+4446312us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:42.790400 (+ 9318us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.799654 (+ 9254us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.807020 (+ 7366us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.809524 (+ 2504us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4446052}
I20260811 18:57:42.812484 4169 tablet_replica.cc:330] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.812805 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7057b8a7bae249caaed304b5c66e6871 (table default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]) successfully deleted
I20260811 18:57:42.812969 4169 raft_consensus.cc:2229] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:42.813534 4169 raft_consensus.cc:2258] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.814790 4169 ts_tablet_manager.cc:1905] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.818451 4163 raft_consensus.cc:348] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.818869 4163 raft_consensus.cc:374] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.818976 4163 raft_consensus.cc:729] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.819267 4163 consensus_queue.cc:260] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.819928 4163 ts_tablet_manager.cc:1428] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.824244 4169 ts_tablet_manager.cc:1918] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:42.824560 4169 log.cc:1196] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:42.825245 4169 ts_tablet_manager.cc:1939] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.828117 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.833169 4163 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.837649 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1169) took 4482 ms. Trace:
I20260811 18:57:42.839075 4169 rpcz_store.cc:276] 0811 18:57:38.355194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.355377 (+ 183us) service_pool.cc:227] Handling call
0811 18:57:42.812402 (+4457025us) ts_tablet_manager.cc:1145] Deleting tablet b4c7f500a5444bae9a015a5e903c0622
0811 18:57:42.812416 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.819622 (+ 7206us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.824081 (+ 4459us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.837404 (+ 13323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4456781}
I20260811 18:57:42.841537 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]) successfully deleted
I20260811 18:57:42.842799 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1173) took 4443 ms. Trace:
I20260811 18:57:42.843394 4169 rpcz_store.cc:276] 0811 18:57:38.399428 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.399489 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:42.841969 (+4442480us) ts_tablet_manager.cc:1145] Deleting tablet 76506ad179a9408c9e4cf8e4275ebddd
0811 18:57:42.841983 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.842772 (+ 789us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4424795}
I20260811 18:57:42.844561 4163 tablet_bootstrap.cc:654] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:42.845448 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 76506ad179a9408c9e4cf8e4275ebddd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76506ad179a9408c9e4cf8e4275ebddd already in progress: creating tablet
I20260811 18:57:42.848579 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1172) took 4449 ms. Trace:
I20260811 18:57:42.849215 4169 rpcz_store.cc:276] 0811 18:57:38.399307 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.401725 (+ 2418us) service_pool.cc:227] Handling call
0811 18:57:42.847377 (+4445652us) ts_tablet_manager.cc:1145] Deleting tablet 31bb11617f1548f88f534289706edb06
0811 18:57:42.847559 (+ 182us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.848386 (+ 827us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4422771}
W20260811 18:57:42.850811 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 31bb11617f1548f88f534289706edb06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31bb11617f1548f88f534289706edb06 already in progress: creating tablet
I20260811 18:57:42.851562 4163 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.851715 4163 ts_tablet_manager.cc:1397] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.005s
I20260811 18:57:42.852439 4163 raft_consensus.cc:348] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.852738 4163 raft_consensus.cc:374] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.852809 4163 raft_consensus.cc:729] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.853048 4163 consensus_queue.cc:260] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.853652 4163 ts_tablet_manager.cc:1428] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:42.860051 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1185) took 4193 ms. Trace:
I20260811 18:57:42.860337 4169 rpcz_store.cc:276] 0811 18:57:38.666184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.666250 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:42.855270 (+4189020us) ts_tablet_manager.cc:1145] Deleting tablet ee5b7d1e32514ad48177371582e85241
0811 18:57:42.859870 (+ 4600us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.860012 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4188713}
I20260811 18:57:42.861363 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.861413 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1186) took 4194 ms. Trace:
I20260811 18:57:42.861634 4169 rpcz_store.cc:276] 0811 18:57:38.666714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.666766 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:42.860840 (+4194074us) ts_tablet_manager.cc:1145] Deleting tablet 4451cc4d7c5940cdb41c021ae486a5a9
0811 18:57:42.861272 (+ 432us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.861386 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4193837}
W20260811 18:57:42.861719 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ee5b7d1e32514ad48177371582e85241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5b7d1e32514ad48177371582e85241 already in progress: creating tablet
I20260811 18:57:42.862159 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1187) took 4195 ms. Trace:
W20260811 18:57:42.862215 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4451cc4d7c5940cdb41c021ae486a5a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4451cc4d7c5940cdb41c021ae486a5a9 already in progress: creating tablet
I20260811 18:57:42.862290 4169 rpcz_store.cc:276] 0811 18:57:38.667145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.667194 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:42.862030 (+4194836us) ts_tablet_manager.cc:1145] Deleting tablet c590c98e39f84ea09b26bbe8640ca93d
0811 18:57:42.862041 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.862141 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4194649}
I20260811 18:57:42.862696 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1188) took 4195 ms. Trace:
W20260811 18:57:42.862754 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c590c98e39f84ea09b26bbe8640ca93d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c590c98e39f84ea09b26bbe8640ca93d already in progress: creating tablet
I20260811 18:57:42.862802 4169 rpcz_store.cc:276] 0811 18:57:38.667507 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.667553 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:42.862590 (+4195037us) ts_tablet_manager.cc:1145] Deleting tablet cb54beb34a654fbdb9c7632901bb0b9e
0811 18:57:42.862600 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.862681 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4194867}
I20260811 18:57:42.863137 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1184) took 4197 ms. Trace:
W20260811 18:57:42.863214 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cb54beb34a654fbdb9c7632901bb0b9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb54beb34a654fbdb9c7632901bb0b9e already in progress: creating tablet
I20260811 18:57:42.863219 4169 rpcz_store.cc:276] 0811 18:57:38.666101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.697528 (+ 31427us) service_pool.cc:227] Handling call
0811 18:57:42.863020 (+4165492us) ts_tablet_manager.cc:1145] Deleting tablet eb43127dda814a718b338906fc2a3c47
0811 18:57:42.863033 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863120 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164924}
I20260811 18:57:42.863545 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1193) took 3934 ms. Trace:
W20260811 18:57:42.863648 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet eb43127dda814a718b338906fc2a3c47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb43127dda814a718b338906fc2a3c47 already in progress: creating tablet
I20260811 18:57:42.863654 4169 rpcz_store.cc:276] 0811 18:57:38.929261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.929319 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.863441 (+3934122us) ts_tablet_manager.cc:1145] Deleting tablet 09366eccf38a4876840cc5ddddbbb103
0811 18:57:42.863452 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863530 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3933941}
I20260811 18:57:42.863960 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1206) took 3643 ms. Trace:
W20260811 18:57:42.864085 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 09366eccf38a4876840cc5ddddbbb103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09366eccf38a4876840cc5ddddbbb103 already in progress: creating tablet
I20260811 18:57:42.864039 4169 rpcz_store.cc:276] 0811 18:57:39.220336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.220391 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:42.863861 (+3643470us) ts_tablet_manager.cc:1145] Deleting tablet 1ccd56a8b7cc454ea3f2f08b59374521
0811 18:57:42.863871 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863944 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3643278}
I20260811 18:57:42.864377 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1207) took 3643 ms. Trace:
I20260811 18:57:42.864455 4169 rpcz_store.cc:276] 0811 18:57:39.221024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.221076 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:42.864267 (+3643191us) ts_tablet_manager.cc:1145] Deleting tablet 78868ff1f8024163ae1d782151100cd4
0811 18:57:42.864276 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.864361 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3642789}
W20260811 18:57:42.864512 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1ccd56a8b7cc454ea3f2f08b59374521 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ccd56a8b7cc454ea3f2f08b59374521 already in progress: creating tablet
I20260811 18:57:42.864707 4169 tablet_replica.cc:330] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.864943 4169 raft_consensus.cc:2229] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:42.864974 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 78868ff1f8024163ae1d782151100cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78868ff1f8024163ae1d782151100cd4 already in progress: creating tablet
I20260811 18:57:42.865180 4169 raft_consensus.cc:2258] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.865797 4169 ts_tablet_manager.cc:1905] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.870751 4163 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.875451 4163 tablet_bootstrap.cc:654] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.875726 4169 ts_tablet_manager.cc:1918] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.884541 4169 log.cc:1196] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cf6c8d542ae2489ba466ffadd518db12
I20260811 18:57:42.884967 4169 ts_tablet_manager.cc:1939] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.885789 4163 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.885814 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1208) took 3664 ms. Trace:
I20260811 18:57:42.886402 4163 ts_tablet_manager.cc:1397] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.006s
I20260811 18:57:42.886302 4169 rpcz_store.cc:276] 0811 18:57:39.221671 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.221717 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:42.864660 (+3642943us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:42.864671 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.872550 (+ 7879us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.875625 (+ 3075us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.885789 (+ 10164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3642779}
I20260811 18:57:42.887570 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cf6c8d542ae2489ba466ffadd518db12 (table default.Table_70493900fc9346389c1d4b71c611b962 [id=97d17499bd684f0abf98e0582fa06f5d]) successfully deleted
I20260811 18:57:42.888541 4163 raft_consensus.cc:348] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:42.888778 4163 raft_consensus.cc:374] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.888837 4163 raft_consensus.cc:729] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.889050 4163 consensus_queue.cc:260] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:42.889820 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1209) took 3517 ms. Trace:
I20260811 18:57:42.889905 4169 rpcz_store.cc:276] 0811 18:57:39.372248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.372327 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:42.889676 (+3517349us) ts_tablet_manager.cc:1145] Deleting tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8
0811 18:57:42.889689 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.889801 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3517076}
I20260811 18:57:42.890331 4163 ts_tablet_manager.cc:1428] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
W20260811 18:57:42.890780 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 already in progress: creating tablet
I20260811 18:57:42.891009 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1210) took 3400 ms. Trace:
I20260811 18:57:42.891079 4169 rpcz_store.cc:276] 0811 18:57:39.490527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.490594 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:42.890115 (+3399521us) ts_tablet_manager.cc:1145] Deleting tablet fa95131ce50943e3b0c6b5bdd5a30939
0811 18:57:42.890484 (+ 369us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.890992 (+ 508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3397060}
I20260811 18:57:42.894704 5593 raft_consensus.cc:482] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.894858 5593 raft_consensus.cc:504] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.895455 5593 leader_election.cc:288] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.898972 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fa95131ce50943e3b0c6b5bdd5a30939 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa95131ce50943e3b0c6b5bdd5a30939 already in progress: creating tablet
W20260811 18:57:42.899210 3824 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.900020 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.902073 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1219) took 3215 ms. Trace:
I20260811 18:57:42.902284 4169 rpcz_store.cc:276] 0811 18:57:39.687067 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687118 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:42.891275 (+3204157us) ts_tablet_manager.cc:1145] Deleting tablet ab03996c5c624e5eb17267b124e27f45
0811 18:57:42.900531 (+ 9256us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.902032 (+ 1501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3197104}
I20260811 18:57:42.903328 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1218) took 3216 ms. Trace:
I20260811 18:57:42.903944 4163 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.903638 4169 rpcz_store.cc:276] 0811 18:57:39.686933 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.686991 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.902922 (+3215931us) ts_tablet_manager.cc:1145] Deleting tablet bc38e6760754449fbd6209d765cce70c
0811 18:57:42.902956 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.903310 (+ 354us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3205919}
I20260811 18:57:42.904718 4169 tablet_replica.cc:330] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:42.905208 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ab03996c5c624e5eb17267b124e27f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab03996c5c624e5eb17267b124e27f45 already in progress: creating tablet
W20260811 18:57:42.905501 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bc38e6760754449fbd6209d765cce70c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc38e6760754449fbd6209d765cce70c already in progress: creating tablet
I20260811 18:57:42.905821 4169 raft_consensus.cc:2229] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.906488 4169 raft_consensus.cc:2258] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.908474 4169 ts_tablet_manager.cc:1905] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.909473 4163 tablet_bootstrap.cc:654] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:42.923552 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 6f3060822021431584a494e6941be756 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.924012 5593 raft_consensus.cc:482] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.924166 5593 raft_consensus.cc:504] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.924728 5593 leader_election.cc:288] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.927804 3824 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.931154 4169 ts_tablet_manager.cc:1918] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:42.931386 4169 log.cc:1196] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:42.932178 4163 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.932394 4163 ts_tablet_manager.cc:1397] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.000s
I20260811 18:57:42.933205 4163 raft_consensus.cc:348] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.933504 4163 raft_consensus.cc:374] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.933624 4163 raft_consensus.cc:729] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.933892 4163 consensus_queue.cc:260] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.934546 4163 ts_tablet_manager.cc:1428] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.934862 4163 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.936463 4163 tablet_bootstrap.cc:654] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.937786 4169 ts_tablet_manager.cc:1939] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.943583 5593 raft_consensus.cc:482] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.943756 5593 raft_consensus.cc:504] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.944429 5593 leader_election.cc:288] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.946077 3824 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.946743 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1216) took 3260 ms. Trace:
I20260811 18:57:42.946883 4169 rpcz_store.cc:276] 0811 18:57:39.686714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687161 (+ 447us) service_pool.cc:227] Handling call
0811 18:57:42.904585 (+3217424us) ts_tablet_manager.cc:1145] Deleting tablet e544b4b1592c4d51a2ed0d6461edc75e
0811 18:57:42.904595 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.922960 (+ 18365us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.931029 (+ 8069us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.946718 (+ 15689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3207455}
I20260811 18:57:42.947392 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]) successfully deleted
I20260811 18:57:42.947619 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1217) took 3260 ms. Trace:
I20260811 18:57:42.947744 4169 rpcz_store.cc:276] 0811 18:57:39.686834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687305 (+ 471us) service_pool.cc:227] Handling call
0811 18:57:42.947403 (+3260098us) ts_tablet_manager.cc:1145] Deleting tablet f9cef27540ff46aebc9b6f4760ae0db4
0811 18:57:42.947417 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.947599 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3250142}
W20260811 18:57:42.948213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f9cef27540ff46aebc9b6f4760ae0db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9cef27540ff46aebc9b6f4760ae0db4 already in progress: creating tablet
I20260811 18:57:42.948372 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1226) took 3148 ms. Trace:
I20260811 18:57:42.948493 4169 rpcz_store.cc:276] 0811 18:57:39.799988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.802413 (+ 2425us) service_pool.cc:227] Handling call
0811 18:57:42.948192 (+3145779us) ts_tablet_manager.cc:1145] Deleting tablet f389d2b7ff354ac4bab5d68b9ff201c6
0811 18:57:42.948202 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.948290 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3145567}
W20260811 18:57:42.948896 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f389d2b7ff354ac4bab5d68b9ff201c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f389d2b7ff354ac4bab5d68b9ff201c6 already in progress: creating tablet
I20260811 18:57:42.948949 4169 tablet_replica.cc:330] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.949268 4169 raft_consensus.cc:2229] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.949478 4169 raft_consensus.cc:2258] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.950413 4169 ts_tablet_manager.cc:1905] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.967990 4163 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.969422 4163 ts_tablet_manager.cc:1397] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.000s
I20260811 18:57:42.970286 4163 raft_consensus.cc:348] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.974908 4163 raft_consensus.cc:374] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.975052 4163 raft_consensus.cc:729] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.979511 4163 consensus_queue.cc:260] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.980389 4163 ts_tablet_manager.cc:1428] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:42.981034 4163 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.983562 4163 tablet_bootstrap.cc:654] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.984753 4169 ts_tablet_manager.cc:1918] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.986092 4169 log.cc:1196] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:42.986570 4169 ts_tablet_manager.cc:1939] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.990451 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.000068 4163 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.000396 4163 ts_tablet_manager.cc:1397] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.000s
I20260811 18:57:43.001586 4163 raft_consensus.cc:348] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.002015 4163 raft_consensus.cc:374] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.002153 4163 raft_consensus.cc:729] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.002660 4163 consensus_queue.cc:260] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.003930 4163 ts_tablet_manager.cc:1428] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:43.005800 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1228) took 2909 ms. Trace:
I20260811 18:57:43.005905 4169 rpcz_store.cc:276] 0811 18:57:40.096155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.096219 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:42.948901 (+2852682us) ts_tablet_manager.cc:1145] Deleting tablet cd1fc828f73642c88171d6ac827de90e
0811 18:57:42.948913 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.972182 (+ 23269us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.984642 (+ 12460us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.005779 (+ 21137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":384,"tablet-delete.queue_time_us":2852464}
I20260811 18:57:43.006543 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cd1fc828f73642c88171d6ac827de90e (table default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]) successfully deleted
I20260811 18:57:43.005860 4163 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.007359 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1229) took 2910 ms. Trace:
I20260811 18:57:43.007448 4169 rpcz_store.cc:276] 0811 18:57:40.096647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.096694 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:43.006302 (+2909608us) ts_tablet_manager.cc:1145] Deleting tablet b9fbac57c9c44a028f5b258593aff86d
0811 18:57:43.007209 (+ 907us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.007340 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2909449}
I20260811 18:57:43.007710 4169 tablet_replica.cc:330] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.007893 4169 raft_consensus.cc:2229] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.008033 4169 raft_consensus.cc:2258] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.008636 4169 ts_tablet_manager.cc:1905] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:43.010542 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b9fbac57c9c44a028f5b258593aff86d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fbac57c9c44a028f5b258593aff86d already in progress: creating tablet
I20260811 18:57:43.027263 4169 ts_tablet_manager.cc:1918] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.027853 4169 log.cc:1196] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:43.028923 4163 tablet_bootstrap.cc:654] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.029531 4169 ts_tablet_manager.cc:1939] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.031494 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1230) took 2934 ms. Trace:
W20260811 18:57:43.033903 3824 consensus_peers.cc:487] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:43.031594 4169 rpcz_store.cc:276] 0811 18:57:40.097032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097079 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:43.007672 (+2910593us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:43.007682 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.016671 (+ 8989us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.026774 (+ 10103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.031471 (+ 4697us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2910434}
I20260811 18:57:43.034417 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 55f97f37fa7c483cacaeed54f93e7a24 (table default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]) successfully deleted
I20260811 18:57:43.034509 4169 tablet_replica.cc:330] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.034823 4169 raft_consensus.cc:2229] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.035024 4169 raft_consensus.cc:2258] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.035640 4169 ts_tablet_manager.cc:1905] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.046277 4163 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.046677 5593 raft_consensus.cc:482] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.046759 4163 ts_tablet_manager.cc:1397] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.005s
I20260811 18:57:43.046856 5593 raft_consensus.cc:504] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.047561 5593 leader_election.cc:288] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:43.049000 3824 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.049409 4163 raft_consensus.cc:348] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.049733 4163 raft_consensus.cc:374] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.049858 4163 raft_consensus.cc:729] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.050163 4163 consensus_queue.cc:260] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:43.062868 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9af9fb059984c9680e78b68120b3726 (table default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9af9fb059984c9680e78b68120b3726 consensus state has no leader
I20260811 18:57:43.068943 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:43.070551 3824 consensus_peers.cc:487] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:43.072276 4163 ts_tablet_manager.cc:1428] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260811 18:57:43.072638 4163 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:43.084622 3361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3659: Timed out: CheckLeaks RPC to 127.3.72.65:32817 timed out after 1.000s (SENT)
I20260811 18:57:43.084980 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3659
I20260811 18:57:43.091089 4163 tablet_bootstrap.cc:654] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.091555 4169 ts_tablet_manager.cc:1918] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.091818 4169 log.cc:1196] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a847581412494e34b2c76e605ff11772
I20260811 18:57:43.092288 4169 ts_tablet_manager.cc:1939] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.093263 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1231) took 2995 ms. Trace:
I20260811 18:57:43.093441 4169 rpcz_store.cc:276] 0811 18:57:40.097414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097455 (+ 41us) service_pool.cc:227] Handling call
0811 18:57:43.034453 (+2936998us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:43.034467 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.038954 (+ 4487us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.091423 (+ 52469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.093239 (+ 1816us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2936851}
I20260811 18:57:43.094101 4169 tablet_replica.cc:330] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.094434 4169 raft_consensus.cc:2229] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.094470 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a847581412494e34b2c76e605ff11772 (table default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]) successfully deleted
I20260811 18:57:43.094694 4169 raft_consensus.cc:2258] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.095714 4169 ts_tablet_manager.cc:1905] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.105684 4169 ts_tablet_manager.cc:1918] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.105989 4169 log.cc:1196] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:43.106462 4169 ts_tablet_manager.cc:1939] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.107517 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1232) took 3009 ms. Trace:
I20260811 18:57:43.107720 4169 rpcz_store.cc:276] 0811 18:57:40.097800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097853 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:43.094039 (+2996186us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:43.094054 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.101056 (+ 7002us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.105221 (+ 4165us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.107489 (+ 2268us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2996020}
I20260811 18:57:43.108196 5593 raft_consensus.cc:482] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.108498 4169 tablet_replica.cc:330] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.108361 5593 raft_consensus.cc:504] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.108722 4169 raft_consensus.cc:2229] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.108912 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
I20260811 18:57:43.109133 5593 leader_election.cc:288] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:43.109623 4163 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.109776 4163 ts_tablet_manager.cc:1397] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.007s
I20260811 18:57:43.109889 4169 raft_consensus.cc:2258] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:43.110769 3824 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.110720 4163 raft_consensus.cc:348] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.110975 4163 raft_consensus.cc:374] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.111043 4163 raft_consensus.cc:729] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.111279 4163 consensus_queue.cc:260] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.111938 4163 ts_tablet_manager.cc:1428] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:43.112012 4169 ts_tablet_manager.cc:1905] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.112263 4163 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.114208 4163 tablet_bootstrap.cc:654] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.120330 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 330801031cc74eb2a5bc2caf53a18ead (table default.Table_04ac8209b46549e3bb3f12971ab01b73 [id=54b88a6b23fe45e8b0f235baf8f7234f]). Expected version 1 got 0
I20260811 18:57:43.126463 4169 ts_tablet_manager.cc:1918] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.127363 4169 log.cc:1196] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:43.128176 4169 ts_tablet_manager.cc:1939] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.129506 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1233) took 3031 ms. Trace:
I20260811 18:57:43.129696 4169 rpcz_store.cc:276] 0811 18:57:40.098470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.098514 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:43.108439 (+3009925us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:43.108453 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.115483 (+ 7030us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.126136 (+ 10653us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.129482 (+ 3346us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3009773}
I20260811 18:57:43.130249 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]) successfully deleted
I20260811 18:57:43.130463 4169 tablet_replica.cc:330] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.130833 4169 raft_consensus.cc:2229] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.131129 4169 raft_consensus.cc:2258] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.132099 4169 ts_tablet_manager.cc:1905] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.132956 5593 raft_consensus.cc:482] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.133085 5593 raft_consensus.cc:504] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.133713 5593 leader_election.cc:288] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:43.135356 3824 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.141155 4163 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.141400 4163 ts_tablet_manager.cc:1397] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.008s
I20260811 18:57:43.142392 4163 raft_consensus.cc:348] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.142674 4163 raft_consensus.cc:374] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.142781 4163 raft_consensus.cc:729] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.143028 4163 consensus_queue.cc:260] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.146592 4163 ts_tablet_manager.cc:1428] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:43.147070 4163 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.159472 4163 tablet_bootstrap.cc:654] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.162932 4169 ts_tablet_manager.cc:1918] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.163528 4169 log.cc:1196] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:43.163542 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.164104 4169 ts_tablet_manager.cc:1939] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.165089 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1234) took 3066 ms. Trace:
I20260811 18:57:43.165422 4169 rpcz_store.cc:276] 0811 18:57:40.099007 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.099051 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:43.130390 (+3031339us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:43.130402 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.139744 (+ 9342us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.162573 (+ 22829us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.165064 (+ 2491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3031076}
I20260811 18:57:43.165982 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]) successfully deleted
I20260811 18:57:43.166253 4169 tablet_replica.cc:330] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.166576 4169 raft_consensus.cc:2229] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.167277 4169 raft_consensus.cc:2258] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.171756 4169 ts_tablet_manager.cc:1905] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.179728 4163 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.180006 4163 ts_tablet_manager.cc:1397] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.000s
I20260811 18:57:43.181025 4163 raft_consensus.cc:348] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.181455 4163 raft_consensus.cc:374] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.181586 4163 raft_consensus.cc:729] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.181903 4163 consensus_queue.cc:260] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.182639 4163 ts_tablet_manager.cc:1428] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:43.187690 4163 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.190026 4163 tablet_bootstrap.cc:654] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.191359 4169 ts_tablet_manager.cc:1918] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.191579 4169 log.cc:1196] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:43.192031 4169 ts_tablet_manager.cc:1939] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.198904 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1235) took 3092 ms. Trace:
I20260811 18:57:43.199122 4169 rpcz_store.cc:276] 0811 18:57:40.106724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.106781 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:43.166173 (+3059392us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:43.166190 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.175659 (+ 9469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.191238 (+ 15579us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.198874 (+ 7636us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3036181}
I20260811 18:57:43.199688 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 27cda53b7ca54346a7a22bf26f5db990 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
I20260811 18:57:43.199995 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1236) took 3092 ms. Trace:
I20260811 18:57:43.200137 4169 rpcz_store.cc:276] 0811 18:57:40.107049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.107101 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:43.199770 (+3092669us) ts_tablet_manager.cc:1145] Deleting tablet d65cb8d901c04d8ba2b7a40efc7fa3af
0811 18:57:43.199787 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.199976 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3069819}
I20260811 18:57:43.200645 4169 tablet_replica.cc:330] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:43.200767 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d65cb8d901c04d8ba2b7a40efc7fa3af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65cb8d901c04d8ba2b7a40efc7fa3af already in progress: creating tablet
I20260811 18:57:43.201010 4169 raft_consensus.cc:2229] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.201275 4169 raft_consensus.cc:2258] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.201946 4169 ts_tablet_manager.cc:1905] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.207346 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.211628 4163 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.212024 4163 ts_tablet_manager.cc:1397] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.006s
I20260811 18:57:43.212898 4163 raft_consensus.cc:348] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.213218 4163 raft_consensus.cc:374] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.213335 4163 raft_consensus.cc:729] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.213631 4163 consensus_queue.cc:260] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.214422 4163 ts_tablet_manager.cc:1428] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:43.210983 4169 ts_tablet_manager.cc:1918] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.214744 4169 log.cc:1196] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:43.215229 4169 ts_tablet_manager.cc:1939] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:43.221938 3824 connection.cc:535] client connection to 127.3.72.65:32817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:43.222505 3542 connection.cc:535] server connection from 127.3.72.65:60889 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:43.222934 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3804
I20260811 18:57:43.223541 4163 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.224720 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1237) took 3099 ms. Trace:
I20260811 18:57:43.224905 4169 rpcz_store.cc:276] 0811 18:57:40.125332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.125391 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:43.200598 (+3075207us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:43.200609 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.205825 (+ 5216us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.210870 (+ 5045us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.224680 (+ 13810us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3070407}
I20260811 18:57:43.225520 4163 tablet_bootstrap.cc:654] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.225628 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1239) took 2949 ms. Trace:
I20260811 18:57:43.225713 4169 rpcz_store.cc:276] 0811 18:57:40.275867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.278408 (+ 2541us) service_pool.cc:227] Handling call
0811 18:57:43.225491 (+2947083us) ts_tablet_manager.cc:1145] Deleting tablet fdab1c3c431c4dceb08b00eabcfa1ceb
0811 18:57:43.225504 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.225611 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2946801}
I20260811 18:57:43.228143 3933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:43.228480 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 452ee450778e4d33adfa7e2745b5d970 (table default.Table_4d6c4d770dc949e78c1707ddbab71740 [id=48f45fcc06e0418185841260a06cb98c]) successfully deleted
W20260811 18:57:43.228825 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fdab1c3c431c4dceb08b00eabcfa1ceb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdab1c3c431c4dceb08b00eabcfa1ceb already in progress: creating tablet
W20260811 18:57:43.235298 3541 connection.cc:535] client connection to 127.3.72.65:32817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:43.724730 5593 raft_consensus.cc:482] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.724920 5593 raft_consensus.cc:504] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:43.725584 5593 leader_election.cc:288] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:43.726123 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1244) took 3288 ms. Trace:
I20260811 18:57:43.726257 4169 rpcz_store.cc:276] 0811 18:57:40.437347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.437405 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:43.226063 (+2788658us) ts_tablet_manager.cc:1145] Deleting tablet 0e556811ad1643f1baa3bbf4edebd157
0811 18:57:43.725933 (+499870us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.726097 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2788425}
I20260811 18:57:43.726758 4169 tablet_replica.cc:330] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.727061 4169 raft_consensus.cc:2229] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.727263 4169 raft_consensus.cc:2258] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.728065 4169 ts_tablet_manager.cc:1905] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:43.730141 3824 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.001s sys 0.009s
W20260811 18:57:43.730281 3824 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.503s user 0.001s sys 0.009s
W20260811 18:57:43.730567 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.730692 3824 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.730793 3824 leader_election.cc:302] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.731070 5593 raft_consensus.cc:2726] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.731115 3824 consensus_peers.cc:487] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731206 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731276 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731343 3824 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.731413 3824 leader_election.cc:302] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.731595 5593 raft_consensus.cc:2726] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.731643 3824 consensus_peers.cc:487] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731721 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731787 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731886 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731966 3824 consensus_peers.cc:487] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.732036 3824 consensus_peers.cc:487] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.732106 3824 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.732183 3824 leader_election.cc:302] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.732366 5593 raft_consensus.cc:2726] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.732414 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.732501 3824 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.732636 3824 raft_consensus.cc:2670] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:43.733222 3824 consensus_peers.cc:487] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.733237 4163 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.020s sys 0.007s
W20260811 18:57:43.733376 4163 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.507s user 0.020s sys 0.007s
I20260811 18:57:43.734635 4163 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.734838 4163 ts_tablet_manager.cc:1397] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.511s user 0.023s sys 0.007s
I20260811 18:57:43.735942 4163 raft_consensus.cc:348] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.736275 4163 raft_consensus.cc:374] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.736338 4163 raft_consensus.cc:729] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.736579 4163 consensus_queue.cc:260] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.737351 4163 ts_tablet_manager.cc:1428] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260811 18:57:43.733379 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.742627 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.742766 3824 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.742944 3824 raft_consensus.cc:2670] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:43.743705 3824 consensus_peers.cc:487] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.743848 3824 consensus_peers.cc:487] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744099 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744216 3824 consensus_peers.cc:487] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744303 3824 consensus_peers.cc:487] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744388 3824 consensus_peers.cc:487] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744448 3824 consensus_peers.cc:487] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744510 3824 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.744587 3824 leader_election.cc:302] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.744923 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.745044 3824 consensus_peers.cc:487] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.745137 3824 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.745231 3824 leader_election.cc:302] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.744357 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet c575ba3060a140759484bea1462ffd73: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.745469 5593 raft_consensus.cc:2726] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:43.745643 5593 raft_consensus.cc:2726] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.746042 3824 consensus_peers.cc:487] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.746193 3824 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.746328 3824 leader_election.cc:302] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.746621 5593 raft_consensus.cc:2726] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.746941 3824 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.747088 3824 leader_election.cc:302] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.747385 5593 raft_consensus.cc:2726] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.747711 3824 consensus_peers.cc:487] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.747870 3824 consensus_peers.cc:487] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.748006 3824 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.748140 3824 leader_election.cc:302] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.748417 5593 raft_consensus.cc:2726] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.748852 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.749027 3824 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.749162 3824 leader_election.cc:302] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.749434 5593 raft_consensus.cc:2726] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:43.750715 4163 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.752833 4163 tablet_bootstrap.cc:654] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:43.759809 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0e556811ad1643f1baa3bbf4edebd157 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e556811ad1643f1baa3bbf4edebd157 already in progress: creating tablet
I20260811 18:57:43.760123 4163 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.760391 4163 ts_tablet_manager.cc:1397] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260811 18:57:43.760501 5593 raft_consensus.cc:482] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.760635 5593 raft_consensus.cc:504] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.761289 5593 leader_election.cc:288] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:43.761168 4163 raft_consensus.cc:348] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.761375 4163 raft_consensus.cc:374] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.761533 4163 raft_consensus.cc:729] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.761756 4163 consensus_queue.cc:260] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.762480 4163 ts_tablet_manager.cc:1428] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:43.762794 4163 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.763890 4169 ts_tablet_manager.cc:1918] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.765357 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3804
I20260811 18:57:43.846863 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 4863
I20260811 18:57:43.861989 5321 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:43.867790 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet b3f0cdcee681460084662e01152ad9dd on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:43.869256 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 9ebe3fed5aaf45918dbadfde06b2c24e on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:44.073136 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 838a9f3025a1466296844c218cedf4fa (table default.Table_9049c4cdc3654b26a7b3d0cad713ec01 [id=0052bf55538f46c5b17831eaa1cfad91], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 838a9f3025a1466296844c218cedf4fa consensus state has no leader
I20260811 18:57:44.082556 5852 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:44.082778 5852 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:44.083618 5852 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:44.086061 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 1658 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:44.092118 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 4863
W20260811 18:57:44.131071 5853 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:44.084008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.0.0.1:36432)
0811 18:57:44.085122 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.085143 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:44.085301 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:44.091709 (+ 6408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:44.091717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:44.091762 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:44.092137 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:44.092155 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.130757 (+ 38602us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":305,"threads_started":1}
I20260811 18:57:44.131776 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 4230
W20260811 18:57:44.132460 4381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:44.139266 4381 leader_election.cc:334] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111)
I20260811 18:57:44.139472 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:44.139828 5852 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:44.141705 4466 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:44.168576 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:44.358069 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 3458 similar messages]
I20260811 18:57:44.361200 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 4230
I20260811 18:57:44.403297 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3522
I20260811 18:57:44.404374 3588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:44.694159 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3522
2026-08-11T18:57:44Z chronyd exiting
I20260811 18:57:44.754619 3361 master-stress-test.cc:586] Tables created: 739
I20260811 18:57:44.754757 3361 master-stress-test.cc:587] Tables altered: 374
I20260811 18:57:44.754853 3361 master-stress-test.cc:588] Tables deleted: 314
I20260811 18:57:44.754935 3361 master-stress-test.cc:589] Tablets replaced: 93
I20260811 18:57:44.754994 3361 master-stress-test.cc:590] Masters restarted: 2
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (47645 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260811 18:57:44.910611 3361 test_util.cc:268] Using random seed: 198008552
2026-08-11T18:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:44Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
2026-08-11 18:57:52: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260811 18:58:01.568192 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--webserver_interface=127.3.72.126
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:01.938983 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:01.939337 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:01.939471 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:01.947355 6106 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:01.947487 6106 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:01.947562 6106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:01.947654 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:01.947737 6106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:01.947861 6106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:01.957542 6106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:01.959156 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:01.961040 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:01.974146 6112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.974092 6113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.974187 6115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:01.974994 6106 server_base.cc:1034] running on GCE node
I20260811 18:58:01.975903 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:01.977639 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:01.978889 6106 hybrid_clock.cc:648] HybridClock initialized: now 1786474681978822 us; error 40 us; skew 500 ppm
I20260811 18:58:01.979322 6106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:01.982731 6106 webserver.cc:460] Webserver started at http://127.3.72.126:35183/ using document root <none> and password file <none>
I20260811 18:58:01.983623 6106 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:01.983834 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:01.984153 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:01.987525 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/instance:
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:01.988421 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal/instance:
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:01.995203 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 18:58:01.999697 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:02.001502 6106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:58:02.001768 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:02.002023 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:02.031759 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:02.032811 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:02.033208 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:02.055088 6106 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:37881
I20260811 18:58:02.055253 6175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:37881 every 8 connection(s)
I20260811 18:58:02.056700 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:02.060891 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:02.065835 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6106
I20260811 18:58:02.066092 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal/instance
I20260811 18:58:02.067395 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:02.068097 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.090065 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.091616 6124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:02.093585 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.143042 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 1
W20260811 18:58:02.145465 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.213821 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 2
W20260811 18:58:02.216202 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.307535 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 3
W20260811 18:58:02.309980 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:02.443028 6179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:02.443362 6179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:02.443445 6179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:02.451092 6179 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:02.451210 6179 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:02.451259 6179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:02.451314 6179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:02.451354 6179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:02.451423 6179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:02.453330 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 4
W20260811 18:58:02.455569 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.461638 6179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:02.463327 6179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:02.465353 6179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:02.478598 6185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.478693 6189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.478745 6187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:02.479851 6179 server_base.cc:1034] running on GCE node
I20260811 18:58:02.480773 6179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:02.482501 6179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:02.483752 6179 hybrid_clock.cc:648] HybridClock initialized: now 1786474682483659 us; error 65 us; skew 500 ppm
I20260811 18:58:02.484192 6179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:02.486918 6179 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:02.487686 6179 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:02.487848 6179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:02.488181 6179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:02.491107 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/instance:
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.491935 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal/instance:
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.498582 6179 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:58:02.502723 6195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:02.504954 6179 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:02.505235 6179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.505491 6179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:02.538228 6179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:02.539300 6179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:02.539727 6179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:02.562268 6179 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:02.562305 6247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:02.564153 6179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:02.568539 6248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:02.568562 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6179
I20260811 18:58:02.568749 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal/instance
I20260811 18:58:02.570181 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:02.578858 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.599882 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.612174 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.613720 6199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:02.615769 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.665220 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 1
W20260811 18:58:02.667695 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.735991 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 2
W20260811 18:58:02.738379 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.754913 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 5
I20260811 18:58:02.762907 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.765645 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.816972 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 1
W20260811 18:58:02.819237 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.829668 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 3
W20260811 18:58:02.832217 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.887493 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 2
W20260811 18:58:02.890017 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:02.958689 6250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:02.959009 6250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:02.959162 6250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:02.967096 6250 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:02.967227 6250 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:02.967272 6250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:02.967430 6250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:02.967509 6250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:02.967624 6250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:02.975507 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 4
W20260811 18:58:02.977712 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.978235 6250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:02.979725 6250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:02.981736 6250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:02.995368 6257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.995453 6258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.995489 6260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:02.996253 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 3
I20260811 18:58:02.996848 6250 server_base.cc:1034] running on GCE node
I20260811 18:58:02.997927 6250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260811 18:58:02.998405 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.999922 6250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:03.001207 6250 hybrid_clock.cc:648] HybridClock initialized: now 1786474683001124 us; error 59 us; skew 500 ppm
I20260811 18:58:03.001639 6250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:03.004542 6250 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:03.005622 6250 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:03.005828 6250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:03.006227 6250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:03.009902 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/instance:
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.011132 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal/instance:
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.017896 6250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:58:03.022256 6266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:03.024258 6250 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:03.024523 6250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.024765 6250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:03.041419 6250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:03.042455 6250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:03.042788 6250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:03.064268 6250 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:03.064324 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:03.065874 6250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:03.069981 6320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:03.074209 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6250
I20260811 18:58:03.074453 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal/instance
I20260811 18:58:03.077189 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.094305 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.103158 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.115254 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:03.118644 6320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.119705 6320 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:03.122584 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: No bootstrap required, opened a new log
I20260811 18:58:03.130726 6320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.131301 6320 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.131434 6320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:03.132228 6320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.133389 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.133791 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.134450 6320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.155464 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.155821 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.148320 6250 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: OK, instance_id { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" instance_seqno: 1786474682048794 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:58:03.161396 6250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:03.167558 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.167714 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.175688 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 4
W20260811 18:58:03.179178 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.179359 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.190821 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.190971 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.191766 6176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap starting.
I20260811 18:58:03.197028 6176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.198691 6176 log.cc:824] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Log is configured to *not* fsync() on all Append() calls
W20260811 18:58:03.202418 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.202561 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.212575 6176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: No bootstrap required, opened a new log
W20260811 18:58:03.214017 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.214164 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.224887 6176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.225368 6176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.225517 6176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Initialized, Role: FOLLOWER
W20260811 18:58:03.225803 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.225946 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.226498 6176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.230253 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.230830 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.231765 6176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.239887 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.240063 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.252086 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.252314 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.264050 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.264216 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.275744 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.275913 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.286407 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 5
W20260811 18:58:03.287503 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.287664 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.295308 6272 tablet.cc:2369] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:58:03.298394 6127 tablet.cc:2369] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:03.276685 6106 master_runner.cc:186] Error getting master registration for 127.3.72.125:34867: OK, instance_id { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" instance_seqno: 1786474682555705 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:58:03.302487 6106 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:03.306535 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.306689 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.308250 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
W20260811 18:58:03.308290 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.308550 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.312841 6248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.314385 6248 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:03.317879 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: No bootstrap required, opened a new log
W20260811 18:58:03.320434 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.320561 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.318280 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.324573 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.329460 6248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.329944 6248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.330077 6248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:03.330844 6248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:03.332108 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.332254 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.336143 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.336292 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.337997 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.338666 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.340508 6248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.343784 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.343955 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.346262 6365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:03.346625 6365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.348320 6365 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:03.348632 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.348757 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.349501 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:03.350178 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 0.
I20260811 18:58:03.350666 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:03.350960 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.350994 6198 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
I20260811 18:58:03.351621 6365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:03.351780 6365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:03.351879 6365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.352609 6326 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:03.353727 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:03.353991 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 0.
I20260811 18:58:03.354127 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:03.354620 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:03.355113 6326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:03.355266 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:03.355356 6326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.355209 6365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:03.356580 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:58:03.356611 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:03.356850 6149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.357548 6222 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 1: Already voted for candidate 6bb6ba710ee24c348d653d524bb73f3d in this term.
I20260811 18:58:03.358727 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:03.359990 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.360244 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 1.
I20260811 18:58:03.360545 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d in current term 1: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:03.361023 6198 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
W20260811 18:58:03.356719 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.364490 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:58:03.364470 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
W20260811 18:58:03.364629 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.364755 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d in current term 1: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:03.365535 6365 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:03.365514 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:03.365899 6149 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 1: Already voted for candidate 6bb6ba710ee24c348d653d524bb73f3d in this term.
I20260811 18:58:03.366180 6365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:03.366554 6365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:03.366891 6326 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:03.367218 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:03.367569 6365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.367758 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:03.368147 6326 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260811 18:58:03.372969 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.373145 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.374848 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.375176 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
W20260811 18:58:03.376204 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.376346 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.381788 6374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 18:58:03.384543 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.384688 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.388159 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.388345 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.396361 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.396529 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.399775 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.399932 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.400394 6374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 18:58:03.408108 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.408280 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.411337 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.411481 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.419849 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.420013 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.423084 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.423236 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.426453 6149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:03.426468 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:03.427423 6365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:03.427778 6370 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:58:03.433526 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.433693 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.461417 6379 mvcc.cc:205] Tried to move back new op lower bound from 7317400303302197248 to 7317400303079038976. Current Snapshot: MvccSnapshot[applied={T|T < 7317400303302197248}]
W20260811 18:58:03.461699 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.461846 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.462613 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.463198 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.481297 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.481676 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.481647 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.482038 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:03.483206 6374 catalog_manager.cc:1347] Generated new cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.483326 6374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 18:58:03.483551 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.483695 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.493520 6370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.493944 6370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
W20260811 18:58:03.500339 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.500505 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.500838 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.501130 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.503328 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.503616 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.504433 6342 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.504516 6342 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
W20260811 18:58:03.505684 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.505812 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.512491 6362 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.512616 6362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: success
W20260811 18:58:03.514058 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.514245 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.517382 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.517516 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.526024 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.526213 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.529563 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.529743 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.538471 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.538616 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.541319 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.541466 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.543642 6374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:03.545537 6374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:03.553316 6362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: success
W20260811 18:58:03.554896 6362 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.555775 6342 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
W20260811 18:58:03.557116 6342 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.566864 6362 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.568836 6342 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.580370 6362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:03.580958 6374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Generated new TSK 0
I20260811 18:58:03.581712 6342 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:03.581945 6374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:03.616145 6395 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:58:03.624933 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.65:0
--local_ip_for_outbound_sockets=127.3.72.65
--webserver_interface=127.3.72.65
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:04.075670 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.075896 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.076956 6397 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:04.076817 6398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.077275 6398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.077646 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:04.077837 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.077983 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 1.
I20260811 18:58:04.078555 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.078876 6398 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.079351 6397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:58:04.079520 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.079689 6397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.079835 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.080456 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:04.082720 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.083144 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a1e448ebc1343319544bc81ea89fed8 in current term 2: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:04.083436 6397 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:04.084148 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8; no voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:04.084568 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:04.085177 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:04.085345 6398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.085810 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:04.086284 6397 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:58:04.088147 6398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:58:04.152276 6396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:04.152598 6396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:04.152763 6396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:04.160766 6396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:04.161006 6396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.65
I20260811 18:58:04.170941 6396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.72.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.65
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:04.172415 6396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:04.174319 6396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:04.189867 6403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.190303 6404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.190333 6406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.190863 6396 server_base.cc:1034] running on GCE node
I20260811 18:58:04.191651 6396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:04.192638 6396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:04.193976 6396 hybrid_clock.cc:648] HybridClock initialized: now 1786474684193801 us; error 120 us; skew 500 ppm
I20260811 18:58:04.194388 6396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:04.195449 6294 raft_consensus.cc:1226] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Rejecting Update request from peer 6bb6ba710ee24c348d653d524bb73f3d for earlier term 1. Current term is 2. Ops: []
I20260811 18:58:04.196235 6365 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:04.196986 6409 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260811 18:58:04.197088 6409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:04.197415 6409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.197858 6409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.203440 6395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:04.204276 6396 webserver.cc:460] Webserver started at http://127.3.72.65:40485/ using document root <none> and password file <none>
I20260811 18:58:04.205212 6396 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:04.205386 6396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:04.205739 6396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:04.208770 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/instance:
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.209607 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance:
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.216593 6396 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 18:58:04.221040 6414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.222846 6396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 18:58:04.223097 6396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.223357 6396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:04.251508 6396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:04.252494 6396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:04.252940 6396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:04.254104 6396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:04.255908 6396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:04.255986 6396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.256112 6396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:04.256160 6396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.297106 6396 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.65:38675
I20260811 18:58:04.297222 6530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.65:38675 every 8 connection(s)
I20260811 18:58:04.298816 6396 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:04.302698 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6396
I20260811 18:58:04.302912 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:04.323276 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:04.323662 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.324679 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:04.324738 6531 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:04.324859 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.325306 6533 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:04.325724 6532 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:04.325878 6532 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.326442 6532 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:04.327411 6284 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.327387 6212 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.328362 6138 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.629313 6541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:04.629539 6541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.630412 6541 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.631090 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:04.631100 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.631390 6222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 2.
I20260811 18:58:04.631394 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 2.
I20260811 18:58:04.632018 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.632376 6541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 18:58:04.632581 6541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:04.632707 6541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.635377 6541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.636231 6541 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.636873 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:04.637182 6222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.637126 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:04.637377 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.640111 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 3.
I20260811 18:58:04.640110 6222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 3.
I20260811 18:58:04.640807 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.641419 6541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:58:04.642122 6541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:04.642637 6541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.644490 6542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:04.644927 6542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:04.645609 6544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:04.648196 6544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:04.649066 6544 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:04.649156 6544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:04.650579 6544 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:04.651449 6544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:04.652603 6544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:04.653432 6544 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:04.654218 6544 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:04.672048 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.66:0
--local_ip_for_outbound_sockets=127.3.72.66
--webserver_interface=127.3.72.66
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:04.832711 6138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.65:37527
W20260811 18:58:05.070482 6545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:05.070899 6545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:05.071051 6545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:05.082237 6545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:05.082494 6545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.66
I20260811 18:58:05.093065 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 18:58:05.093334 6545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.72.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.66
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:05.093919 6542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:05.095343 6545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:05.096531 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.096853 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.097856 6545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:05.099109 6541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.099530 6541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.101063 6222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 18:58:05.101996 6553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:05.110949 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.111195 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.112640 6559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.112999 6559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.113138 6554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.113559 6554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:58:05.121903 6558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.122136 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.122162 6562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.123528 6545 server_base.cc:1034] running on GCE node
I20260811 18:58:05.124293 6545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:05.125324 6545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:05.126600 6545 hybrid_clock.cc:648] HybridClock initialized: now 1786474685126517 us; error 58 us; skew 500 ppm
I20260811 18:58:05.126960 6545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:05.130064 6545 webserver.cc:460] Webserver started at http://127.3.72.66:37167/ using document root <none> and password file <none>
I20260811 18:58:05.130960 6545 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:05.131127 6545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:05.131477 6545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:05.134922 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/instance:
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.135954 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance:
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.142940 6545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:58:05.147426 6568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.149206 6545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:05.149436 6545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.149682 6545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:05.164919 6545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:05.165896 6545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:05.166220 6545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:05.167380 6545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:05.169142 6545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:05.169252 6545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.169420 6545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:05.169493 6545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.208971 6545 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.66:44849
I20260811 18:58:05.209022 6683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.66:44849 every 8 connection(s)
I20260811 18:58:05.210526 6545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:05.213084 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6545
I20260811 18:58:05.213276 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:05.217947 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.67:0
--local_ip_for_outbound_sockets=127.3.72.67
--webserver_interface=127.3.72.67
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:05.236925 6685 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:05.237025 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:05.237285 6685 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.237280 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.238565 6684 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:05.239849 6685 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:05.240993 6138 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.241071 6284 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.242309 6138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.66:52437
I20260811 18:58:05.246439 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:05.246670 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.247215 6686 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:05.248554 6212 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.336058 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:05.599345 6549 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.599577 6549 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.600513 6549 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:05.601207 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:05.601255 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:05.601982 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:05.602392 6549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 18:58:05.605434 6554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.605645 6554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.606510 6554 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:05.607038 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:05.607093 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:05.607353 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 3.
I20260811 18:58:05.607874 6199 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:05.608189 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 18:58:05.608357 6554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.608490 6554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Advancing to term 4
I20260811 18:58:05.610916 6554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.611814 6554 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:05.612473 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:05.612941 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:05.613283 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Advancing to term 4
I20260811 18:58:05.616600 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 4.
I20260811 18:58:05.617239 6199 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:05.617758 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Leader election won for term 4
I20260811 18:58:05.618238 6554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:05.618589 6554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.619884 6698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 4 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.620316 6698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.621012 6699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:05.623153 6699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:05.624259 6699 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:05.624351 6699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 18:58:05.625135 6694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:05.625599 6694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:05.625780 6694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:58:05.626104 6699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:05.627096 6699 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:05.627674 6700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.628319 6700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.628494 6222 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 LEADER]: Rejecting Update request from peer 1a1e448ebc1343319544bc81ea89fed8 for earlier term 3. Current term is 4. Ops: []
I20260811 18:58:05.628851 6699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:05.629923 6699 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:05.631086 6699 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:05.631660 6700 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:58:05.632031 6700 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer 6bb6ba710ee24c348d653d524bb73f3d log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260811 18:58:05.632267 6703 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 18:58:05.632383 6703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:05.632742 6703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.633384 6703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Advancing to term 4
W20260811 18:58:05.637724 6694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:05.637918 6694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.67
I20260811 18:58:05.648341 6694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.72.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.67
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:05.649945 6694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:05.652045 6694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:05.667347 6708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.667470 6706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.667510 6705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.669092 6694 server_base.cc:1034] running on GCE node
I20260811 18:58:05.669891 6694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:05.671204 6694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:05.672538 6694 hybrid_clock.cc:648] HybridClock initialized: now 1786474685672487 us; error 44 us; skew 500 ppm
I20260811 18:58:05.672925 6694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:05.675845 6694 webserver.cc:460] Webserver started at http://127.3.72.67:38631/ using document root <none> and password file <none>
I20260811 18:58:05.676644 6694 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:05.676826 6694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:05.677179 6694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:05.680289 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/instance:
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.681278 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance:
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.687852 6694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 18:58:05.692037 6714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.693789 6694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:05.694020 6694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.694263 6694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:05.718532 6694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:05.719705 6694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:05.720062 6694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:05.721220 6694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:05.723193 6694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:05.723282 6694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.723416 6694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:05.723486 6694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.745836 6685 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:05.764787 6694 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.67:33471
I20260811 18:58:05.764824 6842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.67:33471 every 8 connection(s)
I20260811 18:58:05.766248 6694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:05.767588 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6694
I20260811 18:58:05.767831 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:05.791746 6844 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:05.792080 6844 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.793246 6844 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:05.795297 6139 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.797154 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:05.797186 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:05.797386 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.797390 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.797981 6843 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:05.798122 6846 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:05.799306 6212 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.799443 6284 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.800721 6212 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.67:39425
I20260811 18:58:05.813747 3361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:58:05.847095 6212 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.842221 6211 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.846885 6209 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.843432 6208 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.842561 6210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.850971 6205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.852422 6203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.844342 6207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.851495 6204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_1743b9268cc34cdd95330f6add35aa52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.849637 6206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:05.870471 6212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_91ae855fc2ac45b7bca9c6f666775f82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:05.890920 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 6179
I20260811 18:58:05.892259 6242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:06.083236 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 6179
I20260811 18:58:06.083429 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 18:58:06.083953 6149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260811 18:58:06.116391 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:06.120529 6270 connection.cc:535] server connection from 127.0.0.1:42342 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.121598 6123 connection.cc:535] server connection from 127.0.0.1:58904 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.124042 6861 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:06.118178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:34867 (local address 127.0.0.1:50718)
0811 18:58:06.119162 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:06.119186 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:06.119366 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:06.121880 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":580}
I20260811 18:58:06.181262 6899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:06.181821 6899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:06.183612 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:06.183899 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 4.
I20260811 18:58:06.184605 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:06.188113 6899 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:06.189275 6899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 18:58:06.189590 6899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:06.189869 6899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Advancing to term 5
I20260811 18:58:06.193252 6899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:06.196925 6899 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:06.201397 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:06.201630 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Advancing to term 5
I20260811 18:58:06.204596 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 5.
I20260811 18:58:06.205305 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:06.205773 6901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Leader election won for term 5
I20260811 18:58:06.206151 6901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:06.206676 6901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:06.223069 6124 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:06.224440 6124 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.227881 6899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.228170 6899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:06.240398 6904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:06.243229 6904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:06.244462 6904 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:06.244576 6904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:06.246245 6904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:06.247517 6904 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:06.248157 6904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:06.249228 6904 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:06.250317 6904 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:06.252504 6686 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.254528 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.265173 6138 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_1743b9268cc34cdd95330f6add35aa52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.266037 6137 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.265826 6139 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.272629 6135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.274897 6134 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275521 6130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275686 6131 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.273015 6132 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275774 6136 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.273391 6129 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:06.278467 6138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1743b9268cc34cdd95330f6add35aa52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:58:06.285275 6137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fb574299b4df4b758235b87988424ca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:06.303521 6846 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.305369 6846 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.335906 6533 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.348979 6533 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.445300 6138 catalog_manager.cc:2148] created HMS catalog entry for table [id=d5fd615c99fb4b41b85da755d34202ed]
F20260811 18:58:06.452348 6360 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7efe8762d9ad google::LogMessage::Fail() at ??:0
I20260811 18:58:06.457188 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260811 18:58:06.458030 6899 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
@ 0x7efe8763088a google::LogMessage::SendToLog() at ??:0
@ 0x7efe8762d4de google::LogMessage::Flush() at ??:0
@ 0x7efe8762d6d1 google::LogMessage::~LogMessage() at ??:0
W20260811 18:58:06.465929 6124 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 -> Peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Couldn't send request to peer 6bb6ba710ee24c348d653d524bb73f3d. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7efe9d1cbba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7efe9d1ca901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7efe9d1d166c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260811 18:58:06.499500 6910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.499835 6910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:06.502682 6910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.502983 6910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7efe9d1d14e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efe8a257eaf kudu::Thread::SuperviseThread() at ??:0
I20260811 18:58:06.524391 6138 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58932:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:06.526258 6899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
@ 0x7efe8d5316db start_thread at ??:0
I20260811 18:58:06.526611 6899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:06.532058 6137 catalog_manager.cc:2148] created HMS catalog entry for table [id=c920f925a4074d738b2e2720429bf7e0]
@ 0x7efe864c761f clone at ??:0
W20260811 18:58:06.767418 6857 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.767719 6571 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.768486 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.126:37881)
I20260811 18:58:06.991760 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:07.025858 6844 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:07.025919 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:07.032001 6933 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.032264 6933 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.033022 6933 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:07.034410 6269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:07.036360 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:07.036664 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:07.036819 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:07.037159 6933 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260811 18:58:07.436733 6932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:07.437152 6932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:07.437330 6932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:07.445037 6932 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:07.445195 6932 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:07.445305 6932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:07.445415 6932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:07.445498 6932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:07.445595 6932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:07.455763 6932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:07.457404 6932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:07.459378 6932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:07.474108 6944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:07.474210 6946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:07.475467 6932 server_base.cc:1034] running on GCE node
W20260811 18:58:07.482851 6943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:07.484447 6932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:07.486219 6932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:07.487516 6932 hybrid_clock.cc:648] HybridClock initialized: now 1786474687487423 us; error 68 us; skew 500 ppm
I20260811 18:58:07.487929 6932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:07.491015 6932 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:07.491772 6932 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:07.491930 6932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:07.498662 6932 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:07.502532 6952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:07.504266 6932 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:07.504491 6932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:07.505637 6932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:07.558557 6932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:07.559609 6932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:07.559949 6932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:07.581235 6932 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:07.581254 7007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:07.583498 6932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:07.589978 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6932
I20260811 18:58:07.591367 7009 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:07.596053 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:07.605298 7009 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:07.616009 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:07.616842 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:07.625736 7009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.627393 7009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:07.628142 7009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.629316 7013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:07.629828 7013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:07.630640 7009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:07.645895 7028 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:07.646130 7028 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:07.650799 7028 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:07.643827 6932 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:07.655301 6932 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:07.655519 7028 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:07.731040 7032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.731303 7032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.731987 7032 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:07.736616 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:07.743002 6980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:07.743654 6980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 4.
I20260811 18:58:07.744334 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:07.744707 7032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260811 18:58:07.744866 7032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.744978 7032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Advancing to term 6
I20260811 18:58:07.748433 7032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.749297 7032 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:07.750452 6980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:07.750739 6980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Advancing to term 6
W20260811 18:58:07.751484 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:07.758396 6980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 6.
I20260811 18:58:07.759606 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:07.759996 7032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Leader election won for term 6
I20260811 18:58:07.761157 7032 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 LEADER]: Becoming Leader. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Running, Role: LEADER
I20260811 18:58:07.761770 7032 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.764878 7036 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19621f5b4944a18b25b0ed2e383027f. Latest consensus state: current_term: 6 leader_uuid: "e19621f5b4944a18b25b0ed2e383027f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:07.765311 7036 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:07.770195 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.770494 6686 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
I20260811 18:58:07.774204 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:37365; Asking this server to re-register.
I20260811 18:58:07.775478 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:07.777623 6970 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:07.779181 7039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:07.782810 7039 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:07.785185 7039 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:07.785574 7039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:07.786830 7039 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:07.786931 7039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:07.788800 7039 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:07.790285 7039 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Loaded TSK: 0
I20260811 18:58:07.791868 7039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:07.793068 7039 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:07.794191 7039 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:07.812455 6284 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42346:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:07.812943 6281 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.813508 6280 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.814002 6279 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816380 6277 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816943 6274 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816982 6275 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.818698 6276 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.819223 6278 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.819912 6282 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:07.823487 6341 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
I20260811 18:58:07.825162 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.825490 6846 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
@ 0x7fd4fdd059ad google::LogMessage::Fail() at ??:0
I20260811 18:58:07.826812 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:43675; Asking this server to re-register.
I20260811 18:58:07.827562 6846 heartbeater.cc:464] Registering TS with master...
@ 0x7fd4fdd0888a google::LogMessage::SendToLog() at ??:0
I20260811 18:58:07.829409 6970 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
@ 0x7fd4fdd054de google::LogMessage::Flush() at ??:0
@ 0x7fd4fdd056d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fd5138a3ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fd5138a2901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fd5138a966c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fd5138a94e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd50092feaf kudu::Thread::SuperviseThread() at ??:0
I20260811 18:58:07.868878 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.869145 6533 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
@ 0x7fd503c096db start_thread at ??:0
I20260811 18:58:07.870368 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:56579; Asking this server to re-register.
I20260811 18:58:07.871397 6533 heartbeater.cc:464] Registering TS with master...
@ 0x7fd4fcb9f61f clone at ??:0
I20260811 18:58:07.872831 6970 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
W20260811 18:58:08.079995 6858 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.124:35445)
W20260811 18:58:08.080804 6858 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.124:35445)
I20260811 18:58:08.311600 6843 heartbeater.cc:503] Master 127.3.72.124:35445 was elected leader, sending a full tablet report...
I20260811 18:58:08.384094 7052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:08.384392 7052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:08.386996 7052 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:08.388090 6956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:08.389807 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:08.390092 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:08.390305 6956 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:08.390813 7052 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:09.002782 7057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:09.003034 7057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:09.004098 7057 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:09.006596 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:09.006817 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:09.006976 6957 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:09.007301 7057 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:09.627568 7061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:09.627797 7061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:09.629622 7061 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:09.631485 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:09.631703 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:09.631840 6956 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:09.632117 7061 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:10.122273 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 6932
I20260811 18:58:10.137323 7003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:10.257462 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 6932
I20260811 18:58:11.151294 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:11.534826 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:11.535144 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:11.535224 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:11.544495 7069 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:11.544624 7069 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:11.544667 7069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:11.544729 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:11.544770 7069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:11.544829 7069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:11.554808 7069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:11.556344 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:11.558717 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:11.572418 7075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:11.572438 7074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:11.572556 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:11.573261 7069 server_base.cc:1034] running on GCE node
I20260811 18:58:11.574139 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:11.575948 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:11.577287 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786474691577158 us; error 80 us; skew 500 ppm
I20260811 18:58:11.577783 7069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:11.580549 7069 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:11.581336 7069 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:11.581496 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:11.587618 7069 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:11.591821 7083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:11.593386 7069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:58:11.593585 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:11.594764 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:11.623047 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:11.624017 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:11.624361 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:11.647418 7069 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:11.647437 7137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:11.649135 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:11.650437 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7069
I20260811 18:58:11.658238 7138 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:11.663384 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:11.672245 7138 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:11.682250 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:11.683076 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:11.692831 7138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:11.694249 7138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.695019 7138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:11.695942 7142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:11.696321 7142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:11.697256 7138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:11.715840 7156 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:11.716126 7156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:11.722121 7156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:11.725273 7156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:11.717518 7069 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:11.728484 7069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:11.801949 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.805008 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:52809; Asking this server to re-register.
I20260811 18:58:11.805908 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.807832 7100 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:11.851076 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.852293 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:40289; Asking this server to re-register.
I20260811 18:58:11.853003 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.854477 7100 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:11.895342 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.896765 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:34299; Asking this server to re-register.
I20260811 18:58:11.897547 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.899837 7100 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:12.183928 7142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:12.184216 7142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:12.186159 7142 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:12.186980 7086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:12.188494 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:12.188897 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:12.189069 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:12.189527 7142 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:12.925866 7163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:12.926120 7163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:12.927129 7163 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:12.930639 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:12.930934 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:12.931180 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:12.931533 7163 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:13.451040 7167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:13.451292 7167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:13.451980 7167 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:13.454582 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:13.454792 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:13.454941 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:13.455235 7167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:14.076944 7171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:14.077155 7171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:14.077926 7171 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:14.080896 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:14.081172 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:14.081348 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:14.081650 7171 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:14.755280 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--webserver_interface=127.3.72.126
--webserver_port=35183
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:15.155936 7175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:15.156347 7175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:15.156512 7175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:15.164979 7175 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:15.165210 7175 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:15.165341 7175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:15.165477 7175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:15.165575 7175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:15.165685 7175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:15.177347 7175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=35183
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:15.179021 7175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:15.181062 7175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:15.195993 7180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:15.196062 7181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:15.196236 7183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:15.197364 7175 server_base.cc:1034] running on GCE node
I20260811 18:58:15.198863 7175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:15.200845 7175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:15.202145 7175 hybrid_clock.cc:648] HybridClock initialized: now 1786474695202038 us; error 69 us; skew 500 ppm
I20260811 18:58:15.202607 7175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:15.212163 7175 webserver.cc:460] Webserver started at http://127.3.72.126:35183/ using document root <none> and password file <none>
I20260811 18:58:15.213174 7175 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:15.213351 7175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:15.221449 7175 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:58:15.224818 7189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:15.226504 7175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:15.226701 7175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:15.227849 7175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:15.246827 7175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:15.247871 7175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:15.248216 7175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:15.270087 7175 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:37881
I20260811 18:58:15.270211 7241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:37881 every 8 connection(s)
I20260811 18:58:15.271904 7175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:15.279806 7242 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:15.281828 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7175
I20260811 18:58:15.284410 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap starting.
I20260811 18:58:15.296660 7242 log.cc:824] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:15.305920 6685 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.310240 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:15.310958 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap complete.
I20260811 18:58:15.319761 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.323431 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Initialized, Role: FOLLOWER
I20260811 18:58:15.324278 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.325294 7248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:15.325642 7248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:15.326607 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:15.341362 7262 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:15.341511 7262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: success
I20260811 18:58:15.346840 7262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: success
I20260811 18:58:15.350137 7262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:15.373560 7175 master_runner.cc:186] Error getting master registration for 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111),
I20260811 18:58:15.383657 7175 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:15.562997 6532 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.563474 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:15.564041 6844 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.564282 6844 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:15.566920 7206 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:55045; Asking this server to re-register.
I20260811 18:58:15.567026 7207 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:33633; Asking this server to re-register.
I20260811 18:58:15.568856 6844 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.569373 6532 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.571136 7206 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:15.571208 7207 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:15.728530 7248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:15.728991 7248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.731257 7248 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:15.732177 7194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:15.732266 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:15.732964 7111 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a1e448ebc1343319544bc81ea89fed8 in current term 6: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
W20260811 18:58:15.734061 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:15.734253 7194 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8; no voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:15.734707 7248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Advancing to term 6
I20260811 18:58:15.739266 7248 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260811 18:58:15.809684 7207 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:56075; Asking this server to re-register.
I20260811 18:58:15.810441 6685 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.811821 7207 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:16.208565 7269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:16.208861 7269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.209764 7269 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:16.222565 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.226066 7217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:16.226886 7217 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260811 18:58:16.227926 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:16.228317 7269 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:16.449712 7277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:16.449976 7277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.450846 7277 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:16.451584 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:16.451906 7111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 6.
I20260811 18:58:16.452591 7193 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
I20260811 18:58:16.452950 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260811 18:58:16.453259 7277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:16.453354 7277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Advancing to term 7
W20260811 18:58:16.454066 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.456180 7277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.456832 7277 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:16.457507 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:16.457768 7111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Advancing to term 7
W20260811 18:58:16.458436 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.462527 7111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 7.
I20260811 18:58:16.463217 7193 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters: e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:16.463651 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 FOLLOWER]: Leader election won for term 7
I20260811 18:58:16.464211 7277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:16.464787 7277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.467255 7280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:16.467768 7280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:16.743526 7069 master_runner.cc:186] Error getting master registration for 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111),
I20260811 18:58:16.743829 7069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:16.932271 7194 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 -> Peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Couldn't send request to peer e19621f5b4944a18b25b0ed2e383027f. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:58:16.994429 7111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260811 18:58:16.995357 7280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:58:16.995759 7280 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer 6bb6ba710ee24c348d653d524bb73f3d log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260811 18:58:16.997853 7111 pending_rounds.cc:76] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Aborting all ops after (but not including) 5
I20260811 18:58:16.998124 7111 pending_rounds.cc:97] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260811 18:58:16.998345 7111 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260811 18:58:17.006273 7289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.006363 7292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.006831 7289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.006887 7292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.007138 7290 mvcc.cc:205] Tried to move back new op lower bound from 7317400315719868416 to 7317400314706460672. Current Snapshot: MvccSnapshot[applied={T|T < 7317400315719868416 or (T in {7317400315719868416})}]
I20260811 18:58:17.009758 7292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.009742 7294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.010035 7292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.010161 7294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:17.114589 7282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:17.117508 7282 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:17.119329 7282 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:17.119660 7282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:17.120545 7282 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:17.120654 7282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:17.121866 7282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:17.122826 7282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:17.123703 7282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:17.124460 7282 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:17.125149 7282 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:17.243397 7206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243541 7205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243832 7203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243397 7207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.244823 7202 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36952:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:17.246322 7198 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245195 7204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245253 7199 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245806 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.246572 7201 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:17.251233 7261 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f03187849ad google::LogMessage::Fail() at ??:0
@ 0x7f031878788a google::LogMessage::SendToLog() at ??:0
@ 0x7f03187844de google::LogMessage::Flush() at ??:0
@ 0x7f03187846d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f032e322ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f032e321901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f032e32866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f032e3284e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f031b3aeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f031e6886db start_thread at ??:0
@ 0x7f031761e61f clone at ??:0
W20260811 18:58:17.481992 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.126:37881)
W20260811 18:58:17.482470 6571 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:17.483399 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.126:37881)
I20260811 18:58:17.510010 7292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:17.510231 7292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:17.511168 7292 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:17.512176 7087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260811 18:58:17.513841 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:17.514165 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:17.514331 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:17.514662 7292 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.130678 7316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:18.130929 7316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.131613 7316 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:18.134080 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:18.134441 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:18.134560 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:18.134895 7316 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.160869 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:18.579213 7320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:18.579526 7320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:18.579604 7320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:18.588446 7320 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:18.588569 7320 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:18.588614 7320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:18.588670 7320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:18.588711 7320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:18.588778 7320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:18.599332 7320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:18.601086 7320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:18.603224 7320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:18.617705 7325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.617468 7326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.617535 7328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:18.619343 7320 server_base.cc:1034] running on GCE node
I20260811 18:58:18.620464 7320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:18.622380 7320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:18.623720 7320 hybrid_clock.cc:648] HybridClock initialized: now 1786474698623589 us; error 103 us; skew 500 ppm
I20260811 18:58:18.624238 7320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:18.627234 7320 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:18.628122 7320 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:18.628429 7320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:18.634922 7320 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 18:58:18.638706 7335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.640659 7320 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:58:18.640887 7320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:18.642068 7320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:18.663653 7340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:18.663887 7340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.664752 7340 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:18.667657 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:18.667896 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:18.668056 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:18.668354 7340 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.680785 7320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:18.681886 7320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:18.682264 7320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:18.706449 7320 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:18.706557 7393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:18.708173 7320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:18.710819 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7320
I20260811 18:58:18.716590 7394 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:18.720460 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:18.731840 7394 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:18.744320 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:18.745035 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:18.755316 7394 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.757016 7394 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:18.757952 7394 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.759254 7398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:18.759797 7398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:18.760798 7394 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:18.784077 7413 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:18.784302 7413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:18.776634 7320 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:18.787835 7320 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:18.789924 7413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:18.794412 7413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:18.797214 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:18.800978 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:58771; Asking this server to re-register.
I20260811 18:58:18.802157 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.804374 7357 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:18.851450 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:18.851722 6843 heartbeater.cc:503] Master 127.3.72.124:35445 was elected leader, sending a full tablet report...
I20260811 18:58:18.852751 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:45467; Asking this server to re-register.
I20260811 18:58:18.853403 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.854700 7357 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:19.120090 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:19.121390 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:35365; Asking this server to re-register.
I20260811 18:58:19.122077 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:19.123684 7357 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:19.212527 7398 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:19.212878 7398 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.215232 7398 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:19.216292 7338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:19.218794 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.224226 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:19.224599 7111 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 7: Already voted for candidate 1a1e448ebc1343319544bc81ea89fed8 in this term.
I20260811 18:58:19.225584 7338 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:19.226027 7398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Advancing to term 7
I20260811 18:58:19.230790 7398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:19.332111 7421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:19.332386 7421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.333145 7421 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:19.335733 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.340662 7367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:19.341185 7367 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 7.
I20260811 18:58:19.341707 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:19.342021 7421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260811 18:58:19.342149 7421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:19.342262 7421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Advancing to term 8
I20260811 18:58:19.344748 7421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.345309 7421 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:19.345927 7367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:19.346304 7367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Advancing to term 8
W20260811 18:58:19.347087 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.348814 7367 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 8.
I20260811 18:58:19.349282 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:19.349591 7421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Leader election won for term 8
I20260811 18:58:19.350114 7421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:19.350643 7421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.352610 7425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 8 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:19.353107 7425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:19.353868 7427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:19.357038 7427 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:19.358820 7427 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:19.359169 7427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:19.360124 7427 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:19.360208 7427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:19.361423 7427 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:19.362419 7427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:19.363415 7427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:19.364282 7427 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:19.365123 7427 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:19.379349 7098 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379709 7097 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380532 7096 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.381114 7100 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379395 7099 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380581 7094 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379801 7095 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380630 7093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.382586 7091 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.382556 7092 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:19.386703 7155 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f3a9f9949ad google::LogMessage::Fail() at ??:0
@ 0x7f3a9f99788a google::LogMessage::SendToLog() at ??:0
@ 0x7f3a9f9944de google::LogMessage::Flush() at ??:0
@ 0x7f3a9f9946d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f3ab5532ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f3ab5531901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f3ab553866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f3ab55384e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3aa25beeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3aa58986db start_thread at ??:0
@ 0x7f3a9e82e61f clone at ??:0
W20260811 18:58:19.613148 6417 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:19.613902 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:19.614244 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
I20260811 18:58:20.247174 7446 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:20.247490 7446 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:20.248386 7446 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:20.251160 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:20.251392 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:20.251477 7338 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:20.251751 7446 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:21.003063 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7320
I20260811 18:58:21.019088 7387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:21.136538 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7320
I20260811 18:58:22.065078 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:22.454006 7454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:22.454487 7454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:22.454610 7454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:22.462903 7454 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:22.463021 7454 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:22.463091 7454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:22.463186 7454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:22.463253 7454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:22.463339 7454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:22.473186 7454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:22.474777 7454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:22.476848 7454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:22.491696 7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:22.491734 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:22.491660 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:22.492818 7454 server_base.cc:1034] running on GCE node
I20260811 18:58:22.493657 7454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:22.495466 7454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:22.496719 7454 hybrid_clock.cc:648] HybridClock initialized: now 1786474702496641 us; error 33 us; skew 500 ppm
I20260811 18:58:22.497102 7454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:22.499993 7454 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:22.500773 7454 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:22.500931 7454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:22.506958 7454 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:22.510306 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:22.512290 7454 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:22.512897 7454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:22.514684 7454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:22.546968 7454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:22.547966 7454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:22.548309 7454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:22.571120 7454 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:22.571269 7523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:22.572866 7454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:22.576802 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7454
I20260811 18:58:22.581768 7524 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:22.585700 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:22.596342 7524 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:22.608453 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:22.609170 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:22.617673 7524 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:22.619760 7524 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:22.620573 7524 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:22.621553 7529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:22.622006 7529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:22.623062 7524 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:22.637333 7543 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:22.637522 7543 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:22.648631 7543 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:22.656498 7543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:22.664748 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.660135 7454 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:22.671815 7454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:22.672668 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:41971; Asking this server to re-register.
I20260811 18:58:22.673537 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.675261 7488 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:22.828688 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.830065 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:45081; Asking this server to re-register.
I20260811 18:58:22.830904 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.832340 7488 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:22.876919 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.878934 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:37451; Asking this server to re-register.
I20260811 18:58:22.879678 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.881137 7488 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:23.129132 7549 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:23.129503 7549 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:23.131852 7549 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:23.132725 7471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.134840 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.135144 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:23.135296 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:23.135783 7549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:23.811040 7552 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:23.811245 7552 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:23.811863 7552 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:23.814656 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.815233 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:23.815411 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:23.815737 7552 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:24.431964 7555 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:24.432193 7555 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:24.432960 7555 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:24.435451 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:24.435662 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:24.435817 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:24.436124 7555 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:25.665565 7559 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:25.665766 7559 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:25.666546 7559 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:25.669133 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:25.669296 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:25.669381 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:25.669612 7559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:26.046603 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:26.427449 7563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:26.427870 7563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:26.428045 7563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:26.436300 7563 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:26.436411 7563 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:26.436489 7563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:26.436592 7563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:26.436687 7563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:26.436800 7563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:26.446713 7563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:26.448096 7563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:26.449954 7563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:26.462601 7572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.462641 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.462687 7568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:26.468434 7563 server_base.cc:1034] running on GCE node
I20260811 18:58:26.469488 7563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:26.471302 7563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:26.472623 7563 hybrid_clock.cc:648] HybridClock initialized: now 1786474706472542 us; error 50 us; skew 500 ppm
I20260811 18:58:26.473022 7563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:26.480504 7563 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:26.481271 7563 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:26.481395 7563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:26.487401 7563 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:26.490741 7578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.492349 7563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:58:26.492558 7563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:26.493659 7563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:26.516739 7563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:26.517654 7563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:26.517967 7563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:26.539494 7563 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:26.539499 7632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:26.541378 7563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:26.547780 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7563
I20260811 18:58:26.549981 7633 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:26.554920 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:26.567046 7633 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:26.581060 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:26.581697 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:26.590619 7633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.591827 7633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.592459 7633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.593492 7638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:26.593907 7638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:26.594725 7633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:26.607481 7652 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:26.607698 7652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:26.611864 7652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:26.615330 7652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:26.616937 7563 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:26.627295 7563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:26.642761 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.643400 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.645711 7594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:40415; Asking this server to re-register.
I20260811 18:58:26.645744 7595 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:44173; Asking this server to re-register.
I20260811 18:58:26.646559 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.646886 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.648526 7595 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:26.648576 7594 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:26.871507 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.872848 7594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:38645; Asking this server to re-register.
I20260811 18:58:26.873525 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.874888 7594 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:26.977432 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:26.977814 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.979866 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:26.981527 7581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:26.983706 7581 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:26.989078 7498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:26.989797 7498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 8.
I20260811 18:58:26.990619 7582 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:26.991010 7638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260811 18:58:26.991148 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:26.991283 7638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Advancing to term 9
I20260811 18:58:26.995889 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.997504 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:26.998181 7498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:26.998490 7498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Advancing to term 9
W20260811 18:58:27.000160 7581 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:27.003755 7498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 9.
I20260811 18:58:27.004312 7582 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:27.004729 7638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 FOLLOWER]: Leader election won for term 9
I20260811 18:58:27.005241 7638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:27.005789 7638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:27.008268 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 9 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:27.008605 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:27.009415 7661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:27.012035 7661 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:27.013705 7661 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:27.014037 7661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:27.015039 7661 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:27.015164 7661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:27.016628 7661 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:27.017490 7661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:27.018474 7661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:27.019387 7661 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:27.020229 7661 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:27.047457 7595 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047529 7592 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047832 7593 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047045 7594 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54082:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:27.049403 7590 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049678 7587 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.048995 7588 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049129 7589 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.048820 7591 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049256 7586 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:27.053748 7651 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f9044c629ad google::LogMessage::Fail() at ??:0
@ 0x7f9044c6588a google::LogMessage::SendToLog() at ??:0
@ 0x7f9044c624de google::LogMessage::Flush() at ??:0
@ 0x7f9044c626d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f905a800ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f905a7ff901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f905a80666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f905a8064e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f904788ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f904ab666db start_thread at ??:0
@ 0x7f9043afc61f clone at ??:0
W20260811 18:58:27.279080 6717 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:27.279361 6417 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:27.281122 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:27.281917 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
I20260811 18:58:27.673807 7454 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:27.674050 7454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:28.906725 7678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:28.906991 7678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:28.907785 7678 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:28.909026 7471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260811 18:58:28.910578 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:28.910769 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:28.910862 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:28.911103 7678 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:28.982614 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7454
I20260811 18:58:28.990974 7519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:29.112630 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7454
I20260811 18:58:29.824922 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:30.207415 7686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:30.207782 7686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:30.207916 7686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:30.216292 7686 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:30.216450 7686 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:30.216528 7686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:30.216616 7686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:30.216688 7686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:30.216782 7686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:30.227523 7686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:30.229117 7686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:30.231355 7686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:30.244166 7692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:30.245167 7691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:30.245368 7694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:30.246500 7686 server_base.cc:1034] running on GCE node
I20260811 18:58:30.247448 7686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:30.249172 7686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:30.250505 7686 hybrid_clock.cc:648] HybridClock initialized: now 1786474710250418 us; error 37 us; skew 500 ppm
I20260811 18:58:30.250955 7686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:30.253839 7686 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:30.254781 7686 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:30.254931 7686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:30.261178 7686 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:30.264597 7701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:30.266094 7686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:58:30.266315 7686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:30.267464 7686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:30.284762 7686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:30.285857 7686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:30.286191 7686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:30.308552 7686 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:30.308586 7753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:30.310088 7686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:30.316403 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7686
I20260811 18:58:30.317387 7754 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:30.321951 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:30.334904 7754 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:30.346849 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:30.347606 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:30.356266 7754 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.357605 7754 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:30.358361 7754 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.359400 7758 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:30.359982 7758 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:30.361254 7754 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:30.370199 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.394381 7774 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:30.394516 7774 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:30.399672 7774 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:30.386549 7686 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:30.401043 7686 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:30.402958 7774 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:30.418064 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.421396 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:39531; Asking this server to re-register.
I20260811 18:58:30.422246 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.424360 7719 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:30.715948 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.717590 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:32983; Asking this server to re-register.
I20260811 18:58:30.718437 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.719873 7719 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:30.768415 7758 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:30.768783 7758 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.771076 7758 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:30.772430 7705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:30.774730 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:30.775112 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:30.775266 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:30.775743 7758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:30.874241 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:59101; Asking this server to re-register.
I20260811 18:58:30.874997 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.876370 7719 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:31.453084 7779 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:31.453375 7779 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:31.454185 7779 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:31.457044 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:31.457297 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:31.457458 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:31.457780 7779 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:31.985026 7783 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:31.985240 7783 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:31.985854 7783 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:31.989204 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:31.989494 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:31.989629 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:31.989935 7783 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:32.793278 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7686
I20260811 18:58:32.807073 7749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:32.926050 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7686
W20260811 18:58:32.950824 6718 connection.cc:535] client connection to 127.3.72.124:35445 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:58:33.814267 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:34.204828 7791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:34.205165 7791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:34.205324 7791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:34.213058 7791 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:34.213231 7791 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:34.213325 7791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:34.213428 7791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:34.213503 7791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:34.213603 7791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:34.224558 7791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:34.226066 7791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:34.228116 7791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:34.241853 7796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:34.242003 7797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:34.242698 7799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:34.243450 7791 server_base.cc:1034] running on GCE node
I20260811 18:58:34.244467 7791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:34.246197 7791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:34.247548 7791 hybrid_clock.cc:648] HybridClock initialized: now 1786474714247476 us; error 38 us; skew 500 ppm
I20260811 18:58:34.247948 7791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:34.250706 7791 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:34.251474 7791 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:34.251626 7791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:34.257733 7791 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:58:34.261178 7805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:34.263032 7791 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 18:58:34.263264 7791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:34.264395 7791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:34.301982 7791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:34.303045 7791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:34.303373 7791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:34.324743 7791 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:34.324769 7856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:34.326505 7791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:34.333554 7857 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:34.336295 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7791
I20260811 18:58:34.337266 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:34.349362 7857 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:34.360654 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:34.361279 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:34.369086 7857 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.370491 7857 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:34.371080 7857 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.371953 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:34.372368 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:34.373059 7857 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:34.387579 7876 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:34.387789 7876 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:34.391584 7876 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:34.394950 7876 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:34.386322 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:34.398895 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:34.438792 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.442173 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:55337; Asking this server to re-register.
I20260811 18:58:34.443033 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.444917 7822 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:34.461942 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.463243 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:50885; Asking this server to re-register.
I20260811 18:58:34.463927 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.465098 7822 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:34.742602 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.743986 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:45027; Asking this server to re-register.
I20260811 18:58:34.744664 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.745986 7822 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:34.877432 7880 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:34.877775 7880 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.879869 7880 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:34.881170 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:34.882921 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:34.883232 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:34.883370 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:34.883824 7880 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:35.412868 7883 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:35.413168 7883 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:35.414115 7883 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:35.417181 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:35.417441 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:35.417572 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:35.417845 7883 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:36.492873 7888 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:36.493101 7888 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:36.493782 7888 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:36.496563 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:36.496794 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:36.496879 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:36.497097 7888 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:36.866720 3361 master-stress-test.cc:578] Killing test thread 1/25
I20260811 18:58:37.642323 7892 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:37.642788 7892 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:37.643662 7892 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:37.646428 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:37.646656 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:37.646776 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:37.647027 7892 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:38.961045 7896 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:38.961324 7896 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:38.962260 7896 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:38.965154 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:38.965356 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:38.965438 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:38.965667 7896 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:39.400697 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:39.400923 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:39.593753 7900 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:39.593940 7900 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:39.594584 7900 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:39.597101 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:39.597448 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:39.597599 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:39.597905 7900 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:43.734455 7903 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:43.734743 7903 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:43.735538 7903 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:43.737308 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260811 18:58:43.739181 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:43.739440 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:43.739605 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:43.739969 7903 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:44.402817 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:44.403035 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:45.003635 7908 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:45.003854 7908 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:45.004544 7908 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:45.007624 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:45.007898 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:45.008037 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:45.008617 7908 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:49.405068 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:49.405514 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:53.826318 7912 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:53.826766 7912 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:53.827770 7912 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:53.829479 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 18:58:53.831815 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:53.832099 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:53.832193 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:53.832494 7912 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:54.407380 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:54.407598 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:59.409605 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:59.409950 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:04.412015 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:04.412408 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:06.099373 7922 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:06.099666 7922 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:06.100638 7922 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:06.102135 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:06.103814 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:06.104140 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:06.104275 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:06.104616 7922 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260811 18:59:06.395023 6533 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:06.395159 6846 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:06.504822 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:09.414198 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:09.414541 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:14.416687 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:14.416918 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:18.623854 7931 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:18.624151 7931 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:18.624969 7931 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:18.627087 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:18.628983 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:18.629379 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:18.629485 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:18.629853 7931 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:19.418907 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:19.419219 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:24.426990 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:24.427345 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:29.429617 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:29.429836 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:33.941705 7941 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:33.941988 7941 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:33.943043 7941 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:33.944380 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:33.945920 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:33.946153 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:33.946235 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:33.946528 7941 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:34.431495 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:34.431697 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:36.075011 7947 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:36.075330 7947 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:36.076049 7947 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:36.079867 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:36.080094 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:59:36.080232 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:36.080551 7947 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:39.433455 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:39.433650 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:44.435492 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:44.435676 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:49.437656 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:49.437933 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:54.439867 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:54.440062 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:55.736479 7961 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:55.736788 7961 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:55.737772 7961 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:55.739459 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 18:59:55.744019 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:55.744342 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:55.744493 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:55.744833 7961 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:59.441952 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:59.442210 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:04.444458 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:04.444718 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:00:06.406613 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (217 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:06.406651 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (217 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:06.689261 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (198 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:09.062016 7968 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:09.062394 7968 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:09.063426 7968 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:09.065153 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:09.067040 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:09.067288 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:09.067432 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:09.067826 7968 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:09.446271 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:09.446512 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:14.448335 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:14.448549 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:19.450500 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:19.450686 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:23.433378 7977 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:23.433660 7977 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:23.434664 7977 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:23.435853 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:23.437616 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:23.437847 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:23.437983 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:23.438320 7977 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:24.452404 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:24.452589 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:29.454685 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:29.454983 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:34.456832 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:34.457059 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:34.506237 7811 maintenance_manager.cc:419] P e19621f5b4944a18b25b0ed2e383027f: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033371
I20260811 19:00:34.563489 7810 maintenance_manager.cc:643] P e19621f5b4944a18b25b0ed2e383027f: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.022s sys 0.009s Metrics: {"bytes_written":6567,"cfile_init":1,"dirs.queue_time_us":504,"dirs.run_cpu_time_us":362,"dirs.run_wall_time_us":1015,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":685,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":639,"threads_started":2,"wal-append.queue_time_us":457}
I20260811 19:00:34.564803 7811 maintenance_manager.cc:419] P e19621f5b4944a18b25b0ed2e383027f: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 275 bytes on disk
I20260811 19:00:34.565747 7810 maintenance_manager.cc:643] P e19621f5b4944a18b25b0ed2e383027f: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 19:00:34.835101 7991 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:34.835477 7991 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:34.836282 7991 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:34.837414 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:34.839085 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:34.839300 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:34.839427 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:34.839787 7991 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:39.458923 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:39.459136 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:44.461148 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:44.461371 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:49.463608 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:49.463788 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:51.506369 8058 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:51.506585 8058 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:51.507519 8058 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:51.509214 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:51.510829 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:51.511051 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:51.511188 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:51.511515 8058 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:54.465586 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:54.465862 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:59.467804 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:59.468052 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:02.258162 8105 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:02.258565 8105 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:02.259464 8105 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:02.260838 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:02.262176 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:02.262508 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:02.262612 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:02.262861 8105 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:04.470180 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:04.470563 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:01:06.586760 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (337 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:06.586810 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (337 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:06.869063 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (318 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:09.221571 8133 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:09.221890 8133 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:09.222695 8133 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:09.224210 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:09.226147 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:09.226379 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:09.226521 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:09.226827 8133 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:09.472057 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:09.472260 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:14.474462 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:14.474716 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:19.477075 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:19.477377 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:22.256177 8183 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:22.256407 8183 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:22.257110 8183 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:22.258765 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:22.260535 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:22.260811 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:22.260967 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:22.261402 8183 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:24.479135 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:24.479444 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:29.481608 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:29.481927 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:34.486135 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:34.486570 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:39.249642 8246 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:39.250025 8246 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:39.251255 8246 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:39.252861 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:39.255031 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:39.255342 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:39.255499 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:39.255826 8246 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:39.488129 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:39.488473 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:42.578307 8261 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:42.578610 8261 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:42.579381 8261 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:42.583354 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 19:01:42.583607 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 19:01:42.583731 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:42.584079 8261 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:44.490156 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:44.490409 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:49.492689 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:49.492951 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:54.494848 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:54.495091 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:56.777765 8312 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:56.778113 8312 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:56.779130 8312 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:56.781039 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 19:01:56.782910 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:56.783221 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:56.783366 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:56.783702 8312 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:59.496927 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:59.497186 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:04.498886 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:04.499136 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:02:06.774541 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (457 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:06.774806 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (457 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:07.051815 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (438 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:09.521814 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:09.522025 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:10.254267 8362 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:10.254568 8362 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:10.255378 8362 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:10.257041 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:10.258667 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:10.258915 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:10.259030 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:10.259320 8362 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:14.523857 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:14.524057 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:19.526374 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:19.526613 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:22.908085 8404 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:22.908268 8404 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:22.908975 8404 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:22.910416 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:22.911820 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:22.912019 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:22.912081 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:22.912303 8404 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:24.528486 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:24.528668 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:29.530419 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:29.530627 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:31.317351 8435 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:31.317584 8435 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:31.318262 8435 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:31.319643 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:31.323495 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:31.323706 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:31.323800 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:31.324106 8435 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:34.532498 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:34.532684 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:39.465492 8466 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:39.465740 8466 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:39.466653 8466 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:39.468179 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:39.470016 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:39.470305 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:39.470461 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:39.470801 8466 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:39.534384 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:39.534588 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:44.536438 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:44.536624 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:49.538308 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:49.538552 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:49.864441 8501 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:49.864697 8501 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:49.865558 8501 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:49.866459 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:49.869565 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:49.869715 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:49.869778 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:49.869982 8501 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:54.540390 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:54.540573 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:58.082775 8527 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:58.083009 8527 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:58.083853 8527 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:58.085043 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:58.086257 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:58.086596 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:58.086664 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:58.086905 8527 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:59.542419 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:59.542635 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:03.782367 8546 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:03.782583 8546 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:03:03.783327 8546 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:03:03.784561 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:03:03.785763 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:03:03.785969 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:03:03.786058 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:03:03.786278 8546 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:03:04.544224 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:03:04.544430 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:03:06.216778 6857 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 19:03:06.217228 6857 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
F20260811 19:03:06.228590 6873 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1e31e1132ff54476bf241265a9eefc37 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786474986 (unix time) try "date -d @1786474986" if you are using GNU date ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:0*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
W20260811 19:03:06.234799 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238019 6869 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b4c535b1bf15433bb22c93170702c2e3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238019 6869 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b4c535b1bf15433bb22c93170702c2e3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238483 6864 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_007bd8c95ec94672af614d10b2bb2038 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** SIGABRT (@0x3e800000d21) received by PID 3361 (TID 0x7fc09b271700) from PID 3361; stack trace: ***
*** Check failure stack trace: ***
W20260811 19:03:06.238891 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
@ 0x7fc0ad93da46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fc0b1a7f980 (unknown) at ??:0
@ 0x7fc0aac6be87 gsignal at ??:0
@ 0x7fc0aac6d7f1 abort at ??:0
@ 0x7fc0ad92f9ad google::LogMessage::Fail() at ??:0
@ 0x7fc0ad93288a google::LogMessage::SendToLog() at ??:0
@ 0x7fc0ad92f4de google::LogMessage::Flush() at ??:0
@ 0x7fc0ad930a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x403d42 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:298
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7fc0b63bba4f execute_native_thread_routine at ??:0
@ 0x7fc0b1a746db start_thread at ??:0
@ 0x7fc0aad4e61f clone at ??:0
Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260811 18:56:57.275715 3361 test_util.cc:268] Using random seed: 150373563
2026-08-11T18:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:57Z Disabled control of system clock
I20260811 18:56:57.328140 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--webserver_interface=127.3.72.126
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:56:57.910482 3370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:57.910910 3370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:57.911060 3370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:57.918939 3370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:57.919121 3370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:57.919226 3370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:57.919328 3370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:57.929315 3370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:57.930920 3370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:57.932977 3370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:57.947027 3375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:57.947104 3378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:57.947067 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:57.948069 3370 server_base.cc:1034] running on GCE node
I20260811 18:56:57.948967 3370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:57.950644 3370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:57.951891 3370 hybrid_clock.cc:648] HybridClock initialized: now 1786474617951777 us; error 75 us; skew 500 ppm
I20260811 18:56:57.952395 3370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:56:57.955039 3370 webserver.cc:460] Webserver started at http://127.3.72.126:40953/ using document root <none> and password file <none>
I20260811 18:56:57.955807 3370 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:57.955948 3370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:57.956472 3370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:57.959506 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/instance:
uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-zpfg"
I20260811 18:56:57.960469 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal/instance:
uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-zpfg"
I20260811 18:56:57.966843 3370 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:56:57.971088 3385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:57.972657 3370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:56:57.972855 3370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-zpfg"
I20260811 18:56:57.973089 3370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:57.995613 3370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:57.996596 3370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:57.997004 3370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:58.019178 3370 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:34821
I20260811 18:56:58.019174 3440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:34821 every 8 connection(s)
I20260811 18:56:58.020925 3370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
I20260811 18:56:58.025251 3441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:58.027541 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3370
I20260811 18:56:58.027917 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal/instance
I20260811 18:56:58.030798 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:38889
--webserver_interface=127.3.72.125
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:56:58.032506 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:58.052126 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:56:58.053462 3388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
W20260811 18:56:58.055421 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
I20260811 18:56:58.104859 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } attempt: 1
W20260811 18:56:58.107378 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
I20260811 18:56:58.175725 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } attempt: 2
W20260811 18:56:58.178333 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
I20260811 18:56:58.269753 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } attempt: 3
W20260811 18:56:58.272392 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
I20260811 18:56:58.415722 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } attempt: 4
W20260811 18:56:58.418105 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111)
W20260811 18:56:58.426437 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:58.426872 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:58.427024 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:58.434695 3445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:58.434857 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:58.434950 3445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:58.435048 3445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:58.444865 3445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:58.446386 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:58.448446 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:58.461898 3451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.461923 3454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.461872 3452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:58.468572 3445 server_base.cc:1034] running on GCE node
I20260811 18:56:58.469935 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:58.471830 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:58.473260 3445 hybrid_clock.cc:648] HybridClock initialized: now 1786474618473116 us; error 107 us; skew 500 ppm
I20260811 18:56:58.473745 3445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:56:58.476827 3445 webserver.cc:460] Webserver started at http://127.3.72.125:46243/ using document root <none> and password file <none>
I20260811 18:56:58.477705 3445 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:58.477829 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:58.478215 3445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:58.481577 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/instance:
uuid: "d85eb97df7824bc9bad9dca349b0c180"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-zpfg"
I20260811 18:56:58.482621 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal/instance:
uuid: "d85eb97df7824bc9bad9dca349b0c180"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-zpfg"
I20260811 18:56:58.489256 3445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:56:58.493644 3461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.495640 3445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:56:58.495909 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "d85eb97df7824bc9bad9dca349b0c180"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-zpfg"
I20260811 18:56:58.496167 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:58.512161 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:58.513087 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:58.513443 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:58.534195 3445 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:38889
I20260811 18:56:58.534228 3517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:38889 every 8 connection(s)
I20260811 18:56:58.535919 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:56:58.540925 3518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:58.546249 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3445
I20260811 18:56:58.546478 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal/instance
I20260811 18:56:58.548038 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:44143
--webserver_interface=127.3.72.124
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:56:58.549353 3518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:58.568756 3518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:58.580775 3518 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:56:58.582091 3463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
W20260811 18:56:58.584112 3518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.633560 3518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 1
W20260811 18:56:58.635988 3518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.704326 3518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 2
W20260811 18:56:58.706804 3518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.717533 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } attempt: 5
I20260811 18:56:58.726110 3441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:56:58.728384 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.782491 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 1
W20260811 18:56:58.788699 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.798138 3518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 3
W20260811 18:56:58.800501 3518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.856993 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 2
W20260811 18:56:58.859400 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
I20260811 18:56:58.943877 3518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 4
W20260811 18:56:58.946332 3518 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
W20260811 18:56:58.962245 3522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:58.962671 3522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:58.962764 3522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:56:58.965708 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 3
W20260811 18:56:58.968369 3441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:44143: Network error: Client connection negotiation failed: client connection to 127.3.72.124:44143: connect: Connection refused (error 111)
W20260811 18:56:58.972265 3522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:58.972401 3522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:58.972508 3522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:58.972640 3522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:58.984540 3522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:44143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:58.986126 3522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:58.988358 3522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:59.006142 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:59.006161 3532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:59.006587 3529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:59.007153 3522 server_base.cc:1034] running on GCE node
I20260811 18:56:59.008270 3522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:59.010253 3522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:59.011703 3522 hybrid_clock.cc:648] HybridClock initialized: now 1786474619011502 us; error 61 us; skew 500 ppm
I20260811 18:56:59.012351 3522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:56:59.016845 3522 webserver.cc:460] Webserver started at http://127.3.72.124:34085/ using document root <none> and password file <none>
I20260811 18:56:59.018167 3522 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:59.018514 3522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:59.019177 3522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:59.023362 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data/instance:
uuid: "0a82a7352eb644b4a6baf09d8084b652"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-zpfg"
I20260811 18:56:59.024577 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal/instance:
uuid: "0a82a7352eb644b4a6baf09d8084b652"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-zpfg"
I20260811 18:56:59.032836 3522 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 18:56:59.037384 3538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:59.039588 3522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 18:56:59.039844 3522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "0a82a7352eb644b4a6baf09d8084b652"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-zpfg"
I20260811 18:56:59.040117 3522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:59.057085 3522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:59.058094 3522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:59.058524 3522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:59.080708 3522 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:44143
I20260811 18:56:59.080710 3592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:44143 every 8 connection(s)
I20260811 18:56:59.082707 3522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:56:59.087008 3593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:59.090394 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3522
I20260811 18:56:59.090638 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-2/wal/instance
I20260811 18:56:59.093791 3593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:59.116180 3593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:59.125222 3593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:56:59.141505 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Bootstrap starting.
I20260811 18:56:59.145624 3593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:59.145668 3441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 4
I20260811 18:56:59.147157 3593 log.cc:824] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:59.150442 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: No bootstrap required, opened a new log
I20260811 18:56:59.160471 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap starting.
I20260811 18:56:59.162005 3593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.162556 3593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:59.162706 3593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a82a7352eb644b4a6baf09d8084b652, State: Initialized, Role: FOLLOWER
I20260811 18:56:59.163440 3593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.164602 3604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.165037 3604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.165043 3441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:59.166122 3593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:59.166771 3441 log.cc:824] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:59.170826 3441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: No bootstrap required, opened a new log
W20260811 18:56:59.178974 3617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:56:59.179239 3617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:56:59.182600 3441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.183039 3441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:59.183193 3441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb005c681cf42d4ac278ec73a28e4a9, State: Initialized, Role: FOLLOWER
I20260811 18:56:59.183969 3441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.185075 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.185657 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.186648 3441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:59.175758 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: OK, instance_id { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" instance_seqno: 1786474618012972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:56:59.189575 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:56:59.199581 3630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:56:59.199793 3630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:56:59.199620 3370 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: OK, instance_id { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" instance_seqno: 1786474618528155 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:56:59.214125 3370 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:56:59.245659 3518 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } attempt: 5
W20260811 18:56:59.250429 3392 tablet.cc:2369] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:56:59.260289 3518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap starting.
I20260811 18:56:59.264058 3518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:59.265403 3518 log.cc:824] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:59.268906 3518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: No bootstrap required, opened a new log
I20260811 18:56:59.275142 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:56:59.275471 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.277220 3604 leader_election.cc:288] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889)
I20260811 18:56:59.279125 3414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a82a7352eb644b4a6baf09d8084b652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:56:59.279886 3414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a82a7352eb644b4a6baf09d8084b652 in term 0.
I20260811 18:56:59.280674 3518 raft_consensus.cc:348] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.281054 3540 leader_election.cc:302] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9; no voters:
I20260811 18:56:59.281360 3518 raft_consensus.cc:374] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:59.281584 3518 raft_consensus.cc:729] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85eb97df7824bc9bad9dca349b0c180, State: Initialized, Role: FOLLOWER
I20260811 18:56:59.281695 3604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:56:59.281878 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:56:59.282015 3604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:59.282863 3518 consensus_queue.cc:260] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.286948 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:56:59.289012 3414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a82a7352eb644b4a6baf09d8084b652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
I20260811 18:56:59.289407 3414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:59.292572 3518 sys_catalog.cc:564] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:59.293478 3414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a82a7352eb644b4a6baf09d8084b652 in term 1.
I20260811 18:56:59.290243 3633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.294018 3633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.294324 3490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a82a7352eb644b4a6baf09d8084b652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:56:59.295163 3490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a82a7352eb644b4a6baf09d8084b652 in term 0.
I20260811 18:56:59.301306 3540 leader_election.cc:302] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9; no voters:
I20260811 18:56:59.304930 3604 leader_election.cc:288] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [CANDIDATE]: Term 1 election: Requested vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889)
I20260811 18:56:59.305874 3604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:59.306500 3604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [term 1 LEADER]: Becoming Leader. State: Replica: 0a82a7352eb644b4a6baf09d8084b652, State: Running, Role: LEADER
I20260811 18:56:59.307641 3604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:56:59.311007 3544 tablet.cc:2369] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:56:59.316051 3490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a82a7352eb644b4a6baf09d8084b652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180"
I20260811 18:56:59.316457 3490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:59.321722 3490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a82a7352eb644b4a6baf09d8084b652 in term 1.
I20260811 18:56:59.330106 3636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.336426 3636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:59.342636 3643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:56:59.346879 3643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:56:59.365181 3649 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:56:59.373037 3490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Refusing update from remote peer 0a82a7352eb644b4a6baf09d8084b652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:59.374677 3636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:56:59.380534 3414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Refusing update from remote peer 0a82a7352eb644b4a6baf09d8084b652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:59.384996 3636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:56:59.401306 3633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.401701 3633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.406447 3648 mvcc.cc:205] Tried to move back new op lower bound from 7317400040933785600 to 7317400040701329408. Current Snapshot: MvccSnapshot[applied={T|T < 7317400040933785600}]
I20260811 18:56:59.410952 3636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.411005 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.411250 3650 mvcc.cc:205] Tried to move back new op lower bound from 7317400040933785600 to 7317400040701329408. Current Snapshot: MvccSnapshot[applied={T|T < 7317400040933785600}]
I20260811 18:56:59.411329 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.411334 3636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:59.413327 3643 catalog_manager.cc:1347] Generated new cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:56:59.413475 3643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:56:59.413621 3633 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.413906 3633 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.420149 3604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.420742 3604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:59.432792 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:56:59.433240 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:56:59.446728 3643 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:56:59.448648 3643 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:56:59.458657 3643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Generated new TSK 0
I20260811 18:56:59.461686 3643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:56:59.491406 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.65:0
--local_ip_for_outbound_sockets=127.3.72.65
--webserver_interface=127.3.72.65
--webserver_port=0
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:56:59.961088 3663 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
W20260811 18:56:59.961088 3659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:59.961493 3659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260811 18:56:59.961342 3663 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:56:59.961644 3659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:56:59.962841 3663 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:56:59.963543 3414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:56:59.963744 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:56:59.964581 3463 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:56:59.965024 3663 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:56:59.977244 3659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:59.977607 3659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.65
I20260811 18:56:59.998903 3659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.72.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.65
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:00.001102 3659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:00.003599 3659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:00.022716 3666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:00.022897 3665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:00.024091 3659 server_base.cc:1034] running on GCE node
W20260811 18:57:00.023454 3668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:00.025350 3659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:00.026646 3659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:00.027916 3659 hybrid_clock.cc:648] HybridClock initialized: now 1786474620027845 us; error 45 us; skew 500 ppm
I20260811 18:57:00.028360 3659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:00.031260 3659 webserver.cc:460] Webserver started at http://127.3.72.65:43065/ using document root <none> and password file <none>
I20260811 18:57:00.032063 3659 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:00.032207 3659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:00.032579 3659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:00.035827 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data/instance:
uuid: "bd8c4bdbd811417ab011a7131f17d283"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.036741 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance:
uuid: "bd8c4bdbd811417ab011a7131f17d283"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.043123 3659 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 18:57:00.047436 3675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.049176 3659 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:57:00.049490 3659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal
uuid: "bd8c4bdbd811417ab011a7131f17d283"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.049707 3659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:00.069373 3659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:00.070434 3659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:00.070789 3659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:00.071939 3659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:00.073714 3659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:00.073791 3659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.073961 3659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:00.074045 3659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.119779 3659 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.65:32817
I20260811 18:57:00.119825 3792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.65:32817 every 8 connection(s)
I20260811 18:57:00.121280 3659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:00.122208 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3659
I20260811 18:57:00.122442 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:00.129323 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.66:0
--local_ip_for_outbound_sockets=127.3.72.66
--webserver_interface=127.3.72.66
--webserver_port=0
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:57:00.154613 3796 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:00.154696 3794 heartbeater.cc:347] Connected to a master server at 127.3.72.124:44143
I20260811 18:57:00.154835 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:00.155109 3794 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.155107 3795 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.155109 3796 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.156241 3796 heartbeater.cc:511] Master 127.3.72.125:38889 requested a full tablet report, sending...
I20260811 18:57:00.156272 3795 heartbeater.cc:511] Master 127.3.72.126:34821 requested a full tablet report, sending...
I20260811 18:57:00.157135 3794 heartbeater.cc:511] Master 127.3.72.124:44143 requested a full tablet report, sending...
I20260811 18:57:00.158806 3555 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:00.158828 3403 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:00.159065 3479 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:00.160645 3555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.65:60889
I20260811 18:57:00.202230 3630 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:00.202414 3630 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: loading cluster ID for follower catalog manager: success
I20260811 18:57:00.205273 3630 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: acquiring CA information for follower catalog manager: success
I20260811 18:57:00.207283 3630 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:00.371019 3649 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:00.371179 3649 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: loading cluster ID for follower catalog manager: success
I20260811 18:57:00.375763 3649 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: acquiring CA information for follower catalog manager: success
I20260811 18:57:00.379374 3649 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:57:00.512576 3804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:00.512902 3804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:00.513052 3804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:00.520848 3804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:00.521082 3804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.66
I20260811 18:57:00.525616 3810 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:00.525933 3810 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:00.526830 3810 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:00.527519 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:00.527529 3414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:57:00.528303 3463 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:00.528805 3810 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:00.532970 3804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.72.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.66
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:00.534556 3804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:00.536448 3804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:00.554042 3812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:00.554807 3815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:00.554590 3813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:00.555738 3804 server_base.cc:1034] running on GCE node
I20260811 18:57:00.556516 3804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:00.557678 3804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:00.558970 3804 hybrid_clock.cc:648] HybridClock initialized: now 1786474620558885 us; error 37 us; skew 500 ppm
I20260811 18:57:00.559360 3804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:00.562459 3804 webserver.cc:460] Webserver started at http://127.3.72.66:38085/ using document root <none> and password file <none>
I20260811 18:57:00.563309 3804 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:00.563450 3804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:00.563798 3804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:00.567127 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data/instance:
uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.568178 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance:
uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.574800 3804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 18:57:00.579098 3821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.580704 3804 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:57:00.580866 3804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal
uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
format_stamp: "Formatted at 2026-08-11 18:57:00 on dist-test-slave-zpfg"
I20260811 18:57:00.581041 3804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:00.598042 3804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:00.599092 3804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:00.599378 3804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:00.600544 3804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:00.602470 3804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:00.602555 3804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.602648 3804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:00.602696 3804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:00.643074 3804 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.66:40837
I20260811 18:57:00.643091 3937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.66:40837 every 8 connection(s)
I20260811 18:57:00.644562 3804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:00.650830 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3804
I20260811 18:57:00.651059 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:00.658408 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.67:0
--local_ip_for_outbound_sockets=127.3.72.67
--webserver_interface=127.3.72.67
--webserver_port=0
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:57:00.663390 3794 heartbeater.cc:503] Master 127.3.72.124:44143 was elected leader, sending a full tablet report...
I20260811 18:57:00.681914 3938 heartbeater.cc:347] Connected to a master server at 127.3.72.124:44143
I20260811 18:57:00.682255 3938 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.683456 3938 heartbeater.cc:511] Master 127.3.72.124:44143 requested a full tablet report, sending...
I20260811 18:57:00.685626 3555 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:00.686054 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:00.686200 3939 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.686739 3939 heartbeater.cc:511] Master 127.3.72.126:34821 requested a full tablet report, sending...
I20260811 18:57:00.686841 3555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.66:35625
I20260811 18:57:00.688222 3403 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:00.691557 3940 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:00.691684 3940 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:00.692005 3940 heartbeater.cc:511] Master 127.3.72.125:38889 requested a full tablet report, sending...
I20260811 18:57:00.693320 3479 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:01.065986 3947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:01.066545 3947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:01.066704 3947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:01.077405 3947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:01.077682 3947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.67
I20260811 18:57:01.090703 3947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.72.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.67
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:01.092228 3947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:01.094432 3947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:01.108613 3957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:01.108824 3954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:01.108843 3955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:01.109822 3947 server_base.cc:1034] running on GCE node
I20260811 18:57:01.110559 3947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:01.111569 3947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:01.112847 3947 hybrid_clock.cc:648] HybridClock initialized: now 1786474621112753 us; error 68 us; skew 500 ppm
I20260811 18:57:01.113351 3947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:01.116499 3947 webserver.cc:460] Webserver started at http://127.3.72.67:40053/ using document root <none> and password file <none>
I20260811 18:57:01.117378 3947 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:01.117549 3947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:01.117928 3947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:01.122309 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data/instance:
uuid: "59811bdf82234ef7ac5865dbecfcc388"
format_stamp: "Formatted at 2026-08-11 18:57:01 on dist-test-slave-zpfg"
I20260811 18:57:01.123584 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance:
uuid: "59811bdf82234ef7ac5865dbecfcc388"
format_stamp: "Formatted at 2026-08-11 18:57:01 on dist-test-slave-zpfg"
I20260811 18:57:01.130771 3947 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.006s
I20260811 18:57:01.134903 3963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:01.136777 3947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:57:01.136946 3947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal
uuid: "59811bdf82234ef7ac5865dbecfcc388"
format_stamp: "Formatted at 2026-08-11 18:57:01 on dist-test-slave-zpfg"
I20260811 18:57:01.137224 3947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:01.144922 3964 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:01.145251 3964 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:01.146201 3964 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:01.147253 3414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:57:01.148392 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:01.149178 3463 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:01.149693 3964 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:01.173575 3947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:01.174599 3947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:01.174907 3947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:01.175947 3947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:01.177816 3947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:01.177919 3947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:01.178054 3947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:01.178126 3947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:01.191721 3938 heartbeater.cc:503] Master 127.3.72.124:44143 was elected leader, sending a full tablet report...
I20260811 18:57:01.241528 3947 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.67:36027
I20260811 18:57:01.241753 4081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.67:36027 every 8 connection(s)
I20260811 18:57:01.243248 3947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:01.245146 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 3947
I20260811 18:57:01.245380 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:01.285605 4082 heartbeater.cc:347] Connected to a master server at 127.3.72.124:44143
I20260811 18:57:01.286085 4082 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:01.287289 4082 heartbeater.cc:511] Master 127.3.72.124:44143 requested a full tablet report, sending...
I20260811 18:57:01.289309 3555 ts_manager.cc:194] Registered new tserver with Master: 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:01.290786 3555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.67:55995
I20260811 18:57:01.293311 4083 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:01.293493 4083 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:01.293926 4083 heartbeater.cc:511] Master 127.3.72.126:34821 requested a full tablet report, sending...
I20260811 18:57:01.295415 3403 ts_manager.cc:194] Registered new tserver with Master: 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:01.298146 4084 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:01.298358 4084 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:01.298900 4084 heartbeater.cc:511] Master 127.3.72.125:38889 requested a full tablet report, sending...
I20260811 18:57:01.300163 3479 ts_manager.cc:194] Registered new tserver with Master: 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:01.308703 3361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:01.353055 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d488b371aeb84559bed0d140537db682"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.355693 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d488b371aeb84559bed0d140537db682 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.357501 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_11c4d08f226941fdb0f2d3b17fd33b17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.358227 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11c4d08f226941fdb0f2d3b17fd33b17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.358021 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_99db5c1d5a0744d2a8ef9d9e9a13b124"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.358736 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99db5c1d5a0744d2a8ef9d9e9a13b124 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.358616 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b34b3a71e6394305a534ce2a9ceed7ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.359355 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b34b3a71e6394305a534ce2a9ceed7ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.359630 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f468443f977a4248b2afb6698384491f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.360261 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f468443f977a4248b2afb6698384491f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.360383 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0620769b48f94638bfe2b7a119070134"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.361162 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0620769b48f94638bfe2b7a119070134 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.361940 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0fc6ae15936246b7a4e102d95ac45648"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.362663 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fc6ae15936246b7a4e102d95ac45648 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.371412 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_057c92f7267b4af1bb92c56b041bdeba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.372463 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_057c92f7267b4af1bb92c56b041bdeba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.378453 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b6b85dc73bcd4412a6f7539f25ba8146"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.379201 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6b85dc73bcd4412a6f7539f25ba8146 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.388626 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5bb8e34ec2a24d83bbc88d2df2aa7522"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.389601 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bb8e34ec2a24d83bbc88d2df2aa7522 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.500180 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_11c4d08f226941fdb0f2d3b17fd33b17" } new_table_name: "default.Table_9da032177152446581e6392740034c1a" modify_external_catalogs: true
I20260811 18:57:01.501566 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d488b371aeb84559bed0d140537db682" } modify_external_catalogs: true
I20260811 18:57:01.501928 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d488b371aeb84559bed0d140537db682" } modify_external_catalogs: true
I20260811 18:57:01.502828 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0620769b48f94638bfe2b7a119070134" } new_table_name: "default.Table_5834a4f3e75d4290b27a666d9071e0bc" modify_external_catalogs: true
I20260811 18:57:01.504611 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b6b85dc73bcd4412a6f7539f25ba8146" } modify_external_catalogs: true
I20260811 18:57:01.504900 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b6b85dc73bcd4412a6f7539f25ba8146" } modify_external_catalogs: true
I20260811 18:57:01.505407 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_057c92f7267b4af1bb92c56b041bdeba" } modify_external_catalogs: true
I20260811 18:57:01.505684 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_057c92f7267b4af1bb92c56b041bdeba" } modify_external_catalogs: true
I20260811 18:57:01.510664 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b34b3a71e6394305a534ce2a9ceed7ac" } new_table_name: "default.Table_e1462b7a03ef40b8998e682f7311fc09" modify_external_catalogs: true
I20260811 18:57:01.511976 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f468443f977a4248b2afb6698384491f" } new_table_name: "default.Table_4df58e6e7b7941c1b397a058da5823c1" modify_external_catalogs: true
I20260811 18:57:01.512415 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0fc6ae15936246b7a4e102d95ac45648" } modify_external_catalogs: true
I20260811 18:57:01.512933 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_99db5c1d5a0744d2a8ef9d9e9a13b124" } new_table_name: "default.Table_2d1dc92a1a5b4a0b803b335d7a825430" modify_external_catalogs: true
I20260811 18:57:01.514921 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3445
I20260811 18:57:01.515712 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0fc6ae15936246b7a4e102d95ac45648" } modify_external_catalogs: true
I20260811 18:57:01.516860 3513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:01.526211 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5bb8e34ec2a24d83bbc88d2df2aa7522" } modify_external_catalogs: true
I20260811 18:57:01.526552 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5bb8e34ec2a24d83bbc88d2df2aa7522" } modify_external_catalogs: true
I20260811 18:57:01.532204 3547 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 9cf3bab4621e49c4b9d535c4f6b19b2c
I20260811 18:57:01.533372 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 798118d3d13f448daca412c75e35d29f
W20260811 18:57:01.533668 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 89b5c0f3673445bbba9136a040ca9bbb has no consensus state
I20260811 18:57:01.534268 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 4fcb0d17338544b9bdd04ceef527cc58
W20260811 18:57:01.534541 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e2f74e1a43242469ae646cd72708273 has no consensus state
I20260811 18:57:01.536092 3548 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 369a8cf2f842442dbbc6f884c9c4ca8d
W20260811 18:57:01.541133 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 90175b8fc5e34f40adb97de7f877ebc4 has no consensus state
W20260811 18:57:01.541450 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce825a4b0853440090c12cf6387d94b4 has no consensus state
W20260811 18:57:01.541956 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 621b56ba8f2f4392beea060871508a70 has no consensus state
I20260811 18:57:01.543599 3546 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 34fac26019ba4a34bbeefc79bfcece9f
W20260811 18:57:01.565395 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_5834a4f3e75d4290b27a666d9071e0bc [id=7a048f5d108d462ba534f3398356d38d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 89b5c0f3673445bbba9136a040ca9bbb consensus state has no leader
I20260811 18:57:01.568836 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2f74e1a43242469ae646cd72708273 (DEFAULT_TABLE table=default.Table_e1462b7a03ef40b8998e682f7311fc09 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.570775 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 (DEFAULT_TABLE table=default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.572229 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 621b56ba8f2f4392beea060871508a70 (DEFAULT_TABLE table=default.Table_9da032177152446581e6392740034c1a [id=7131807af30d498e993d5343a99c9a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.573259 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2f74e1a43242469ae646cd72708273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.573714 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ce825a4b0853440090c12cf6387d94b4 (DEFAULT_TABLE table=default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.574168 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce825a4b0853440090c12cf6387d94b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.574756 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb (DEFAULT_TABLE table=default.Table_5834a4f3e75d4290b27a666d9071e0bc [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.575615 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b5c0f3673445bbba9136a040ca9bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.576356 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90175b8fc5e34f40adb97de7f877ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.577339 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b56ba8f2f4392beea060871508a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.577436 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2f74e1a43242469ae646cd72708273 (DEFAULT_TABLE table=default.Table_e1462b7a03ef40b8998e682f7311fc09 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.578788 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 (DEFAULT_TABLE table=default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.579663 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90175b8fc5e34f40adb97de7f877ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.582787 3867 tablet_service.cc:1467] Processing CreateTablet for tablet ce825a4b0853440090c12cf6387d94b4 (DEFAULT_TABLE table=default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.583174 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce825a4b0853440090c12cf6387d94b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.583923 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb (DEFAULT_TABLE table=default.Table_5834a4f3e75d4290b27a666d9071e0bc [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.584331 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b5c0f3673445bbba9136a040ca9bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.589200 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 (DEFAULT_TABLE table=default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.589599 4011 tablet_service.cc:1467] Processing CreateTablet for tablet ce825a4b0853440090c12cf6387d94b4 (DEFAULT_TABLE table=default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.590798 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb (DEFAULT_TABLE table=default.Table_5834a4f3e75d4290b27a666d9071e0bc [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.590854 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2f74e1a43242469ae646cd72708273 (DEFAULT_TABLE table=default.Table_e1462b7a03ef40b8998e682f7311fc09 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.591152 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 621b56ba8f2f4392beea060871508a70 (DEFAULT_TABLE table=default.Table_9da032177152446581e6392740034c1a [id=7131807af30d498e993d5343a99c9a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.591688 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b56ba8f2f4392beea060871508a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.591780 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90175b8fc5e34f40adb97de7f877ebc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.589279 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 621b56ba8f2f4392beea060871508a70 (DEFAULT_TABLE table=default.Table_9da032177152446581e6392740034c1a [id=7131807af30d498e993d5343a99c9a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.592737 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2f74e1a43242469ae646cd72708273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.594833 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce825a4b0853440090c12cf6387d94b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.595654 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621b56ba8f2f4392beea060871508a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.596526 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89b5c0f3673445bbba9136a040ca9bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.597764 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2f74e1a43242469ae646cd72708273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.617914 4162 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:01.625483 4162 tablet_bootstrap.cc:654] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.627120 4162 log.cc:824] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:01.637285 4165 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:01.641899 4165 tablet_bootstrap.cc:654] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.644857 4165 log.cc:824] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:01.647351 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e2f74e1a43242469ae646cd72708273 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.649394 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e2f74e1a43242469ae646cd72708273 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.649749 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 89b5c0f3673445bbba9136a040ca9bbb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.650995 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e2f74e1a43242469ae646cd72708273 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.651333 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 89b5c0f3673445bbba9136a040ca9bbb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.662016 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 89b5c0f3673445bbba9136a040ca9bbb
I20260811 18:57:01.662803 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 89b5c0f3673445bbba9136a040ca9bbb deleted and replaced by tablet 39a040ea35974f709a7369080925d470
I20260811 18:57:01.664033 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:01.664078 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.668057 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1e2f74e1a43242469ae646cd72708273
I20260811 18:57:01.668704 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 1e2f74e1a43242469ae646cd72708273 deleted and replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:01.669902 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.670882 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:01.671885 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cdf0c1635bbe4ff8be0051eb9b492d6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.672392 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f6ec869a56104ffdbae33e27716e84bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.672590 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cdf0c1635bbe4ff8be0051eb9b492d6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:01.672946 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6ec869a56104ffdbae33e27716e84bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.674314 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24b2a8f4fc584443bfff00e6a3fa82a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.674911 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24b2a8f4fc584443bfff00e6a3fa82a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.674999 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:01.680053 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f85edf313fb440a98e508cc2ad36e6e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.680729 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f85edf313fb440a98e508cc2ad36e6e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.682588 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_199c131b75ca41e4880a6f69bc66672f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.683630 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_199c131b75ca41e4880a6f69bc66672f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.684365 4163 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:01.697160 4163 tablet_bootstrap.cc:654] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.684778 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a9e2f6b881884a519a2d275a62dee8ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.701506 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9e2f6b881884a519a2d275a62dee8ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.686861 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_485f4b3dc8ce4720a70f2d3a66648484"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.703017 4162 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:01.703269 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_485f4b3dc8ce4720a70f2d3a66648484 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.703732 4162 ts_tablet_manager.cc:1397] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.086s user 0.025s sys 0.024s
I20260811 18:57:01.704308 4163 log.cc:824] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:01.687335 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6dfd166d96d04f9d9c5abf9333ec4ba0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.704788 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dfd166d96d04f9d9c5abf9333ec4ba0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:01.689191 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 89b5c0f3673445bbba9136a040ca9bbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89b5c0f3673445bbba9136a040ca9bbb already in progress: creating tablet
W20260811 18:57:01.706694 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1e2f74e1a43242469ae646cd72708273 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e2f74e1a43242469ae646cd72708273 already in progress: creating tablet
I20260811 18:57:01.699247 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c57cb1125a64a5da0fd42e533faec2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.707365 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c57cb1125a64a5da0fd42e533faec2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.714952 4162 raft_consensus.cc:348] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.715497 4162 raft_consensus.cc:374] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.715596 4162 raft_consensus.cc:729] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.716488 4162 consensus_queue.cc:260] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.726485 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3445
I20260811 18:57:01.728154 4162 ts_tablet_manager.cc:1428] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.024s user 0.009s sys 0.010s
I20260811 18:57:01.729835 4162 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:01.731398 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c842a73deafd496bab0a3bc0b96b071d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.731779 4162 tablet_bootstrap.cc:654] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:01.732082 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c842a73deafd496bab0a3bc0b96b071d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.734746 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet ce825a4b0853440090c12cf6387d94b4 (table default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]). Expected version 1 got 0
I20260811 18:57:01.735281 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:01.735735 4162 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:01.735904 4162 ts_tablet_manager.cc:1397] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:57:01.736245 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cdf0c1635bbe4ff8be0051eb9b492d6e" } modify_external_catalogs: true
I20260811 18:57:01.736502 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cdf0c1635bbe4ff8be0051eb9b492d6e" } modify_external_catalogs: true
I20260811 18:57:01.737116 4162 raft_consensus.cc:348] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.737378 4162 raft_consensus.cc:374] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.737473 4162 raft_consensus.cc:729] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.737833 4162 consensus_queue.cc:260] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.737967 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d1dc92a1a5b4a0b803b335d7a825430" } modify_external_catalogs: true
I20260811 18:57:01.738269 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d1dc92a1a5b4a0b803b335d7a825430" } modify_external_catalogs: true
I20260811 18:57:01.738631 4162 ts_tablet_manager.cc:1428] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:01.738926 4162 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:01.740824 4162 tablet_bootstrap.cc:654] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.741331 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f85edf313fb440a98e508cc2ad36e6e1" } new_table_name: "default.Table_425ac46cd0ab4d87bca0094367567ae8" modify_external_catalogs: true
I20260811 18:57:01.743096 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4df58e6e7b7941c1b397a058da5823c1" } modify_external_catalogs: true
I20260811 18:57:01.743386 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4df58e6e7b7941c1b397a058da5823c1" } modify_external_catalogs: true
I20260811 18:57:01.744282 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:01.744696 4167 tablet_replica.cc:330] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:01.745399 4167 raft_consensus.cc:2229] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.746017 4167 raft_consensus.cc:2258] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:01.748687 4167 ts_tablet_manager.cc:1905] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.757109 4167 ts_tablet_manager.cc:1918] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:01.757334 4167 log.cc:1196] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/89b5c0f3673445bbba9136a040ca9bbb
I20260811 18:57:01.758153 4167 ts_tablet_manager.cc:1939] T 89b5c0f3673445bbba9136a040ca9bbb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:01.759039 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6ec869a56104ffdbae33e27716e84bc" } new_table_name: "default.Table_3b5916fe57274e32948594178031611f" modify_external_catalogs: true
I20260811 18:57:01.760914 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_5834a4f3e75d4290b27a666d9071e0bc [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:01.765427 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24b2a8f4fc584443bfff00e6a3fa82a5" } modify_external_catalogs: true
I20260811 18:57:01.765740 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24b2a8f4fc584443bfff00e6a3fa82a5" } modify_external_catalogs: true
I20260811 18:57:01.765818 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5834a4f3e75d4290b27a666d9071e0bc" } new_table_name: "default.Table_ecacdcdd1638424aa0a8cd6298159e98" modify_external_catalogs: true
I20260811 18:57:01.767246 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e1462b7a03ef40b8998e682f7311fc09" } new_table_name: "default.Table_ac6f106b51c54467b6bf3f075a7d9422" modify_external_catalogs: true
I20260811 18:57:01.768489 4162 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:01.768707 4162 ts_tablet_manager.cc:1397] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.006s
I20260811 18:57:01.769698 4162 raft_consensus.cc:348] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.769994 4162 raft_consensus.cc:374] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.770083 4162 raft_consensus.cc:729] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.770449 4162 consensus_queue.cc:260] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.771293 4162 ts_tablet_manager.cc:1428] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:57:01.771642 4162 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:01.773502 4162 tablet_bootstrap.cc:654] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:01.781526 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 89b5c0f3673445bbba9136a040ca9bbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89b5c0f3673445bbba9136a040ca9bbb already in progress: creating tablet
W20260811 18:57:01.794306 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 1e2f74e1a43242469ae646cd72708273 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e2f74e1a43242469ae646cd72708273 already in progress: creating tablet
I20260811 18:57:01.794878 4082 heartbeater.cc:503] Master 127.3.72.124:44143 was elected leader, sending a full tablet report...
W20260811 18:57:01.796677 3940 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:38889 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.3.72.125:38889 (error 108)
W20260811 18:57:01.797043 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 89b5c0f3673445bbba9136a040ca9bbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89b5c0f3673445bbba9136a040ca9bbb already in progress: creating tablet
W20260811 18:57:01.799427 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1e2f74e1a43242469ae646cd72708273 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e2f74e1a43242469ae646cd72708273 already in progress: creating tablet
I20260811 18:57:01.807973 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 5428282d7dcc4295b121b54055d39b71
W20260811 18:57:01.808218 3677 connection.cc:535] client connection to 127.3.72.125:38889 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:01.808531 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9da032177152446581e6392740034c1a" } new_table_name: "default.Table_59f1f5420c904158829be2c9a3524f26" modify_external_catalogs: true
W20260811 18:57:01.809165 3796 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:38889 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:01.820437 4163 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:01.820914 4163 ts_tablet_manager.cc:1397] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.137s user 0.017s sys 0.054s
I20260811 18:57:01.829878 4165 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:01.830276 4165 ts_tablet_manager.cc:1397] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.193s user 0.015s sys 0.000s
W20260811 18:57:01.834978 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 637bb23ea30c4de0a6fe23f7aaf70c25 has no consensus state
I20260811 18:57:01.835173 3548 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 621b56ba8f2f4392beea060871508a70 (table default.Table_9da032177152446581e6392740034c1a [id=7131807af30d498e993d5343a99c9a6c]). Expected version 1 got 0
I20260811 18:57:01.835458 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a9e2f6b881884a519a2d275a62dee8ce" } modify_external_catalogs: true
I20260811 18:57:01.835742 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a9e2f6b881884a519a2d275a62dee8ce" } modify_external_catalogs: true
I20260811 18:57:01.836109 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:01.837572 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet ce825a4b0853440090c12cf6387d94b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:01.837602 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet ce825a4b0853440090c12cf6387d94b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.834450 4163 raft_consensus.cc:348] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.838414 4163 raft_consensus.cc:374] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.838606 4163 raft_consensus.cc:729] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:01.838896 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet ce825a4b0853440090c12cf6387d94b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce825a4b0853440090c12cf6387d94b4 already in progress: creating tablet
I20260811 18:57:01.839627 4163 consensus_queue.cc:260] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:01.841307 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9de9acdb33a2438eb4378a5eac844c35 has no consensus state
W20260811 18:57:01.841957 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39a040ea35974f709a7369080925d470 has no consensus state
W20260811 18:57:01.844074 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ce825a4b0853440090c12cf6387d94b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce825a4b0853440090c12cf6387d94b4 already in progress: creating tablet
W20260811 18:57:01.845985 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.72.125:38889 (error 108)
I20260811 18:57:01.851646 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ce825a4b0853440090c12cf6387d94b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:01.851989 4167 tablet_replica.cc:330] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:01.852279 4167 raft_consensus.cc:2229] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.852492 4167 raft_consensus.cc:2258] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:01.853350 4167 ts_tablet_manager.cc:1905] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.853931 4162 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:01.854135 4162 ts_tablet_manager.cc:1397] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.083s user 0.047s sys 0.032s
I20260811 18:57:01.855111 4162 raft_consensus.cc:348] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.855393 4162 raft_consensus.cc:374] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.855510 4162 raft_consensus.cc:729] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.855792 4162 consensus_queue.cc:260] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.857156 4162 ts_tablet_manager.cc:1428] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:01.857470 4162 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:01.860203 4177 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:01.809900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:38889 (local address 127.3.72.65:48921)
0811 18:57:01.851243 (+ 41343us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:01.851534 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":41192,"thread_start_us":41054,"threads_started":1}
W20260811 18:57:01.862107 4176 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:01.802553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:38889 (local address 127.3.72.67:43375)
0811 18:57:01.852428 (+ 49875us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:01.852610 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49723,"thread_start_us":50594,"threads_started":1}
I20260811 18:57:01.862883 4162 tablet_bootstrap.cc:654] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:01.864773 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889): Couldn't send request to peer d85eb97df7824bc9bad9dca349b0c180. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:01.865378 4167 ts_tablet_manager.cc:1918] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:01.865499 4167 log.cc:1196] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:01.865897 4167 ts_tablet_manager.cc:1939] T ce825a4b0853440090c12cf6387d94b4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:01.866084 4084 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:38889 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection reset by peer (error 104)
I20260811 18:57:01.868043 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ce825a4b0853440090c12cf6387d94b4 (table default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]) successfully deleted
W20260811 18:57:01.868638 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ae2221b54a44f6db10f6c98ee2344a3 has no consensus state
I20260811 18:57:01.868695 4162 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:01.868947 4162 ts_tablet_manager.cc:1397] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260811 18:57:01.869072 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f6f2a50034ed4402b3ed6250fe88181a
I20260811 18:57:01.869441 3547 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 6a7eae91d2854a09b58236751ea33258
W20260811 18:57:01.869819 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 621b56ba8f2f4392beea060871508a70 consensus state has no leader
I20260811 18:57:01.869851 4162 raft_consensus.cc:348] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.870023 4163 ts_tablet_manager.cc:1428] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.049s user 0.020s sys 0.010s
I20260811 18:57:01.870129 4162 raft_consensus.cc:374] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.870229 4162 raft_consensus.cc:729] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.870602 4163 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:01.870512 4162 consensus_queue.cc:260] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.871268 4162 ts_tablet_manager.cc:1428] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:01.873234 4163 tablet_bootstrap.cc:654] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.878858 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:01.879376 4169 tablet_replica.cc:330] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:01.880066 4169 raft_consensus.cc:2229] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.880668 4169 raft_consensus.cc:2258] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:01.882037 3798 tablet.cc:2369] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:01.885080 4169 ts_tablet_manager.cc:1905] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.882124 4165 raft_consensus.cc:348] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.887398 4165 raft_consensus.cc:374] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.887622 4165 raft_consensus.cc:729] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.888705 4165 consensus_queue.cc:260] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.896673 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 621b56ba8f2f4392beea060871508a70 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.897147 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 90175b8fc5e34f40adb97de7f877ebc4 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.897468 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 89b5c0f3673445bbba9136a040ca9bbb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.899935 4165 ts_tablet_manager.cc:1428] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.069s user 0.014s sys 0.009s
I20260811 18:57:01.900769 4165 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:01.901175 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 89b5c0f3673445bbba9136a040ca9bbb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.901960 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet ce825a4b0853440090c12cf6387d94b4 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:01.902379 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_40c43b6498dc499585e6573d95a86be5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.902736 4165 tablet_bootstrap.cc:654] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:01.903054 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40c43b6498dc499585e6573d95a86be5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.903266 4163 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:01.903452 4163 ts_tablet_manager.cc:1397] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.020s
I20260811 18:57:01.905416 4163 raft_consensus.cc:348] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.905768 4163 raft_consensus.cc:374] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.905906 4163 raft_consensus.cc:729] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.906253 4163 consensus_queue.cc:260] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.906957 4163 ts_tablet_manager.cc:1428] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:01.906760 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4579e89d4503429badc8868ce44d947a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.906898 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_20068dcc1a8744ae9b1469af52723a1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.907459 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4579e89d4503429badc8868ce44d947a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:01.907454 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20068dcc1a8744ae9b1469af52723a1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.907881 4163 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:01.910598 4165 tablet_bootstrap.cc:492] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
W20260811 18:57:01.910622 3942 tablet.cc:2369] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:01.910794 4163 tablet_bootstrap.cc:654] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.910878 4165 ts_tablet_manager.cc:1397] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 18:57:01.913254 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:01.913525 4165 raft_consensus.cc:348] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.913879 4165 raft_consensus.cc:374] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.914054 4165 raft_consensus.cc:729] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.914410 4163 tablet_bootstrap.cc:492] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:01.914462 4165 consensus_queue.cc:260] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.914634 4163 ts_tablet_manager.cc:1397] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:57:01.915134 4165 ts_tablet_manager.cc:1428] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:01.915606 4165 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:01.915514 4163 raft_consensus.cc:348] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.915828 4163 raft_consensus.cc:374] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.915961 4163 raft_consensus.cc:729] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.916823 4163 consensus_queue.cc:260] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.917181 4171 raft_consensus.cc:482] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:01.917456 4163 ts_tablet_manager.cc:1428] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:01.917798 4163 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:01.917521 4171 raft_consensus.cc:504] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:01.918458 4169 ts_tablet_manager.cc:1918] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:01.918653 4169 log.cc:1196] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/1e2f74e1a43242469ae646cd72708273
I20260811 18:57:01.919183 4169 ts_tablet_manager.cc:1939] T 1e2f74e1a43242469ae646cd72708273 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:01.919817 4163 tablet_bootstrap.cc:654] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.919865 4171 leader_election.cc:288] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:01.921572 4169 tablet_replica.cc:330] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:01.921809 4169 raft_consensus.cc:2229] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.921998 4169 raft_consensus.cc:2258] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:01.922960 4169 ts_tablet_manager.cc:1905] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.923774 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 1e2f74e1a43242469ae646cd72708273 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
I20260811 18:57:01.924912 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 90175b8fc5e34f40adb97de7f877ebc4
I20260811 18:57:01.928572 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 (DEFAULT_TABLE table=default.Table_3b5916fe57274e32948594178031611f [id=263ab2d6963f45debd14127cc7d122c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.929147 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de9acdb33a2438eb4378a5eac844c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.929924 4165 tablet_bootstrap.cc:654] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.930433 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 (DEFAULT_TABLE table=default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.931068 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae2221b54a44f6db10f6c98ee2344a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.931020 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 (DEFAULT_TABLE table=default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.931576 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae2221b54a44f6db10f6c98ee2344a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.934830 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 39a040ea35974f709a7369080925d470 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.934885 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (DEFAULT_TABLE table=default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.935365 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a040ea35974f709a7369080925d470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.935421 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637bb23ea30c4de0a6fe23f7aaf70c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.938841 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (DEFAULT_TABLE table=default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.939381 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637bb23ea30c4de0a6fe23f7aaf70c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.939347 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 (DEFAULT_TABLE table=default.Table_3b5916fe57274e32948594178031611f [id=263ab2d6963f45debd14127cc7d122c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.939944 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de9acdb33a2438eb4378a5eac844c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.942847 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 (DEFAULT_TABLE table=default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.943393 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae2221b54a44f6db10f6c98ee2344a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.946928 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (DEFAULT_TABLE table=default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.947511 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637bb23ea30c4de0a6fe23f7aaf70c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.950860 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 39a040ea35974f709a7369080925d470 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.951417 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a040ea35974f709a7369080925d470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.957957 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 89b5c0f3673445bbba9136a040ca9bbb with delete_type TABLET_DATA_DELETED (replaced by tablet 39a040ea35974f709a7369080925d470) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.958525 4168 tablet_replica.cc:330] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:01.959187 4168 raft_consensus.cc:2229] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.959381 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.960114 4168 raft_consensus.cc:2258] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:01.962554 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:01.963867 4167 tablet_replica.cc:330] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:01.964124 4167 raft_consensus.cc:2229] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:01.964344 4167 raft_consensus.cc:2258] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:01.965296 4167 ts_tablet_manager.cc:1905] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.962321 4168 ts_tablet_manager.cc:1905] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:01.965708 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet ce825a4b0853440090c12cf6387d94b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.966037 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:01.972983 4162 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:01.973865 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:01.974408 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:01.975063 4162 tablet_bootstrap.cc:654] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:01.975795 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f76c62e348114008bdd5b3b8a5697df7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:01.976785 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f76c62e348114008bdd5b3b8a5697df7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.978688 4167 ts_tablet_manager.cc:1918] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:01.979055 4167 log.cc:1196] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/90175b8fc5e34f40adb97de7f877ebc4
I20260811 18:57:01.979676 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e02d62ef6e5947918f7fbcdd1c2fd560"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:01.979966 4167 ts_tablet_manager.cc:1939] T 90175b8fc5e34f40adb97de7f877ebc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:01.980521 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e02d62ef6e5947918f7fbcdd1c2fd560 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:01.982921 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 90175b8fc5e34f40adb97de7f877ebc4 (table default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]) successfully deleted
I20260811 18:57:01.988063 4162 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:01.988314 4162 ts_tablet_manager.cc:1397] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260811 18:57:01.988240 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 39a040ea35974f709a7369080925d470 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:01.988936 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a040ea35974f709a7369080925d470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:01.989244 4162 raft_consensus.cc:348] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:01.989614 4162 raft_consensus.cc:374] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:01.989749 4162 raft_consensus.cc:729] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:01.990073 4162 consensus_queue.cc:260] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:01.994876 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b56ba8f2f4392beea060871508a70" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:01.996675 3678 leader_election.cc:341] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:01.998935 4162 ts_tablet_manager.cc:1428] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260811 18:57:01.999485 4162 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.001796 4162 tablet_bootstrap.cc:654] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.004798 4165 tablet_bootstrap.cc:492] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.005059 4165 ts_tablet_manager.cc:1397] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.090s user 0.011s sys 0.008s
I20260811 18:57:02.006263 4165 raft_consensus.cc:348] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.006709 4165 raft_consensus.cc:374] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.006825 4165 raft_consensus.cc:729] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.007227 4165 consensus_queue.cc:260] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.011926 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_75948600dcf347f89478127c9f9b96fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.012368 4087 tablet.cc:2369] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:02.013316 4165 ts_tablet_manager.cc:1428] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260811 18:57:02.014410 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 (DEFAULT_TABLE table=default.Table_3b5916fe57274e32948594178031611f [id=263ab2d6963f45debd14127cc7d122c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.014714 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.015188 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de9acdb33a2438eb4378a5eac844c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.018685 4165 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.025295 4165 tablet_bootstrap.cc:654] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.026209 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.029036 4171 raft_consensus.cc:482] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.029350 4171 raft_consensus.cc:504] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.030556 4171 leader_election.cc:288] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.031291 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2f74e1a43242469ae646cd72708273" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:02.031847 4036 raft_consensus.cc:2454] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:02.032594 3678 leader_election.cc:302] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.033159 4171 raft_consensus.cc:2781] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.033373 4171 raft_consensus.cc:482] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.033535 4171 raft_consensus.cc:3037] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.034963 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_efd83594c1f04b33b212055bafe30b88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.037055 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_318043c37396470886587403377a976f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.035104 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9ccfd97c9c604a7b9f786709962c107c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.038301 4171 raft_consensus.cc:504] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.040266 4171 leader_election.cc:288] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.040593 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2f74e1a43242469ae646cd72708273" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:02.040918 4036 raft_consensus.cc:3037] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:02.043545 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_318043c37396470886587403377a976f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.044343 4036 raft_consensus.cc:2454] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:02.045009 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.044970 3678 leader_election.cc:302] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.045477 4171 raft_consensus.cc:2781] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:02.046051 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efd83594c1f04b33b212055bafe30b88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:02.047122 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ccfd97c9c604a7b9f786709962c107c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:02.048180 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75948600dcf347f89478127c9f9b96fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.052806 4168 ts_tablet_manager.cc:1918] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.053032 4168 log.cc:1196] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/89b5c0f3673445bbba9136a040ca9bbb
I20260811 18:57:02.053539 4168 ts_tablet_manager.cc:1939] T 89b5c0f3673445bbba9136a040ca9bbb P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.054186 4171 raft_consensus.cc:686] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:02.054862 4171 consensus_queue.cc:237] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.055635 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:02.055830 4168 tablet_replica.cc:330] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
W20260811 18:57:02.056001 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 90175b8fc5e34f40adb97de7f877ebc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90175b8fc5e34f40adb97de7f877ebc4 already in progress: creating tablet
I20260811 18:57:02.056035 4168 raft_consensus.cc:2229] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.056229 4168 raft_consensus.cc:2258] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.062695 4165 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.062925 4165 ts_tablet_manager.cc:1397] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.007s
I20260811 18:57:02.064090 4165 raft_consensus.cc:348] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.064507 4165 raft_consensus.cc:374] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.064639 4165 raft_consensus.cc:729] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.064944 4165 consensus_queue.cc:260] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.067099 4165 ts_tablet_manager.cc:1428] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260811 18:57:02.068423 4165 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.070967 4168 ts_tablet_manager.cc:1905] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.077353 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59f1f5420c904158829be2c9a3524f26" } modify_external_catalogs: true
I20260811 18:57:02.077636 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59f1f5420c904158829be2c9a3524f26" } modify_external_catalogs: true
I20260811 18:57:02.079058 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_199c131b75ca41e4880a6f69bc66672f" } new_table_name: "default.Table_0a4ee223798a47db804a96036f506e0c" modify_external_catalogs: true
I20260811 18:57:02.080792 4163 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.080993 4163 ts_tablet_manager.cc:1397] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.163s user 0.045s sys 0.057s
I20260811 18:57:02.071063 4165 tablet_bootstrap.cc:654] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.089048 4163 raft_consensus.cc:348] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.089675 4163 raft_consensus.cc:374] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.089895 4162 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.090081 4162 ts_tablet_manager.cc:1397] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.091s user 0.026s sys 0.044s
I20260811 18:57:02.090699 4163 raft_consensus.cc:729] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.090961 4162 raft_consensus.cc:348] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.091049 4163 consensus_queue.cc:260] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.091266 4162 raft_consensus.cc:374] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.091346 4162 raft_consensus.cc:729] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.091569 4162 consensus_queue.cc:260] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.091892 4165 tablet_bootstrap.cc:492] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.092103 4165 ts_tablet_manager.cc:1397] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.000s
I20260811 18:57:02.092989 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet ce825a4b0853440090c12cf6387d94b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.093007 4165 raft_consensus.cc:348] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.093467 4165 raft_consensus.cc:374] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.093570 4165 raft_consensus.cc:729] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.093874 4165 consensus_queue.cc:260] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.094776 4163 ts_tablet_manager.cc:1428] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.000s sys 0.003s
I20260811 18:57:02.095127 4163 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.095888 4165 ts_tablet_manager.cc:1428] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:57:02.096694 4162 ts_tablet_manager.cc:1428] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:02.098838 4165 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.100775 4165 tablet_bootstrap.cc:654] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.102149 4162 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.102998 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_425ac46cd0ab4d87bca0094367567ae8" } modify_external_catalogs: true
I20260811 18:57:02.103248 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_425ac46cd0ab4d87bca0094367567ae8" } modify_external_catalogs: true
I20260811 18:57:02.104226 4162 tablet_bootstrap.cc:654] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.104245 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_20068dcc1a8744ae9b1469af52723a1c" } new_table_name: "default.Table_3f2571e177d44cc8bcbea5387c116638" modify_external_catalogs: true
I20260811 18:57:02.106061 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3b5916fe57274e32948594178031611f" } new_table_name: "default.Table_62a6e4fda91c4b1bad9616d8f8dcdb23" modify_external_catalogs: true
W20260811 18:57:02.108968 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7d1b5cc176f54784ad79ad1ebd385acc has no consensus state
I20260811 18:57:02.109341 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6dfd166d96d04f9d9c5abf9333ec4ba0" } new_table_name: "default.Table_178c66b5e0f1479a9a97248ecab4e779" modify_external_catalogs: true
I20260811 18:57:02.095908 4168 ts_tablet_manager.cc:1918] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.110451 4168 log.cc:1196] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:02.110991 4168 ts_tablet_manager.cc:1939] T ce825a4b0853440090c12cf6387d94b4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.112264 4168 tablet_replica.cc:330] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.112428 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet ce825a4b0853440090c12cf6387d94b4 (table default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]) successfully deleted
I20260811 18:57:02.112480 4168 raft_consensus.cc:2229] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.112703 4168 raft_consensus.cc:2258] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.114470 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac6f106b51c54467b6bf3f075a7d9422" } modify_external_catalogs: true
I20260811 18:57:02.114701 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac6f106b51c54467b6bf3f075a7d9422" } modify_external_catalogs: true
I20260811 18:57:02.115645 4168 ts_tablet_manager.cc:1905] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.115926 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c842a73deafd496bab0a3bc0b96b071d" } modify_external_catalogs: true
I20260811 18:57:02.116235 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c842a73deafd496bab0a3bc0b96b071d" } modify_external_catalogs: true
I20260811 18:57:02.118201 4165 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.118388 4165 ts_tablet_manager.cc:1397] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.001s
I20260811 18:57:02.118610 4162 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.118752 4162 ts_tablet_manager.cc:1397] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260811 18:57:02.119640 4162 raft_consensus.cc:348] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.119874 4162 raft_consensus.cc:374] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:02.119874 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b227bc7305f4b2dae28f04e7001553b has no consensus state
I20260811 18:57:02.119958 4162 raft_consensus.cc:729] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.120208 4162 consensus_queue.cc:260] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.120338 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_40c43b6498dc499585e6573d95a86be5" } new_table_name: "default.Table_75b82b1226924777a76c6b0f311c9fe0" modify_external_catalogs: true
I20260811 18:57:02.120770 4162 ts_tablet_manager.cc:1428] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:02.121081 4162 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:02.122745 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9de9acdb33a2438eb4378a5eac844c35 consensus state has no leader
I20260811 18:57:02.123231 4162 tablet_bootstrap.cc:654] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.123855 4168 ts_tablet_manager.cc:1918] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.124004 4168 log.cc:1196] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/1e2f74e1a43242469ae646cd72708273
W20260811 18:57:02.124070 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e355634612f64c14965d7af2fa442cb7 has no consensus state
I20260811 18:57:02.124450 4168 ts_tablet_manager.cc:1939] T 1e2f74e1a43242469ae646cd72708273 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.125598 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 1e2f74e1a43242469ae646cd72708273 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
I20260811 18:57:02.126673 4165 raft_consensus.cc:348] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.126919 4165 raft_consensus.cc:374] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.127002 4165 raft_consensus.cc:729] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.127310 4165 consensus_queue.cc:260] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.127489 4162 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.127794 4162 ts_tablet_manager.cc:1397] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260811 18:57:02.128909 4162 raft_consensus.cc:348] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.129117 4165 ts_tablet_manager.cc:1428] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:02.129268 4162 raft_consensus.cc:374] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.129415 4162 raft_consensus.cc:729] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.129731 4162 consensus_queue.cc:260] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.130380 4162 ts_tablet_manager.cc:1428] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:02.131891 4165 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
W20260811 18:57:02.136739 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ef0a376939d4d57b0d95db644f25dd9 has no consensus state
I20260811 18:57:02.140933 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621b56ba8f2f4392beea060871508a70" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:02.141815 3676 leader_election.cc:341] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 1e2f74e1a43242469ae646cd72708273
I20260811 18:57:02.142304 4165 tablet_bootstrap.cc:654] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:02.142411 3676 leader_election.cc:341] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 1e2f74e1a43242469ae646cd72708273
W20260811 18:57:02.142825 3676 leader_election.cc:341] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.142966 3676 leader_election.cc:302] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:02.143494 4171 raft_consensus.cc:2726] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:02.150104 4169 ts_tablet_manager.cc:1918] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.150695 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 89b5c0f3673445bbba9136a040ca9bbb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.153046 4165 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.153227 4165 ts_tablet_manager.cc:1397] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260811 18:57:02.154218 4165 raft_consensus.cc:348] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.154711 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc (DEFAULT_TABLE table=default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.154915 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc (DEFAULT_TABLE table=default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.155190 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d1b5cc176f54784ad79ad1ebd385acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.155333 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d1b5cc176f54784ad79ad1ebd385acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.155596 4009 tablet_service.cc:1467] Processing CreateTablet for tablet abb530181d7b4d2baf5391808246c74b (DEFAULT_TABLE table=default.Table_8c57cb1125a64a5da0fd42e533faec2c [id=f89b3f14c6e74c52a901945c5bfd97a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.156165 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb530181d7b4d2baf5391808246c74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.150326 4169 log.cc:1196] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/89b5c0f3673445bbba9136a040ca9bbb
I20260811 18:57:02.158454 4169 ts_tablet_manager.cc:1939] T 89b5c0f3673445bbba9136a040ca9bbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.158664 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1bd859a80441068cccd993263dc4e1 (DEFAULT_TABLE table=default.Table_485f4b3dc8ce4720a70f2d3a66648484 [id=115cebb7c7fa490f82d30693d7bf06d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.158660 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1bd859a80441068cccd993263dc4e1 (DEFAULT_TABLE table=default.Table_485f4b3dc8ce4720a70f2d3a66648484 [id=115cebb7c7fa490f82d30693d7bf06d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.159034 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1bd859a80441068cccd993263dc4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.159039 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1bd859a80441068cccd993263dc4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.153334 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 3b1bd859a80441068cccd993263dc4e1 (DEFAULT_TABLE table=default.Table_485f4b3dc8ce4720a70f2d3a66648484 [id=115cebb7c7fa490f82d30693d7bf06d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.159583 4169 tablet_replica.cc:330] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.159803 4169 raft_consensus.cc:2229] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.160035 4169 raft_consensus.cc:2258] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.160661 3866 tablet_service.cc:1467] Processing CreateTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b (DEFAULT_TABLE table=default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.161141 3863 tablet_service.cc:1467] Processing CreateTablet for tablet abb530181d7b4d2baf5391808246c74b (DEFAULT_TABLE table=default.Table_8c57cb1125a64a5da0fd42e533faec2c [id=f89b3f14c6e74c52a901945c5bfd97a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.162182 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:02.162639 4014 tablet_service.cc:1467] Processing CreateTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b (DEFAULT_TABLE table=default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.162668 3724 tablet_service.cc:1467] Processing CreateTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b (DEFAULT_TABLE table=default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.163028 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e51b07e1104bc9bbcb429d29ac590b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.162945 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc (DEFAULT_TABLE table=default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.163053 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e51b07e1104bc9bbcb429d29ac590b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.159809 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b1bd859a80441068cccd993263dc4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.164898 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb530181d7b4d2baf5391808246c74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.166680 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 7b227bc7305f4b2dae28f04e7001553b (DEFAULT_TABLE table=default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.166697 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 7b227bc7305f4b2dae28f04e7001553b (DEFAULT_TABLE table=default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.167101 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b227bc7305f4b2dae28f04e7001553b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.167101 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b227bc7305f4b2dae28f04e7001553b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.169695 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d1b5cc176f54784ad79ad1ebd385acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.170922 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e51b07e1104bc9bbcb429d29ac590b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.170842 3722 tablet_service.cc:1467] Processing CreateTablet for tablet abb530181d7b4d2baf5391808246c74b (DEFAULT_TABLE table=default.Table_8c57cb1125a64a5da0fd42e533faec2c [id=f89b3f14c6e74c52a901945c5bfd97a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.171285 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb530181d7b4d2baf5391808246c74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.170928 4171 raft_consensus.cc:482] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.174482 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0a4ee223798a47db804a96036f506e0c" } modify_external_catalogs: true
I20260811 18:57:02.174504 4171 raft_consensus.cc:504] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.174799 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0a4ee223798a47db804a96036f506e0c" } modify_external_catalogs: true
I20260811 18:57:02.175290 4171 leader_election.cc:288] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.175980 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637bb23ea30c4de0a6fe23f7aaf70c25" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:02.176265 4036 raft_consensus.cc:2454] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:02.176776 3678 leader_election.cc:302] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.177246 4171 raft_consensus.cc:2781] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.177347 4171 raft_consensus.cc:482] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.177443 4171 raft_consensus.cc:3037] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.177467 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 39a040ea35974f709a7369080925d470 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.178642 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 90175b8fc5e34f40adb97de7f877ebc4 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.180233 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 621b56ba8f2f4392beea060871508a70 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.182713 4171 raft_consensus.cc:504] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.183183 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 7b227bc7305f4b2dae28f04e7001553b (DEFAULT_TABLE table=default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.183840 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b227bc7305f4b2dae28f04e7001553b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.184190 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637bb23ea30c4de0a6fe23f7aaf70c25" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:02.184425 4036 raft_consensus.cc:3037] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.186465 4171 leader_election.cc:288] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.187565 4036 raft_consensus.cc:2454] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:02.188064 3678 leader_election.cc:302] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.188514 4171 raft_consensus.cc:2781] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.188731 4171 raft_consensus.cc:686] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:02.189065 4171 consensus_queue.cc:237] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.192171 4163 tablet_bootstrap.cc:654] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.193240 4165 raft_consensus.cc:374] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.193389 4165 raft_consensus.cc:729] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.193661 4165 consensus_queue.cc:260] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.197734 4165 ts_tablet_manager.cc:1428] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:02.198122 4165 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.204409 4165 tablet_bootstrap.cc:654] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.208474 4162 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.209066 4165 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.209246 4165 ts_tablet_manager.cc:1397] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260811 18:57:02.210124 4165 raft_consensus.cc:348] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.210426 4165 raft_consensus.cc:374] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.210528 4165 raft_consensus.cc:729] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.210775 4162 tablet_bootstrap.cc:654] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.210901 4165 consensus_queue.cc:260] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:02.210995 3676 leader_election.cc:341] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 637bb23ea30c4de0a6fe23f7aaf70c25
W20260811 18:57:02.211489 3676 leader_election.cc:341] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:02.212826 4165 ts_tablet_manager.cc:1428] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:02.214193 4165 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.216434 4165 tablet_bootstrap.cc:654] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.223841 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:02.224457 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:02.225481 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.225657 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 621b56ba8f2f4392beea060871508a70
I20260811 18:57:02.225801 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.226089 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 621b56ba8f2f4392beea060871508a70 deleted and replaced by tablet f0f19cee5abf45a4a526809f56c7a9c7
I20260811 18:57:02.227061 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.227391 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.227602 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b56ba8f2f4392beea060871508a70 with delete_type TABLET_DATA_DELETED (replaced by tablet f0f19cee5abf45a4a526809f56c7a9c7) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.227556 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.227816 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b56ba8f2f4392beea060871508a70 with delete_type TABLET_DATA_DELETED (replaced by tablet f0f19cee5abf45a4a526809f56c7a9c7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.228073 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.228156 4168 tablet_replica.cc:330] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.228482 4168 raft_consensus.cc:2229] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.228696 4168 raft_consensus.cc:2258] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.229305 4167 tablet_replica.cc:330] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.229542 4167 raft_consensus.cc:2229] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.229745 4167 raft_consensus.cc:2258] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.229898 4162 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.230051 4162 ts_tablet_manager.cc:1397] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.009s
W20260811 18:57:02.230726 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 7d1b5cc176f54784ad79ad1ebd385acc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1b5cc176f54784ad79ad1ebd385acc already in progress: creating tablet
W20260811 18:57:02.231349 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7d1b5cc176f54784ad79ad1ebd385acc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1b5cc176f54784ad79ad1ebd385acc already in progress: creating tablet
I20260811 18:57:02.231383 4168 ts_tablet_manager.cc:1905] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.231973 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b56ba8f2f4392beea060871508a70 with delete_type TABLET_DATA_DELETED (replaced by tablet f0f19cee5abf45a4a526809f56c7a9c7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.232509 4167 ts_tablet_manager.cc:1905] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.232748 4165 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.232964 4165 ts_tablet_manager.cc:1397] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.010s
I20260811 18:57:02.233867 4165 raft_consensus.cc:348] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.234155 4165 raft_consensus.cc:374] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.234247 4165 raft_consensus.cc:729] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.234555 4165 consensus_queue.cc:260] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.235885 4165 ts_tablet_manager.cc:1428] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:02.236274 4165 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.238386 4165 tablet_bootstrap.cc:654] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.239768 4168 ts_tablet_manager.cc:1918] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.240013 4168 log.cc:1196] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:02.244163 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5c59d98631a74a07a2d35f6b4bc209a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.245074 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c59d98631a74a07a2d35f6b4bc209a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.246523 4162 raft_consensus.cc:348] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.246840 4162 raft_consensus.cc:374] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.246945 4162 raft_consensus.cc:729] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.247123 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9f924af711624a8298197fae459c7e15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.247882 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f924af711624a8298197fae459c7e15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.249208 4167 ts_tablet_manager.cc:1918] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.247123 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ad98d202f325429eb1d20960973e3e0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.249529 4167 log.cc:1196] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/621b56ba8f2f4392beea060871508a70
W20260811 18:57:02.249823 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad98d202f325429eb1d20960973e3e0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.249996 4167 ts_tablet_manager.cc:1939] T 621b56ba8f2f4392beea060871508a70 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.250535 4162 consensus_queue.cc:260] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.251154 4168 ts_tablet_manager.cc:1939] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.251946 4162 ts_tablet_manager.cc:1428] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.022s user 0.000s sys 0.003s
I20260811 18:57:02.254660 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (table default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]) successfully deleted
I20260811 18:57:02.255151 4167 tablet_replica.cc:330] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.255246 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 621b56ba8f2f4392beea060871508a70 (table default.Table_59f1f5420c904158829be2c9a3524f26 [id=7131807af30d498e993d5343a99c9a6c]) successfully deleted
I20260811 18:57:02.255789 4162 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.257359 4168 tablet_replica.cc:330] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.257740 4162 tablet_bootstrap.cc:654] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.258050 4168 raft_consensus.cc:2229] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.261549 4168 raft_consensus.cc:2258] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.263726 4168 ts_tablet_manager.cc:1905] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.271518 4162 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.271756 4162 ts_tablet_manager.cc:1397] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.002s
I20260811 18:57:02.272677 4162 raft_consensus.cc:348] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.272987 4162 raft_consensus.cc:374] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.273101 4162 raft_consensus.cc:729] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.273425 4162 consensus_queue.cc:260] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.273561 4009 tablet_service.cc:1467] Processing CreateTablet for tablet dd465168aecf43a1b08b9c11d241f898 (DEFAULT_TABLE table=default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.274461 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd465168aecf43a1b08b9c11d241f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.274515 4162 ts_tablet_manager.cc:1428] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:02.274415 3721 tablet_service.cc:1467] Processing CreateTablet for tablet dd465168aecf43a1b08b9c11d241f898 (DEFAULT_TABLE table=default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.274906 4162 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.275038 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd465168aecf43a1b08b9c11d241f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.275835 4165 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.276015 4165 ts_tablet_manager.cc:1397] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.011s
I20260811 18:57:02.278589 4165 raft_consensus.cc:348] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.278896 4165 raft_consensus.cc:374] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.280207 4165 raft_consensus.cc:729] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.280571 4165 consensus_queue.cc:260] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.285225 4165 ts_tablet_manager.cc:1428] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260811 18:57:02.287212 4168 ts_tablet_manager.cc:1918] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.291422 4168 log.cc:1196] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/621b56ba8f2f4392beea060871508a70
I20260811 18:57:02.291981 4168 ts_tablet_manager.cc:1939] T 621b56ba8f2f4392beea060871508a70 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.291263 4165 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.292941 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd (DEFAULT_TABLE table=default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.293406 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f43a7646be343cda0e8f5ad1cfb76cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.294889 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 621b56ba8f2f4392beea060871508a70 (table default.Table_59f1f5420c904158829be2c9a3524f26 [id=7131807af30d498e993d5343a99c9a6c]) successfully deleted
I20260811 18:57:02.295173 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 41558efe0344434db6e755804442d2de (DEFAULT_TABLE table=default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.295681 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41558efe0344434db6e755804442d2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.296341 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd (DEFAULT_TABLE table=default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.296806 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f43a7646be343cda0e8f5ad1cfb76cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.297899 4162 tablet_bootstrap.cc:654] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.300372 4183 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.300611 4183 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.301988 4165 tablet_bootstrap.cc:654] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.303011 4183 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:02.307781 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 41558efe0344434db6e755804442d2de (DEFAULT_TABLE table=default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.308295 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41558efe0344434db6e755804442d2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.313516 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd (DEFAULT_TABLE table=default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.318670 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 34aeab84310d49c79805ea898b4f87ce (DEFAULT_TABLE table=default.Table_e02d62ef6e5947918f7fbcdd1c2fd560 [id=7cda514ae5f24bf785720ec89d44b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.319255 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34aeab84310d49c79805ea898b4f87ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.325282 4178 raft_consensus.cc:482] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.325011 3870 tablet_service.cc:1467] Processing CreateTablet for tablet dd465168aecf43a1b08b9c11d241f898 (DEFAULT_TABLE table=default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.327023 4012 tablet_service.cc:1467] Processing CreateTablet for tablet e355634612f64c14965d7af2fa442cb7 (DEFAULT_TABLE table=default.Table_3f2571e177d44cc8bcbea5387c116638 [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.328150 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e355634612f64c14965d7af2fa442cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.325615 4178 raft_consensus.cc:504] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.328637 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 34aeab84310d49c79805ea898b4f87ce (DEFAULT_TABLE table=default.Table_e02d62ef6e5947918f7fbcdd1c2fd560 [id=7cda514ae5f24bf785720ec89d44b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.329036 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f43a7646be343cda0e8f5ad1cfb76cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.329640 4162 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.329831 4162 ts_tablet_manager.cc:1397] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.005s
I20260811 18:57:02.330801 4162 raft_consensus.cc:348] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.331065 4162 raft_consensus.cc:374] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.331178 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34aeab84310d49c79805ea898b4f87ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.331192 4162 raft_consensus.cc:729] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.331485 4162 consensus_queue.cc:260] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.332506 4162 ts_tablet_manager.cc:1428] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:02.332854 4162 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.335047 4162 tablet_bootstrap.cc:654] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.335660 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_34ca03d0c0144111b5d797fed15d35a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.336566 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34ca03d0c0144111b5d797fed15d35a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.339119 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3444887e982747ff9c397d48da9ae8c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.339717 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3444887e982747ff9c397d48da9ae8c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.345079 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe (DEFAULT_TABLE table=default.Table_f76c62e348114008bdd5b3b8a5697df7 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.345532 4169 ts_tablet_manager.cc:1905] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.340279 4178 leader_election.cc:288] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.347766 3547 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f0f19cee5abf45a4a526809f56c7a9c7
I20260811 18:57:02.348359 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_84792678a95b4b7a9ceffcde9ddf3fdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.349059 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84792678a95b4b7a9ceffcde9ddf3fdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.349998 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65f9d97ef0514e40893e87f1885ad5f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.350607 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65f9d97ef0514e40893e87f1885ad5f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.350805 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:02.351653 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6300adb327414f04a6ef25bfe7da7e02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.352243 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6300adb327414f04a6ef25bfe7da7e02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.352415 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd465168aecf43a1b08b9c11d241f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.354681 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_62a6e4fda91c4b1bad9616d8f8dcdb23" } new_table_name: "default.Table_a3015128d6b741138d21edcf5a9c5785" modify_external_catalogs: true
I20260811 18:57:02.356805 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.356801 4196 raft_consensus.cc:482] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.357206 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.357056 4196 raft_consensus.cc:504] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.358116 4196 leader_election.cc:288] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.358327 4165 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.358593 4165 ts_tablet_manager.cc:1397] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.067s user 0.004s sys 0.007s
I20260811 18:57:02.359535 4165 raft_consensus.cc:348] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.359807 4165 raft_consensus.cc:374] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.359874 4165 raft_consensus.cc:729] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.360216 4165 consensus_queue.cc:260] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.362311 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e584229a7f164fc8be8d7963583c65c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.362996 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e584229a7f164fc8be8d7963583c65c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.362996 4165 ts_tablet_manager.cc:1428] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:02.364856 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_433dca44f8124f6db8b076cf7816ec52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.365376 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_433dca44f8124f6db8b076cf7816ec52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.367794 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 39a040ea35974f709a7369080925d470
I20260811 18:57:02.367683 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e51b07e1104bc9bbcb429d29ac590b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:02.368253 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 39a040ea35974f709a7369080925d470 deleted and replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:02.368851 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e02d62ef6e5947918f7fbcdd1c2fd560" } new_table_name: "default.Table_738a85e77f9e49efa9899428ec31a9ab" modify_external_catalogs: true
I20260811 18:57:02.369149 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a040ea35974f709a7369080925d470 with delete_type TABLET_DATA_DELETED (replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:02.369927 3676 leader_election.cc:341] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.370416 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a1e51b07e1104bc9bbcb429d29ac590b
I20260811 18:57:02.371160 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_485f4b3dc8ce4720a70f2d3a66648484" } new_table_name: "default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c" modify_external_catalogs: true
I20260811 18:57:02.371438 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.372661 3865 tablet_service.cc:1467] Processing CreateTablet for tablet e355634612f64c14965d7af2fa442cb7 (DEFAULT_TABLE table=default.Table_3f2571e177d44cc8bcbea5387c116638 [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.373376 4162 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.373335 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:02.373574 4162 ts_tablet_manager.cc:1397] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.002s
W20260811 18:57:02.374228 3965 leader_election.cc:341] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.374451 4162 raft_consensus.cc:348] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.374720 4162 raft_consensus.cc:374] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.374814 4162 raft_consensus.cc:729] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.375082 4162 consensus_queue.cc:260] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.375402 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e355634612f64c14965d7af2fa442cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.375730 4162 ts_tablet_manager.cc:1428] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:02.377270 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e51b07e1104bc9bbcb429d29ac590b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:02.377503 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d1339366a24a87b74bd1d411a3f6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.377553 4036 raft_consensus.cc:2454] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:02.378172 3678 leader_election.cc:302] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.378691 4196 raft_consensus.cc:2781] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.378818 4196 raft_consensus.cc:482] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:02.378870 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 1e2f74e1a43242469ae646cd72708273 because the tablet was not found. No further retry: Not found: Tablet not found: 1e2f74e1a43242469ae646cd72708273
I20260811 18:57:02.378924 4196 raft_consensus.cc:3037] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.376058 4162 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.380097 4168 tablet_replica.cc:330] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.380321 4168 raft_consensus.cc:2229] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.380497 4168 raft_consensus.cc:2258] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.381183 4162 tablet_bootstrap.cc:654] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.382000 4196 raft_consensus.cc:504] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.383476 4196 leader_election.cc:288] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:02.384534 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9de9acdb33a2438eb4378a5eac844c35 consensus state has no leader
I20260811 18:57:02.384786 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e51b07e1104bc9bbcb429d29ac590b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:02.385047 4036 raft_consensus.cc:3037] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.386145 4014 tablet_service.cc:1467] Processing CreateTablet for tablet d24799ded256403e9fad5c1a22b8aa80 (DEFAULT_TABLE table=default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.386394 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c57cb1125a64a5da0fd42e533faec2c" } new_table_name: "default.Table_29c83b98881f4edaba5330f38e7a67ed" modify_external_catalogs: true
I20260811 18:57:02.386691 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 927fb981490b4cecabe40314d6f71048 (DEFAULT_TABLE table=default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.387439 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ecacdcdd1638424aa0a8cd6298159e98" } new_table_name: "default.Table_c038c8ee6e694c6f99d5cb717ba90ca9" modify_external_catalogs: true
I20260811 18:57:02.387993 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 7d1b5cc176f54784ad79ad1ebd385acc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.389422 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d24799ded256403e9fad5c1a22b8aa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.389613 4036 raft_consensus.cc:2454] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:02.389813 4168 ts_tablet_manager.cc:1905] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.390307 3678 leader_election.cc:302] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.390749 4196 raft_consensus.cc:2781] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.393762 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fb981490b4cecabe40314d6f71048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.395835 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 (DEFAULT_TABLE table=default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.396349 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef0a376939d4d57b0d95db644f25dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.398732 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3414c134374aa59ac23d276a018ec9 (DEFAULT_TABLE table=default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.399124 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3414c134374aa59ac23d276a018ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.399924 4167 raft_consensus.cc:2229] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:02.400893 4167 pending_rounds.cc:62] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:02.401047 4167 pending_rounds.cc:68] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400052489834496 op_type: NO_OP noop_request { }
I20260811 18:57:02.401355 4167 raft_consensus.cc:2866] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:02.401491 4167 raft_consensus.cc:2258] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.403076 4196 raft_consensus.cc:686] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:02.403393 4196 consensus_queue.cc:237] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.406652 4219 raft_consensus.cc:482] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.406852 4219 raft_consensus.cc:504] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.409821 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 41558efe0344434db6e755804442d2de (DEFAULT_TABLE table=default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.410322 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41558efe0344434db6e755804442d2de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:02.410519 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 34aeab84310d49c79805ea898b4f87ce consensus state has no leader
I20260811 18:57:02.418077 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 34aeab84310d49c79805ea898b4f87ce (DEFAULT_TABLE table=default.Table_e02d62ef6e5947918f7fbcdd1c2fd560 [id=7cda514ae5f24bf785720ec89d44b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.418686 3721 tablet_service.cc:1467] Processing CreateTablet for tablet e355634612f64c14965d7af2fa442cb7 (DEFAULT_TABLE table=default.Table_3f2571e177d44cc8bcbea5387c116638 [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.419608 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.421154 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a040ea35974f709a7369080925d470" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:02.421412 4036 raft_consensus.cc:2454] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:02.421870 3678 leader_election.cc:302] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:02.421933 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b1bd859a80441068cccd993263dc4e1 consensus state has no leader
I20260811 18:57:02.422484 4171 raft_consensus.cc:2781] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.422631 4171 raft_consensus.cc:482] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.422792 4171 raft_consensus.cc:3037] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.425905 4171 raft_consensus.cc:504] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.430895 4168 ts_tablet_manager.cc:1918] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.431097 4168 log.cc:1196] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:02.431526 4168 ts_tablet_manager.cc:1939] T 4ae2221b54a44f6db10f6c98ee2344a3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.432773 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 4ae2221b54a44f6db10f6c98ee2344a3 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
I20260811 18:57:02.433387 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_433dca44f8124f6db8b076cf7816ec52" } modify_external_catalogs: true
I20260811 18:57:02.433658 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_433dca44f8124f6db8b076cf7816ec52" } modify_external_catalogs: true
I20260811 18:57:02.435206 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_efd83594c1f04b33b212055bafe30b88" } modify_external_catalogs: true
I20260811 18:57:02.435444 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_efd83594c1f04b33b212055bafe30b88" } modify_external_catalogs: true
I20260811 18:57:02.442104 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a040ea35974f709a7369080925d470" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:02.443631 4165 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.445291 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e51b07e1104bc9bbcb429d29ac590b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:02.445474 4165 tablet_bootstrap.cc:654] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.448796 4168 tablet_replica.cc:330] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.449036 4168 raft_consensus.cc:2229] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.449375 4168 raft_consensus.cc:2258] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.450268 4168 ts_tablet_manager.cc:1905] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:02.450577 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet abb530181d7b4d2baf5391808246c74b consensus state has no leader
I20260811 18:57:02.452942 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet a1e51b07e1104bc9bbcb429d29ac590b from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.453533 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 637bb23ea30c4de0a6fe23f7aaf70c25 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.455431 4165 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.455672 4165 ts_tablet_manager.cc:1397] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260811 18:57:02.456620 4165 raft_consensus.cc:348] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.457019 4165 raft_consensus.cc:374] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.457196 4165 raft_consensus.cc:729] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.457505 4165 consensus_queue.cc:260] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.458626 4168 ts_tablet_manager.cc:1918] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.458829 4168 log.cc:1196] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/39a040ea35974f709a7369080925d470
I20260811 18:57:02.459329 4168 ts_tablet_manager.cc:1939] T 39a040ea35974f709a7369080925d470 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.459548 4169 ts_tablet_manager.cc:1918] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.460001 4169 log.cc:1196] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/90175b8fc5e34f40adb97de7f877ebc4
I20260811 18:57:02.460745 3549 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_a3015128d6b741138d21edcf5a9c5785 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 3 got 0
I20260811 18:57:02.461014 3549 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 1 got 0
I20260811 18:57:02.462476 3551 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 312f7b8df2c34e74b1e86cd96f275465
W20260811 18:57:02.462666 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cd3b82603ec340ed9cabe9b2ff62a8b3 consensus state has no leader
I20260811 18:57:02.467584 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 634c06d76eca404a88f326e905882930 (DEFAULT_TABLE table=default.Table_ad98d202f325429eb1d20960973e3e0a [id=93a135134c1542eb8b5031fc7f223526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.468040 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.467998 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.468128 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634c06d76eca404a88f326e905882930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.468205 4169 ts_tablet_manager.cc:1939] T 90175b8fc5e34f40adb97de7f877ebc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.468392 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 (DEFAULT_TABLE table=default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.468499 4010 tablet_service.cc:1467] Processing CreateTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.469108 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3085d74b0bf7447d9c247b7e8872b215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.469229 4168 tablet_replica.cc:330] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.469441 4169 tablet_replica.cc:330] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.469647 4169 raft_consensus.cc:2229] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.469676 4168 raft_consensus.cc:2229] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.469822 4169 raft_consensus.cc:2258] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.469879 4168 raft_consensus.cc:2258] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.471046 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 9c18120da2d245f495c52a670047760d (DEFAULT_TABLE table=default.Table_9f924af711624a8298197fae459c7e15 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.471506 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c18120da2d245f495c52a670047760d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.473788 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3b82603ec340ed9cabe9b2ff62a8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.473896 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 39a040ea35974f709a7369080925d470 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:02.477468 4163 tablet_bootstrap.cc:492] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.477663 4163 ts_tablet_manager.cc:1397] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.383s user 0.013s sys 0.017s
I20260811 18:57:02.478133 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:02.478423 4165 ts_tablet_manager.cc:1428] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260811 18:57:02.478750 4165 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.478612 4163 raft_consensus.cc:348] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.478874 4163 raft_consensus.cc:374] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.478962 4163 raft_consensus.cc:729] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.479205 4163 consensus_queue.cc:260] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.479760 4163 ts_tablet_manager.cc:1428] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:02.480123 4163 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.481880 4163 tablet_bootstrap.cc:654] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.486663 4168 ts_tablet_manager.cc:1905] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.486774 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet dd465168aecf43a1b08b9c11d241f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.426509 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 90175b8fc5e34f40adb97de7f877ebc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:01 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.427155 4219 leader_election.cc:288] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.427708 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b56ba8f2f4392beea060871508a70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.428894 4167 ts_tablet_manager.cc:1905] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.429596 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2f74e1a43242469ae646cd72708273 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ae2221b54a44f6db10f6c98ee2344a3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.430282 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a040ea35974f709a7369080925d470 with delete_type TABLET_DATA_DELETED (replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.431005 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.432440 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe (DEFAULT_TABLE table=default.Table_f76c62e348114008bdd5b3b8a5697df7 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.433528 3716 tablet_service.cc:1467] Processing CreateTablet for tablet d24799ded256403e9fad5c1a22b8aa80 (DEFAULT_TABLE table=default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.490932 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 90175b8fc5e34f40adb97de7f877ebc4 (table default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]) successfully deleted
I20260811 18:57:02.434669 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 927fb981490b4cecabe40314d6f71048 (DEFAULT_TABLE table=default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.435842 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 (DEFAULT_TABLE table=default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.437043 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3414c134374aa59ac23d276a018ec9 (DEFAULT_TABLE table=default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.418658 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34aeab84310d49c79805ea898b4f87ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:02.494045 3678 leader_election.cc:341] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 39a040ea35974f709a7369080925d470
W20260811 18:57:02.443049 3676 leader_election.cc:341] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.494370 4169 ts_tablet_manager.cc:1905] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:02.494465 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 51449.7us
W20260811 18:57:02.494997 3676 leader_election.cc:341] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.444161 4162 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.495375 4162 ts_tablet_manager.cc:1397] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.009s sys 0.016s
I20260811 18:57:02.496335 3550 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 3b1bd859a80441068cccd993263dc4e1 (table default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c [id=115cebb7c7fa490f82d30693d7bf06d4]). Expected version 1 got 0
I20260811 18:57:02.496253 4162 raft_consensus.cc:348] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.496526 4162 raft_consensus.cc:374] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.496619 4162 raft_consensus.cc:729] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.497534 4162 consensus_queue.cc:260] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.498950 4162 ts_tablet_manager.cc:1428] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:02.499013 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef0a376939d4d57b0d95db644f25dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.499804 4162 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.501925 3550 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 3b1bd859a80441068cccd993263dc4e1 (table default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c [id=115cebb7c7fa490f82d30693d7bf06d4]). Expected version 1 got 0
I20260811 18:57:02.502550 4162 tablet_bootstrap.cc:654] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.502622 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3414c134374aa59ac23d276a018ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.500810 4171 leader_election.cc:288] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:02.505498 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:02.505925 3746 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:02.506490 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.506561 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fb981490b4cecabe40314d6f71048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.507704 3872 tablet_service.cc:1467] Processing CreateTablet for tablet d24799ded256403e9fad5c1a22b8aa80 (DEFAULT_TABLE table=default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.508176 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d24799ded256403e9fad5c1a22b8aa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.510330 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d24799ded256403e9fad5c1a22b8aa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.511358 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3414c134374aa59ac23d276a018ec9 (DEFAULT_TABLE table=default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.511655 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 9c18120da2d245f495c52a670047760d (DEFAULT_TABLE table=default.Table_9f924af711624a8298197fae459c7e15 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.512391 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 634c06d76eca404a88f326e905882930 (DEFAULT_TABLE table=default.Table_ad98d202f325429eb1d20960973e3e0a [id=93a135134c1542eb8b5031fc7f223526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.512874 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634c06d76eca404a88f326e905882930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.514848 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 927fb981490b4cecabe40314d6f71048 (DEFAULT_TABLE table=default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.515591 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 (DEFAULT_TABLE table=default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.516249 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 634c06d76eca404a88f326e905882930 (DEFAULT_TABLE table=default.Table_ad98d202f325429eb1d20960973e3e0a [id=93a135134c1542eb8b5031fc7f223526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.517103 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 (DEFAULT_TABLE table=default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.517589 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3085d74b0bf7447d9c247b7e8872b215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.518289 3860 tablet_service.cc:1467] Processing CreateTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.518770 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet dd465168aecf43a1b08b9c11d241f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.519265 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet dd465168aecf43a1b08b9c11d241f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.519452 4168 ts_tablet_manager.cc:1918] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.519650 4168 log.cc:1196] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/a1e51b07e1104bc9bbcb429d29ac590b
I20260811 18:57:02.519608 3720 tablet_service.cc:1467] Processing CreateTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (DEFAULT_TABLE table=default.Table_ecacdcdd1638424aa0a8cd6298159e98 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.511766 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3414c134374aa59ac23d276a018ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.520073 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3b82603ec340ed9cabe9b2ff62a8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.520061 4168 ts_tablet_manager.cc:1939] T a1e51b07e1104bc9bbcb429d29ac590b P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.521237 4168 tablet_replica.cc:330] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.521440 4168 raft_consensus.cc:2229] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.521620 4168 raft_consensus.cc:2258] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.522966 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 9c18120da2d245f495c52a670047760d (DEFAULT_TABLE table=default.Table_9f924af711624a8298197fae459c7e15 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.523527 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c18120da2d245f495c52a670047760d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.526590 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d1339366a24a87b74bd1d411a3f6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.526638 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 (DEFAULT_TABLE table=default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.527032 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 494c09652d9e408880ca606ae0b30773 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.527237 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3085d74b0bf7447d9c247b7e8872b215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.527510 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c09652d9e408880ca606ae0b30773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.530625 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 (DEFAULT_TABLE table=default.Table_65f9d97ef0514e40893e87f1885ad5f2 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.531133 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1d58835ad24749a00ce27b4bc73c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.532534 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634c06d76eca404a88f326e905882930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.532601 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 8efb663060ef457cbece962fce701eb1 (DEFAULT_TABLE table=default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.533092 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8efb663060ef457cbece962fce701eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.535142 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 8964f92255884b3aae0cac60f019fdb7 (DEFAULT_TABLE table=default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.535513 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8964f92255884b3aae0cac60f019fdb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.538591 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b (DEFAULT_TABLE table=default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.538802 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef0a376939d4d57b0d95db644f25dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.539055 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df7a07983174dd1aaf48951c9ff1a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.539306 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e355634612f64c14965d7af2fa442cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.544091 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_68b0781f65bb450a9b74324ca7d6d5ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.544808 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68b0781f65bb450a9b74324ca7d6d5ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.545228 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5332d093ac81423fa33cb860f7d9a460"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.545646 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 494c09652d9e408880ca606ae0b30773 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:02.545737 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5332d093ac81423fa33cb860f7d9a460 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.546137 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c09652d9e408880ca606ae0b30773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.546662 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 08b8dcf7a56043efbbd40bb888616492 (DEFAULT_TABLE table=default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.546841 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b (DEFAULT_TABLE table=default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.547066 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b8dcf7a56043efbbd40bb888616492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.547421 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 8964f92255884b3aae0cac60f019fdb7 (DEFAULT_TABLE table=default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.547773 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8964f92255884b3aae0cac60f019fdb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.550535 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df7a07983174dd1aaf48951c9ff1a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.551184 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet a1e51b07e1104bc9bbcb429d29ac590b (table default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]) successfully deleted
W20260811 18:57:02.551481 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 4ae2221b54a44f6db10f6c98ee2344a3 because the tablet was not found. No further retry: Not found: Tablet not found: 4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:02.546072 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 8efb663060ef457cbece962fce701eb1 (DEFAULT_TABLE table=default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.554880 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8efb663060ef457cbece962fce701eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.556021 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fb981490b4cecabe40314d6f71048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.556702 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 08b8dcf7a56043efbbd40bb888616492 (DEFAULT_TABLE table=default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.557382 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b8dcf7a56043efbbd40bb888616492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.546146 3967 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.562045 4183 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.562171 4183 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.562254 4183 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.562886 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 (DEFAULT_TABLE table=default.Table_65f9d97ef0514e40893e87f1885ad5f2 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.563401 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1d58835ad24749a00ce27b4bc73c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.565837 4183 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.566612 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c18120da2d245f495c52a670047760d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.567813 4183 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:02.570174 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd3b82603ec340ed9cabe9b2ff62a8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.575490 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.576380 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe (DEFAULT_TABLE table=default.Table_f76c62e348114008bdd5b3b8a5697df7 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.576958 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82d1339366a24a87b74bd1d411a3f6fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:02.577894 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet abb530181d7b4d2baf5391808246c74b consensus state has no leader
I20260811 18:57:02.596251 4169 ts_tablet_manager.cc:1918] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.597319 4169 log.cc:1196] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:02.598385 4169 ts_tablet_manager.cc:1939] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.601128 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ce825a4b0853440090c12cf6387d94b4 (table default.Table_4df58e6e7b7941c1b397a058da5823c1 [id=ef61e3beec2246fa83002f3e24cda248]) successfully deleted
I20260811 18:57:02.607030 4165 tablet_bootstrap.cc:654] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.608664 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a040ea35974f709a7369080925d470 with delete_type TABLET_DATA_DELETED (replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:02.609287 3824 leader_election.cc:341] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:02.610029 4219 raft_consensus.cc:482] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.611558 4219 raft_consensus.cc:504] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.612675 4219 leader_election.cc:288] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:02.616842 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1bd859a80441068cccd993263dc4e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:02.617128 4036 raft_consensus.cc:2454] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:02.618734 3824 leader_election.cc:341] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: ce825a4b0853440090c12cf6387d94b4
I20260811 18:57:02.619025 3824 leader_election.cc:302] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:02.619266 4163 tablet_bootstrap.cc:492] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.619485 4163 ts_tablet_manager.cc:1397] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.139s user 0.023s sys 0.047s
W20260811 18:57:02.619544 3824 raft_consensus.cc:2670] T ce825a4b0853440090c12cf6387d94b4 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:02.622781 4169 tablet_replica.cc:330] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.622988 4169 raft_consensus.cc:2229] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:02.623870 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7d1b5cc176f54784ad79ad1ebd385acc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d1b5cc176f54784ad79ad1ebd385acc already in progress: creating tablet
I20260811 18:57:02.623991 4169 raft_consensus.cc:2258] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:02.626220 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 637bb23ea30c4de0a6fe23f7aaf70c25 already in progress: creating tablet
I20260811 18:57:02.620720 4163 raft_consensus.cc:348] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.627465 3678 leader_election.cc:302] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.627731 4163 raft_consensus.cc:374] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.627935 4163 raft_consensus.cc:729] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.628320 4163 consensus_queue.cc:260] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.629216 4163 ts_tablet_manager.cc:1428] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260811 18:57:02.629622 4163 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.631687 4219 raft_consensus.cc:2781] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.631966 4219 raft_consensus.cc:482] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.632301 4165 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.632516 4165 ts_tablet_manager.cc:1397] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.154s user 0.004s sys 0.000s
I20260811 18:57:02.633244 4219 raft_consensus.cc:3037] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.633453 4165 raft_consensus.cc:348] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.633809 4165 raft_consensus.cc:374] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.633921 4165 raft_consensus.cc:729] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.634233 4165 consensus_queue.cc:260] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.636109 4165 ts_tablet_manager.cc:1428] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:02.639405 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:02.640079 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a040ea35974f709a7369080925d470" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:02.640486 3893 raft_consensus.cc:3037] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.641186 4165 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
W20260811 18:57:02.643181 3965 leader_election.cc:341] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.643916 4013 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.644289 4168 ts_tablet_manager.cc:1905] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.644400 4165 tablet_bootstrap.cc:654] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.644482 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:38889
--webserver_interface=127.3.72.125
--webserver_port=46243
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:57:02.647024 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1bd859a80441068cccd993263dc4e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:02.652845 3676 leader_election.cc:341] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.655066 4169 ts_tablet_manager.cc:1905] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.659077 4219 raft_consensus.cc:504] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.664922 4167 ts_tablet_manager.cc:1918] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:02.666757 4167 log.cc:1196] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:02.667423 4167 ts_tablet_manager.cc:1939] T 637bb23ea30c4de0a6fe23f7aaf70c25 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.668231 3893 raft_consensus.cc:2454] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:02.672981 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (table default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]) successfully deleted
I20260811 18:57:02.674571 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3b1bd859a80441068cccd993263dc4e1 (table default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c [id=115cebb7c7fa490f82d30693d7bf06d4]). Expected version 1 got 0
I20260811 18:57:02.674829 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]). Expected version 1 got 0
I20260811 18:57:02.675016 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 1 got 0
I20260811 18:57:02.675159 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_a3015128d6b741138d21edcf5a9c5785 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 3 got 0
I20260811 18:57:02.675724 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_318043c37396470886587403377a976f" } modify_external_catalogs: true
I20260811 18:57:02.676045 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_318043c37396470886587403377a976f" } modify_external_catalogs: true
I20260811 18:57:02.677588 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet e355634612f64c14965d7af2fa442cb7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.677788 4168 ts_tablet_manager.cc:1918] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.677963 4168 log.cc:1196] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/90175b8fc5e34f40adb97de7f877ebc4
I20260811 18:57:02.678117 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fe796d2df98748a1a996fd9e9e0cf128"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.678551 4168 ts_tablet_manager.cc:1939] T 90175b8fc5e34f40adb97de7f877ebc4 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
W20260811 18:57:02.678836 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe796d2df98748a1a996fd9e9e0cf128 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.680089 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_abaf87a40e04427fba978e8a67b89f0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.680410 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34ca03d0c0144111b5d797fed15d35a6" } modify_external_catalogs: true
W20260811 18:57:02.680809 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_abaf87a40e04427fba978e8a67b89f0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.680740 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34ca03d0c0144111b5d797fed15d35a6" } modify_external_catalogs: true
I20260811 18:57:02.682488 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b1bd859a80441068cccd993263dc4e1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.683741 4163 tablet_bootstrap.cc:654] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.683892 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c" } modify_external_catalogs: true
I20260811 18:57:02.684105 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c" } modify_external_catalogs: true
I20260811 18:57:02.684762 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f76c62e348114008bdd5b3b8a5697df7" } new_table_name: "default.Table_52e96fad41744bc5bd6efbbb18e5f692" modify_external_catalogs: true
I20260811 18:57:02.696008 4163 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.696274 4163 ts_tablet_manager.cc:1397] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.003s
I20260811 18:57:02.698662 3676 leader_election.cc:302] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:02.698621 4163 raft_consensus.cc:348] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.698915 4163 raft_consensus.cc:374] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.699015 4163 raft_consensus.cc:729] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.699352 4163 consensus_queue.cc:260] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.701326 4163 ts_tablet_manager.cc:1428] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:02.701675 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:02.701947 3746 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.702168 4163 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.702526 4167 tablet_replica.cc:330] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.702798 4167 raft_consensus.cc:2229] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.703032 4167 raft_consensus.cc:2258] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.703755 4167 ts_tablet_manager.cc:1905] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.704421 4163 tablet_bootstrap.cc:654] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.704595 4169 ts_tablet_manager.cc:1918] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.704763 4169 log.cc:1196] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/621b56ba8f2f4392beea060871508a70
I20260811 18:57:02.705276 4169 ts_tablet_manager.cc:1939] T 621b56ba8f2f4392beea060871508a70 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.706447 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 (DEFAULT_TABLE table=default.Table_65f9d97ef0514e40893e87f1885ad5f2 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.706946 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 08b8dcf7a56043efbbd40bb888616492 (DEFAULT_TABLE table=default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.708376 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 494c09652d9e408880ca606ae0b30773 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.708791 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1bd859a80441068cccd993263dc4e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
W20260811 18:57:02.711782 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a1e51b07e1104bc9bbcb429d29ac590b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1e51b07e1104bc9bbcb429d29ac590b already in progress: creating tablet
W20260811 18:57:02.712836 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dd465168aecf43a1b08b9c11d241f898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd465168aecf43a1b08b9c11d241f898 already in progress: creating tablet
I20260811 18:57:02.712921 4171 raft_consensus.cc:2781] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:02.713683 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4ae2221b54a44f6db10f6c98ee2344a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ae2221b54a44f6db10f6c98ee2344a3 already in progress: creating tablet
I20260811 18:57:02.713903 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 8efb663060ef457cbece962fce701eb1 (DEFAULT_TABLE table=default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.714169 4169 tablet_replica.cc:330] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.707791 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b (DEFAULT_TABLE table=default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.683039 4165 tablet_bootstrap.cc:492] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.691287 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b1bd859a80441068cccd993263dc4e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:02.714583 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d1d58835ad24749a00ce27b4bc73c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.714609 4036 raft_consensus.cc:3037] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.714603 4165 ts_tablet_manager.cc:1397] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.074s user 0.000s sys 0.010s
I20260811 18:57:02.714745 4169 raft_consensus.cc:2229] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:02.714885 3676 leader_election.cc:341] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.714960 4169 raft_consensus.cc:2258] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.713215 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.715761 4165 raft_consensus.cc:348] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.716037 4165 raft_consensus.cc:374] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.716377 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 90175b8fc5e34f40adb97de7f877ebc4 (table default.Table_2d1dc92a1a5b4a0b803b335d7a825430 [id=5c3976e9c306494d897f27bbf8690632]) successfully deleted
I20260811 18:57:02.717178 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c09652d9e408880ca606ae0b30773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.717715 4165 raft_consensus.cc:729] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.717836 4169 ts_tablet_manager.cc:1905] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.717924 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df7a07983174dd1aaf48951c9ff1a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.718103 4165 consensus_queue.cc:260] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.718571 4183 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.718727 4183 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.719432 4183 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:02.705891 3746 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:02.720046 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8efb663060ef457cbece962fce701eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.721042 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08b8dcf7a56043efbbd40bb888616492. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:02.722280 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet dd465168aecf43a1b08b9c11d241f898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd465168aecf43a1b08b9c11d241f898 already in progress: creating tablet
I20260811 18:57:02.722652 4168 tablet_replica.cc:330] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.723075 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:02.723387 3967 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.723753 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 8964f92255884b3aae0cac60f019fdb7 (DEFAULT_TABLE table=default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:02.723713 3965 leader_election.cc:341] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.724375 4183 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.724545 4183 raft_consensus.cc:686] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:02.724572 4171 raft_consensus.cc:686] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:02.725016 4183 consensus_queue.cc:237] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.725517 4171 consensus_queue.cc:237] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.727092 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8964f92255884b3aae0cac60f019fdb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.727126 4036 raft_consensus.cc:2454] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:02.729699 4168 raft_consensus.cc:2229] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.730319 4168 raft_consensus.cc:2258] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.732172 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:02.734126 3678 leader_election.cc:302] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.734730 4168 ts_tablet_manager.cc:1905] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:02.735045 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 55365.4us
I20260811 18:57:02.735811 4219 leader_election.cc:288] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:02.736179 4196 raft_consensus.cc:2781] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.739980 3546 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]). Expected version 1 got 0
I20260811 18:57:02.741737 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1fbb53e0b2d44010bbb9da4dc739a78d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.745793 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 621b56ba8f2f4392beea060871508a70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:02.746421 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fbb53e0b2d44010bbb9da4dc739a78d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.746507 4165 ts_tablet_manager.cc:1428] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.032s user 0.000s sys 0.005s
I20260811 18:57:02.746959 4165 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.748800 4165 tablet_bootstrap.cc:654] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.756760 3746 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:02.757408 4168 ts_tablet_manager.cc:1918] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.757596 4168 log.cc:1196] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:02.758698 3967 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.759727 4234 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.759859 4234 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.759989 4234 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.761750 4162 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.762995 4234 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.764271 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:02.764462 3550 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]). Expected version 1 got 0
I20260811 18:57:02.764777 3550 catalog_manager.cc:5495] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67). New cstate: current_term: 1 leader_uuid: "59811bdf82234ef7ac5865dbecfcc388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:02.765748 4162 ts_tablet_manager.cc:1397] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.262s user 0.015s sys 0.040s
I20260811 18:57:02.765779 3550 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 1 got 0
I20260811 18:57:02.765985 3746 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.768569 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.770328 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:02.771442 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d24799ded256403e9fad5c1a22b8aa80
W20260811 18:57:02.771430 3965 leader_election.cc:341] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:02.771551 4234 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:02.772003 4169 ts_tablet_manager.cc:1918] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.772171 4169 log.cc:1196] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/39a040ea35974f709a7369080925d470
I20260811 18:57:02.772502 4163 tablet_bootstrap.cc:492] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.772676 4163 ts_tablet_manager.cc:1397] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.005s
I20260811 18:57:02.772895 4168 ts_tablet_manager.cc:1939] T 7d1b5cc176f54784ad79ad1ebd385acc P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.772975 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e355634612f64c14965d7af2fa442cb7
I20260811 18:57:02.773393 3548 catalog_manager.cc:6565] ReplaceTablet: tablet e355634612f64c14965d7af2fa442cb7 deleted and replaced by tablet b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:02.773622 4163 raft_consensus.cc:348] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.773871 4163 raft_consensus.cc:374] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.773969 4163 raft_consensus.cc:729] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.773877 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1ad9254eaa184967a0d48ef4468c434b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.774214 4163 consensus_queue.cc:260] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.775002 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab16863006cb44f5825c317ad3ecdea2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.774544 3746 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
W20260811 18:57:02.775544 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab16863006cb44f5825c317ad3ecdea2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.775827 3967 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:02.777218 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_deebcc7412e6466986509a6858c4bc70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.777736 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_deebcc7412e6466986509a6858c4bc70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.777789 4163 ts_tablet_manager.cc:1428] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:02.777788 4234 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.778970 4234 raft_consensus.cc:686] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:02.772672 4169 ts_tablet_manager.cc:1939] T 39a040ea35974f709a7369080925d470 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.779254 4234 consensus_queue.cc:237] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.780500 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3b1bd859a80441068cccd993263dc4e1
I20260811 18:57:02.780932 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 3b1bd859a80441068cccd993263dc4e1 deleted and replaced by tablet ed809a37f7864b439ec0cf276f16bd13
I20260811 18:57:02.781165 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ae2221b54a44f6db10f6c98ee2344a3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.781700 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3f2571e177d44cc8bcbea5387c116638" } new_table_name: "default.Table_524727faeeb3483298947337811873bf" modify_external_catalogs: true
I20260811 18:57:02.782685 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8964f92255884b3aae0cac60f019fdb7
I20260811 18:57:02.783075 4163 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.783377 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f30cf1eb759b4a7ab689610c645803e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.783931 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f30cf1eb759b4a7ab689610c645803e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.788035 4163 tablet_bootstrap.cc:654] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.788529 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.790529 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet d24799ded256403e9fad5c1a22b8aa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.791477 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.793454 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b1bd859a80441068cccd993263dc4e1 with delete_type TABLET_DATA_DELETED (replaced by tablet ed809a37f7864b439ec0cf276f16bd13) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.793502 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet d24799ded256403e9fad5c1a22b8aa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.793663 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 8964f92255884b3aae0cac60f019fdb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.795495 4013 tablet_service.cc:1514] Processing DeleteTablet for tablet d24799ded256403e9fad5c1a22b8aa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.795881 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.796198 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b1bd859a80441068cccd993263dc4e1 with delete_type TABLET_DATA_DELETED (replaced by tablet ed809a37f7864b439ec0cf276f16bd13) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.796419 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 8964f92255884b3aae0cac60f019fdb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.797796 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 8964f92255884b3aae0cac60f019fdb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.798072 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b1bd859a80441068cccd993263dc4e1 with delete_type TABLET_DATA_DELETED (replaced by tablet ed809a37f7864b439ec0cf276f16bd13) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.798632 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 7d1b5cc176f54784ad79ad1ebd385acc (table default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]) successfully deleted
I20260811 18:57:02.800227 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f7b3ede6cd24ee2a34f982ff0a93cdf" } new_table_name: "default.Table_14ea27ae9206400f82d35d43e532e99b" modify_external_catalogs: true
I20260811 18:57:02.800663 4162 raft_consensus.cc:348] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.800949 4162 raft_consensus.cc:374] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.801010 4162 raft_consensus.cc:729] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.802057 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ad98d202f325429eb1d20960973e3e0a" } new_table_name: "default.Table_de7a638d5c564c84b20b870f9c3a97e5" modify_external_catalogs: true
I20260811 18:57:02.802992 4162 consensus_queue.cc:260] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.804677 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4fbcb57a32f54fc3a493bc5412e94051"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.805272 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fbcb57a32f54fc3a493bc5412e94051 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.806038 4196 raft_consensus.cc:686] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
W20260811 18:57:02.806542 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ad9254eaa184967a0d48ef4468c434b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.806607 4196 consensus_queue.cc:237] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.808912 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 39a040ea35974f709a7369080925d470 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:02.809188 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 621b56ba8f2f4392beea060871508a70 because the tablet was not found. No further retry: Not found: Tablet not found: 621b56ba8f2f4392beea060871508a70
W20260811 18:57:02.809417 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e355634612f64c14965d7af2fa442cb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e355634612f64c14965d7af2fa442cb7 already in progress: creating tablet
W20260811 18:57:02.809633 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d24799ded256403e9fad5c1a22b8aa80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d24799ded256403e9fad5c1a22b8aa80 already in progress: creating tablet
W20260811 18:57:02.809901 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8964f92255884b3aae0cac60f019fdb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8964f92255884b3aae0cac60f019fdb7 already in progress: creating tablet
W20260811 18:57:02.810154 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3b1bd859a80441068cccd993263dc4e1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b1bd859a80441068cccd993263dc4e1 already in progress: creating tablet
I20260811 18:57:02.810693 4167 ts_tablet_manager.cc:1918] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.810868 4167 log.cc:1196] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:02.811314 4167 ts_tablet_manager.cc:1939] T 4ae2221b54a44f6db10f6c98ee2344a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.811691 4168 tablet_replica.cc:330] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.811973 4168 raft_consensus.cc:2229] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.812283 4168 raft_consensus.cc:2258] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.813457 4168 ts_tablet_manager.cc:1905] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:02.814494 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 634c06d76eca404a88f326e905882930 consensus state has no leader
I20260811 18:57:02.815522 3547 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet ed809a37f7864b439ec0cf276f16bd13
W20260811 18:57:02.816179 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f43a7646be343cda0e8f5ad1cfb76cd consensus state has no leader
I20260811 18:57:02.817705 4163 tablet_bootstrap.cc:492] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.811739 4162 ts_tablet_manager.cc:1428] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.046s user 0.001s sys 0.001s
I20260811 18:57:02.818001 4163 ts_tablet_manager.cc:1397] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.006s
W20260811 18:57:02.818501 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d24799ded256403e9fad5c1a22b8aa80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d24799ded256403e9fad5c1a22b8aa80 already in progress: creating tablet
W20260811 18:57:02.818845 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet e355634612f64c14965d7af2fa442cb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e355634612f64c14965d7af2fa442cb7 already in progress: creating tablet
I20260811 18:57:02.819013 4162 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.819113 4163 raft_consensus.cc:348] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.819447 4163 raft_consensus.cc:374] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.819584 4163 raft_consensus.cc:729] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.819916 4163 consensus_queue.cc:260] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.820964 4162 tablet_bootstrap.cc:654] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:02.821749 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82d1339366a24a87b74bd1d411a3f6fe consensus state has no leader
I20260811 18:57:02.822047 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 1f43a7646be343cda0e8f5ad1cfb76cd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.823293 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4579e89d4503429badc8868ce44d947a" } modify_external_catalogs: true
I20260811 18:57:02.823424 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a3015128d6b741138d21edcf5a9c5785" } new_table_name: "default.Table_08b93745a5f44ce8bc61f090e881f952" modify_external_catalogs: true
I20260811 18:57:02.823536 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4579e89d4503429badc8868ce44d947a" } modify_external_catalogs: true
I20260811 18:57:02.824461 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4ae2221b54a44f6db10f6c98ee2344a3 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
W20260811 18:57:02.825829 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 90175b8fc5e34f40adb97de7f877ebc4 because the tablet was not found. No further retry: Not found: Tablet not found: 90175b8fc5e34f40adb97de7f877ebc4
W20260811 18:57:02.827271 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 621b56ba8f2f4392beea060871508a70 because the tablet was not found. No further retry: Not found: Tablet not found: 621b56ba8f2f4392beea060871508a70
I20260811 18:57:02.828299 4167 tablet_replica.cc:330] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.828559 4167 raft_consensus.cc:2229] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:02.828718 4163 ts_tablet_manager.cc:1428] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:02.828858 4167 pending_rounds.cc:62] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:02.828969 4167 pending_rounds.cc:68] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400051944697856 op_type: NO_OP noop_request { }
I20260811 18:57:02.829123 4167 raft_consensus.cc:2866] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:02.829267 4167 raft_consensus.cc:2258] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.832811 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 82d1339366a24a87b74bd1d411a3f6fe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:02.834136 4162 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.834332 4162 ts_tablet_manager.cc:1397] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20260811 18:57:02.834794 4167 ts_tablet_manager.cc:1905] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.835201 4162 raft_consensus.cc:348] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.835947 4165 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.836208 4165 ts_tablet_manager.cc:1397] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.008s
I20260811 18:57:02.836515 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6300adb327414f04a6ef25bfe7da7e02" } modify_external_catalogs: true
I20260811 18:57:02.836833 4168 ts_tablet_manager.cc:1918] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.836766 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6300adb327414f04a6ef25bfe7da7e02" } modify_external_catalogs: true
I20260811 18:57:02.837445 4162 raft_consensus.cc:374] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.837636 4162 raft_consensus.cc:729] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.837978 4162 consensus_queue.cc:260] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:02.840638 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 has no consensus state
I20260811 18:57:02.842537 4162 ts_tablet_manager.cc:1428] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260811 18:57:02.842962 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]). Expected version 1 got 0
I20260811 18:57:02.844863 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a040ea35974f709a7369080925d470 with delete_type TABLET_DATA_DELETED (replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.845093 4162 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.845311 4167 ts_tablet_manager.cc:1918] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:02.845469 4167 log.cc:1196] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/1e2f74e1a43242469ae646cd72708273
W20260811 18:57:02.845817 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 39a040ea35974f709a7369080925d470 because the tablet was not found. No further retry: Not found: Tablet not found: 39a040ea35974f709a7369080925d470
I20260811 18:57:02.846105 4167 ts_tablet_manager.cc:1939] T 1e2f74e1a43242469ae646cd72708273 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.847781 4162 tablet_bootstrap.cc:654] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.850044 4167 tablet_replica.cc:330] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.850200 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 1e2f74e1a43242469ae646cd72708273 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
I20260811 18:57:02.851713 4167 raft_consensus.cc:2229] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:02.852185 4167 pending_rounds.cc:62] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:02.852385 4167 pending_rounds.cc:68] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400054685683712 op_type: NO_OP noop_request { }
I20260811 18:57:02.852684 4167 raft_consensus.cc:2866] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:02.852974 4167 raft_consensus.cc:2258] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.837199 4165 raft_consensus.cc:348] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.855319 4165 raft_consensus.cc:374] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.855504 4165 raft_consensus.cc:729] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.855846 4167 ts_tablet_manager.cc:1905] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.855852 4165 consensus_queue.cc:260] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.855309 4168 log.cc:1196] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/3b1bd859a80441068cccd993263dc4e1
I20260811 18:57:02.856606 4165 ts_tablet_manager.cc:1428] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260811 18:57:02.857002 4165 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.858908 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d1b5cc176f54784ad79ad1ebd385acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:02.859030 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9de9acdb33a2438eb4378a5eac844c35 consensus state has no leader
I20260811 18:57:02.859054 4165 tablet_bootstrap.cc:654] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.859222 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e51b07e1104bc9bbcb429d29ac590b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.859477 4169 tablet_replica.cc:330] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.859670 4169 raft_consensus.cc:2229] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.859869 4169 raft_consensus.cc:2258] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.860673 4169 ts_tablet_manager.cc:1905] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.864694 3551 catalog_manager.cc:5495] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67). New cstate: current_term: 1 leader_uuid: "59811bdf82234ef7ac5865dbecfcc388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:02.865239 3551 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:02.866699 4165 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.866901 4165 ts_tablet_manager.cc:1397] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260811 18:57:02.866863 4167 ts_tablet_manager.cc:1918] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:02.867096 4167 log.cc:1196] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/39a040ea35974f709a7369080925d470
I20260811 18:57:02.867749 4167 ts_tablet_manager.cc:1939] T 39a040ea35974f709a7369080925d470 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.867784 4169 ts_tablet_manager.cc:1918] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.867928 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:02.868446 3549 catalog_manager.cc:6565] ReplaceTablet: tablet 82d1339366a24a87b74bd1d411a3f6fe deleted and replaced by tablet 7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:02.868690 4165 raft_consensus.cc:348] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.868981 4165 raft_consensus.cc:374] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.869061 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8efb663060ef457cbece962fce701eb1
I20260811 18:57:02.869098 4165 raft_consensus.cc:729] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.869375 4165 consensus_queue.cc:260] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.869726 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.867758 4162 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.870405 4162 ts_tablet_manager.cc:1397] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.002s
I20260811 18:57:02.870468 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.870877 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 39a040ea35974f709a7369080925d470 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:02.870990 4167 tablet_replica.cc:330] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.868846 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.871189 4169 log.cc:1196] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:02.871228 4167 raft_consensus.cc:2229] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:02.871615 4169 ts_tablet_manager.cc:1939] T 7d1b5cc176f54784ad79ad1ebd385acc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:02.871582 4167 pending_rounds.cc:62] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:02.871567 4162 raft_consensus.cc:348] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.871938 4162 raft_consensus.cc:374] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.872085 4162 raft_consensus.cc:729] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.870404 4163 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.870993 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.872449 4162 consensus_queue.cc:260] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.871848 4167 pending_rounds.cc:68] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400053367726080 op_type: NO_OP noop_request { }
I20260811 18:57:02.874799 4162 ts_tablet_manager.cc:1428] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:02.874886 4163 tablet_bootstrap.cc:654] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.869758 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.874924 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7d1b5cc176f54784ad79ad1ebd385acc (table default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]) successfully deleted
I20260811 18:57:02.870457 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.875300 4169 tablet_replica.cc:330] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:02.876175 4165 ts_tablet_manager.cc:1428] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260811 18:57:02.876958 4167 raft_consensus.cc:2866] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:02.877177 4167 raft_consensus.cc:2258] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.878646 4162 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.879773 4167 ts_tablet_manager.cc:1905] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.880074 4165 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.874567 4168 ts_tablet_manager.cc:1939] T 3b1bd859a80441068cccd993263dc4e1 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.881717 4163 tablet_bootstrap.cc:492] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.881949 4163 ts_tablet_manager.cc:1397] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260811 18:57:02.883133 4163 raft_consensus.cc:348] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.884145 4163 raft_consensus.cc:374] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.884316 4162 tablet_bootstrap.cc:654] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.884313 4163 raft_consensus.cc:729] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:02.885217 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 8964f92255884b3aae0cac60f019fdb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8964f92255884b3aae0cac60f019fdb7 already in progress: creating tablet
I20260811 18:57:02.885139 4163 consensus_queue.cc:260] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.886497 4165 tablet_bootstrap.cc:654] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:02.887585 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 82d1339366a24a87b74bd1d411a3f6fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82d1339366a24a87b74bd1d411a3f6fe already in progress: creating tablet
I20260811 18:57:02.888566 4163 ts_tablet_manager.cc:1428] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260811 18:57:02.889427 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 8efb663060ef457cbece962fce701eb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8efb663060ef457cbece962fce701eb1 already in progress: creating tablet
I20260811 18:57:02.889956 4163 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.892024 4163 tablet_bootstrap.cc:654] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.894732 4165 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:02.894917 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a040ea35974f709a7369080925d470 with delete_type TABLET_DATA_DELETED (replaced by tablet cd3b82603ec340ed9cabe9b2ff62a8b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.894994 4165 ts_tablet_manager.cc:1397] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.001s
I20260811 18:57:02.895032 4162 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.895184 4162 ts_tablet_manager.cc:1397] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.004s
I20260811 18:57:02.895861 4165 raft_consensus.cc:348] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.896147 4165 raft_consensus.cc:374] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.896090 4162 raft_consensus.cc:348] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.896243 4165 raft_consensus.cc:729] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.896369 4162 raft_consensus.cc:374] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.896484 4162 raft_consensus.cc:729] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.896530 4165 consensus_queue.cc:260] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.896649 3721 tablet_service.cc:1467] Processing CreateTablet for tablet a81334e7da3741a9b24f8978312181b9 (DEFAULT_TABLE table=default.Table_5332d093ac81423fa33cb860f7d9a460 [id=53b2edaad71944edb32a2b604deebbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.897107 4165 ts_tablet_manager.cc:1428] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:02.897125 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81334e7da3741a9b24f8978312181b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.896724 4162 consensus_queue.cc:260] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.896248 4011 tablet_service.cc:1467] Processing CreateTablet for tablet a81334e7da3741a9b24f8978312181b9 (DEFAULT_TABLE table=default.Table_5332d093ac81423fa33cb860f7d9a460 [id=53b2edaad71944edb32a2b604deebbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.897825 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81334e7da3741a9b24f8978312181b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.898855 4162 ts_tablet_manager.cc:1428] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 18:57:02.899181 4162 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.901026 4162 tablet_bootstrap.cc:654] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.901588 4163 tablet_bootstrap.cc:492] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.901783 4163 ts_tablet_manager.cc:1397] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.002s
I20260811 18:57:02.902634 4163 raft_consensus.cc:348] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.903250 3866 tablet_service.cc:1467] Processing CreateTablet for tablet a81334e7da3741a9b24f8978312181b9 (DEFAULT_TABLE table=default.Table_5332d093ac81423fa33cb860f7d9a460 [id=53b2edaad71944edb32a2b604deebbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.903777 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81334e7da3741a9b24f8978312181b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.909404 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1f43a7646be343cda0e8f5ad1cfb76cd
I20260811 18:57:02.909613 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:02.909930 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 1f43a7646be343cda0e8f5ad1cfb76cd deleted and replaced by tablet ed3346bd12d04c45a511308c427856e5
I20260811 18:57:02.912736 4169 raft_consensus.cc:2229] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.912976 4169 raft_consensus.cc:2258] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.913406 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd with delete_type TABLET_DATA_DELETED (replaced by tablet ed3346bd12d04c45a511308c427856e5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.913892 4169 ts_tablet_manager.cc:1905] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.914160 4162 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:02.914417 4162 ts_tablet_manager.cc:1397] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260811 18:57:02.915359 4162 raft_consensus.cc:348] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.915668 4162 raft_consensus.cc:374] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.915786 4162 raft_consensus.cc:729] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.916054 4162 consensus_queue.cc:260] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:02.916400 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd with delete_type TABLET_DATA_DELETED (replaced by tablet ed3346bd12d04c45a511308c427856e5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.917814 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:02.918058 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 34aeab84310d49c79805ea898b4f87ce (table default.Table_738a85e77f9e49efa9899428ec31a9ab [id=7cda514ae5f24bf785720ec89d44b923]). Expected version 1 got 0
I20260811 18:57:02.919287 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd with delete_type TABLET_DATA_DELETED (replaced by tablet ed3346bd12d04c45a511308c427856e5) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.920020 4163 raft_consensus.cc:374] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.920308 4163 raft_consensus.cc:729] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.920675 4163 consensus_queue.cc:260] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:02.920913 4168 tablet_replica.cc:330] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.921226 4168 raft_consensus.cc:2229] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.921437 4168 raft_consensus.cc:2258] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.922735 4168 ts_tablet_manager.cc:1905] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.923561 4163 ts_tablet_manager.cc:1428] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.003s sys 0.000s
I20260811 18:57:02.926707 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:02.926857 4163 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.928889 4163 tablet_bootstrap.cc:654] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.930418 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet dd465168aecf43a1b08b9c11d241f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.931569 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3414c134374aa59ac23d276a018ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:02.931872 4163 tablet_bootstrap.cc:492] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.932078 4163 ts_tablet_manager.cc:1397] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 18:57:02.933183 4163 raft_consensus.cc:348] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.933521 4163 raft_consensus.cc:374] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.933693 4163 raft_consensus.cc:729] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.934012 4163 consensus_queue.cc:260] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.934825 4163 ts_tablet_manager.cc:1428] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:02.935169 4163 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.935956 4167 ts_tablet_manager.cc:1918] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:02.937228 4163 tablet_bootstrap.cc:654] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.938143 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3414c134374aa59ac23d276a018ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.938416 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae2221b54a44f6db10f6c98ee2344a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.939606 4168 ts_tablet_manager.cc:1918] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.941783 4168 log.cc:1196] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/1f43a7646be343cda0e8f5ad1cfb76cd
I20260811 18:57:02.943284 4168 ts_tablet_manager.cc:1939] T 1f43a7646be343cda0e8f5ad1cfb76cd P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.944793 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.946961 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 1f43a7646be343cda0e8f5ad1cfb76cd (table default.Table_14ea27ae9206400f82d35d43e532e99b [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
I20260811 18:57:02.947276 4168 tablet_replica.cc:330] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:02.947505 4168 raft_consensus.cc:2229] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.947739 4168 raft_consensus.cc:2258] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.947570 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_420484ea1b7f491bb6f1c10bfffed8f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.936260 4167 log.cc:1196] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a1e51b07e1104bc9bbcb429d29ac590b
W20260811 18:57:02.948308 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_420484ea1b7f491bb6f1c10bfffed8f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.948814 4168 ts_tablet_manager.cc:1905] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.949095 4167 ts_tablet_manager.cc:1939] T a1e51b07e1104bc9bbcb429d29ac590b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:02.940654 4162 ts_tablet_manager.cc:1428] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20260811 18:57:02.947149 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3414c134374aa59ac23d276a018ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:02.953339 4163 tablet_bootstrap.cc:492] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.954030 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]). Expected version 1 got 0
I20260811 18:57:02.954306 4163 ts_tablet_manager.cc:1397] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.007s
I20260811 18:57:02.955133 3858 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba169c32ef48b984e649b211ee2af1 (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.955530 3893 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.955600 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba169c32ef48b984e649b211ee2af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.955861 4012 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba169c32ef48b984e649b211ee2af1 (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.956313 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba169c32ef48b984e649b211ee2af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.957216 4162 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:02.958810 3746 raft_consensus.cc:1261] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 59811bdf82234ef7ac5865dbecfcc388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:02.960086 4162 tablet_bootstrap.cc:654] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.961457 4167 tablet_replica.cc:330] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:02.961694 4167 raft_consensus.cc:2229] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:02.961927 4167 raft_consensus.cc:2258] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:02.962497 4163 raft_consensus.cc:348] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.962754 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 8af3e74889b64c408c9ed9c54916f890 (DEFAULT_TABLE table=default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.962896 4167 ts_tablet_manager.cc:1905] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:02.962883 4163 raft_consensus.cc:374] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.963068 4163 raft_consensus.cc:729] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.963152 3893 raft_consensus.cc:1261] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer 59811bdf82234ef7ac5865dbecfcc388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:02.963295 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af3e74889b64c408c9ed9c54916f890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.963411 4163 consensus_queue.cc:260] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.964378 4183 consensus_queue.cc:1035] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:02.964955 4163 ts_tablet_manager.cc:1428] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260811 18:57:02.965138 4183 raft_consensus.cc:482] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:02.965288 4163 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.965322 4183 raft_consensus.cc:504] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.965991 4183 leader_election.cc:288] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:02.967393 4234 consensus_queue.cc:1035] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:02.969178 4163 tablet_bootstrap.cc:654] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.969957 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a1e51b07e1104bc9bbcb429d29ac590b (table default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]) successfully deleted
I20260811 18:57:02.897431 4165 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:02.971400 3721 tablet_service.cc:1467] Processing CreateTablet for tablet a0ba169c32ef48b984e649b211ee2af1 (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.971897 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ba169c32ef48b984e649b211ee2af1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.972180 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]). Expected version 1 got 0
I20260811 18:57:02.973114 4165 tablet_bootstrap.cc:654] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.973498 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_233cfa4d8fe948dbac05d78693cd6d9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.974418 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_233cfa4d8fe948dbac05d78693cd6d9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.975078 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b1bd859a80441068cccd993263dc4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:02.975216 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 8af3e74889b64c408c9ed9c54916f890 (DEFAULT_TABLE table=default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:02.975643 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af3e74889b64c408c9ed9c54916f890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.976692 4168 ts_tablet_manager.cc:1918] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:02.977607 4168 log.cc:1196] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:02.978277 4168 ts_tablet_manager.cc:1939] T dd465168aecf43a1b08b9c11d241f898 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:02.979089 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:02.982664 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet dd465168aecf43a1b08b9c11d241f898 (table default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]) successfully deleted
W20260811 18:57:02.983702 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 0f3414c134374aa59ac23d276a018ec9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3414c134374aa59ac23d276a018ec9 already in progress: creating tablet
I20260811 18:57:02.986056 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:02.986464 4163 tablet_bootstrap.cc:492] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:02.986429 3745 raft_consensus.cc:2454] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:02.986742 4163 ts_tablet_manager.cc:1397] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.002s
I20260811 18:57:02.987102 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e584229a7f164fc8be8d7963583c65c0" } modify_external_catalogs: true
I20260811 18:57:02.987576 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e584229a7f164fc8be8d7963583c65c0" } modify_external_catalogs: true
I20260811 18:57:02.987706 4163 raft_consensus.cc:348] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.987985 4163 raft_consensus.cc:374] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.988157 4163 raft_consensus.cc:729] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.988480 4163 consensus_queue.cc:260] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:02.989516 4163 ts_tablet_manager.cc:1428] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:02.989847 4163 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:02.990083 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_deebcc7412e6466986509a6858c4bc70" } modify_external_catalogs: true
I20260811 18:57:02.990410 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_deebcc7412e6466986509a6858c4bc70" } modify_external_catalogs: true
I20260811 18:57:02.991686 4163 tablet_bootstrap.cc:654] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.994412 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c56b0614a7843bea868d1e3fd95de68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:02.994746 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1f27ea0d4f084ac081cca10255b06c11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:02.995046 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c56b0614a7843bea868d1e3fd95de68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:02.995338 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f27ea0d4f084ac081cca10255b06c11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:02.997162 3967 leader_election.cc:302] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:02.997558 4234 raft_consensus.cc:2781] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:02.997725 4234 raft_consensus.cc:482] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:02.997855 4234 raft_consensus.cc:3037] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.002147 4234 raft_consensus.cc:504] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.002267 4009 tablet_service.cc:1467] Processing CreateTablet for tablet d106e0e1f4b845429b10f4e100f69a8b (DEFAULT_TABLE table=default.Table_1ad9254eaa184967a0d48ef4468c434b [id=17b08a7402a34713a7329c297b221da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.002312 3866 tablet_service.cc:1467] Processing CreateTablet for tablet d106e0e1f4b845429b10f4e100f69a8b (DEFAULT_TABLE table=default.Table_1ad9254eaa184967a0d48ef4468c434b [id=17b08a7402a34713a7329c297b221da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.002818 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106e0e1f4b845429b10f4e100f69a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.002835 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106e0e1f4b845429b10f4e100f69a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.003165 4234 leader_election.cc:288] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:03.005273 3965 leader_election.cc:341] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.006961 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c (DEFAULT_TABLE table=default.Table_abaf87a40e04427fba978e8a67b89f0c [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.007442 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ace00158b744bd8a97dc1c47fc6049c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.007608 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:03.007880 3745 raft_consensus.cc:3037] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.009644 4169 ts_tablet_manager.cc:1918] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.009842 4169 log.cc:1196] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a1e51b07e1104bc9bbcb429d29ac590b
I20260811 18:57:03.010272 4169 ts_tablet_manager.cc:1939] T a1e51b07e1104bc9bbcb429d29ac590b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:03.011471 3745 raft_consensus.cc:2454] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:03.012421 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c (DEFAULT_TABLE table=default.Table_abaf87a40e04427fba978e8a67b89f0c [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.012521 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.012938 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ace00158b744bd8a97dc1c47fc6049c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.013983 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d106e0e1f4b845429b10f4e100f69a8b (DEFAULT_TABLE table=default.Table_1ad9254eaa184967a0d48ef4468c434b [id=17b08a7402a34713a7329c297b221da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.014467 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106e0e1f4b845429b10f4e100f69a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.016661 4014 tablet_service.cc:1467] Processing CreateTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (DEFAULT_TABLE table=default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.017133 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.021011 4196 raft_consensus.cc:482] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.021293 4196 raft_consensus.cc:504] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:03.022014 3967 leader_election.cc:302] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:03.021970 3870 tablet_service.cc:1467] Processing CreateTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (DEFAULT_TABLE table=default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.022224 4196 leader_election.cc:288] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:03.022365 4234 raft_consensus.cc:2781] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.022508 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.023072 4234 raft_consensus.cc:686] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:03.023525 4234 consensus_queue.cc:237] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.023876 4163 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.024063 4163 ts_tablet_manager.cc:1397] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.000s
I20260811 18:57:03.025024 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c (DEFAULT_TABLE table=default.Table_abaf87a40e04427fba978e8a67b89f0c [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.024953 4163 raft_consensus.cc:348] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.025357 4163 raft_consensus.cc:374] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.025457 4163 raft_consensus.cc:729] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.025767 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ace00158b744bd8a97dc1c47fc6049c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.025754 4163 consensus_queue.cc:260] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.026499 4163 ts_tablet_manager.cc:1428] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:57:03.027781 3678 leader_election.cc:341] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:03.029016 4165 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.029168 4165 ts_tablet_manager.cc:1397] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.001s
I20260811 18:57:03.029968 4165 raft_consensus.cc:348] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.030227 4165 raft_consensus.cc:374] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.030472 4165 raft_consensus.cc:729] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.030733 4165 consensus_queue.cc:260] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.031325 4165 ts_tablet_manager.cc:1428] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:03.031599 4165 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:03.031709 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 8af3e74889b64c408c9ed9c54916f890 (DEFAULT_TABLE table=default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.032212 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af3e74889b64c408c9ed9c54916f890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.033382 4165 tablet_bootstrap.cc:654] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:03.035368 3965 leader_election.cc:341] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.037184 3724 tablet_service.cc:1467] Processing CreateTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (DEFAULT_TABLE table=default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.037736 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.038470 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd619a3970764d0fb914fcc6f59ea7a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.039165 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd619a3970764d0fb914fcc6f59ea7a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.039629 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_174aeae5133245c3ba894064a8d08dd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.040316 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_174aeae5133245c3ba894064a8d08dd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.041942 4178 raft_consensus.cc:482] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.041864 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a311fc9dc1724795a63f4df79f8ebcbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.042140 4178 raft_consensus.cc:504] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:03.042415 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a311fc9dc1724795a63f4df79f8ebcbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.042837 4178 leader_election.cc:288] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.043774 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_68b0781f65bb450a9b74324ca7d6d5ed" } modify_external_catalogs: true
I20260811 18:57:03.044059 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_68b0781f65bb450a9b74324ca7d6d5ed" } modify_external_catalogs: true
I20260811 18:57:03.046190 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_524727faeeb3483298947337811873bf" } new_table_name: "default.Table_3fe670d2d1474485834592fa8f239e62" modify_external_catalogs: true
W20260811 18:57:03.046455 3824 leader_election.cc:341] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:03.048244 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1ad9254eaa184967a0d48ef4468c434b" } new_table_name: "default.Table_8e4b85a9b7d5460b83544a3677cde81a" modify_external_catalogs: true
W20260811 18:57:03.048781 3676 leader_election.cc:341] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:03.048959 3676 leader_election.cc:302] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.049373 4196 raft_consensus.cc:2726] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:03.050688 3824 leader_election.cc:341] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:03.050920 3824 leader_election.cc:302] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:03.051322 4178 raft_consensus.cc:2726] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.057328 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:03.058882 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.059176 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.059572 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:03.061432 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 2852f9e5f67c4025836018af553388a5 (DEFAULT_TABLE table=default.Table_f30cf1eb759b4a7ab689610c645803e5 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.061540 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.061756 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 2852f9e5f67c4025836018af553388a5 (DEFAULT_TABLE table=default.Table_f30cf1eb759b4a7ab689610c645803e5 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.062146 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2852f9e5f67c4025836018af553388a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.062282 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2852f9e5f67c4025836018af553388a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.062529 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.069914 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:03.070741 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 5df7a07983174dd1aaf48951c9ff1a1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df7a07983174dd1aaf48951c9ff1a1b already in progress: creating tablet
I20260811 18:57:03.072978 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 2852f9e5f67c4025836018af553388a5 (DEFAULT_TABLE table=default.Table_f30cf1eb759b4a7ab689610c645803e5 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.073654 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2852f9e5f67c4025836018af553388a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.074083 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a1e51b07e1104bc9bbcb429d29ac590b (table default.Table_c842a73deafd496bab0a3bc0b96b071d [id=509a652504ae405b843cb7f859b6d9cc]) successfully deleted
I20260811 18:57:03.075495 4012 tablet_service.cc:1467] Processing CreateTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (DEFAULT_TABLE table=default.Table_ab16863006cb44f5825c317ad3ecdea2 [id=255f252ebd6245bd928387a45609ddcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.076046 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b3a14979e84b1e8ec6ed3f49b93b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.076906 3862 tablet_service.cc:1467] Processing CreateTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (DEFAULT_TABLE table=default.Table_ab16863006cb44f5825c317ad3ecdea2 [id=255f252ebd6245bd928387a45609ddcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.077383 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b3a14979e84b1e8ec6ed3f49b93b00. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:03.078146 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 82d1339366a24a87b74bd1d411a3f6fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82d1339366a24a87b74bd1d411a3f6fe already in progress: creating tablet
I20260811 18:57:03.079697 3722 tablet_service.cc:1467] Processing CreateTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (DEFAULT_TABLE table=default.Table_ab16863006cb44f5825c317ad3ecdea2 [id=255f252ebd6245bd928387a45609ddcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.080153 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b3a14979e84b1e8ec6ed3f49b93b00. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:03.086687 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8efb663060ef457cbece962fce701eb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8efb663060ef457cbece962fce701eb1 already in progress: creating tablet
I20260811 18:57:03.088178 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7f640c60e81b4a369458b1497a269c9c (DEFAULT_TABLE table=default.Table_52e96fad41744bc5bd6efbbb18e5f692 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.088654 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f640c60e81b4a369458b1497a269c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.091436 4163 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.093338 4163 tablet_bootstrap.cc:654] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.093835 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 7f640c60e81b4a369458b1497a269c9c (DEFAULT_TABLE table=default.Table_52e96fad41744bc5bd6efbbb18e5f692 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.093948 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 7f640c60e81b4a369458b1497a269c9c (DEFAULT_TABLE table=default.Table_52e96fad41744bc5bd6efbbb18e5f692 [id=5c9e64742a0a4d80b8035ccb50459a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.094297 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f640c60e81b4a369458b1497a269c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.094439 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f640c60e81b4a369458b1497a269c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.097158 4234 raft_consensus.cc:482] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:03.097239 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1f43a7646be343cda0e8f5ad1cfb76cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f43a7646be343cda0e8f5ad1cfb76cd already in progress: creating tablet
I20260811 18:57:03.097395 4234 raft_consensus.cc:504] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:03.098584 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0f3414c134374aa59ac23d276a018ec9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3414c134374aa59ac23d276a018ec9 already in progress: creating tablet
I20260811 18:57:03.098780 4234 leader_election.cc:288] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.099500 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb530181d7b4d2baf5391808246c74b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.099720 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 (DEFAULT_TABLE table=default.Table_1fbb53e0b2d44010bbb9da4dc739a78d [id=8931a7530e1b483d864b7c8a64612f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.099715 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 (DEFAULT_TABLE table=default.Table_1fbb53e0b2d44010bbb9da4dc739a78d [id=8931a7530e1b483d864b7c8a64612f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.099851 3893 raft_consensus.cc:2454] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:03.100178 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c8443df21949ff9bdc63fa6aa34108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.100224 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c8443df21949ff9bdc63fa6aa34108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.100379 3965 leader_election.cc:302] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
I20260811 18:57:03.101130 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb530181d7b4d2baf5391808246c74b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:03.101449 3745 raft_consensus.cc:2454] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:03.102626 4234 raft_consensus.cc:2781] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:03.102828 4234 raft_consensus.cc:482] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.103024 4234 raft_consensus.cc:3037] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:03.105937 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 49751us
I20260811 18:57:03.106593 4169 tablet_replica.cc:330] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:03.107801 4169 raft_consensus.cc:2229] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.108069 4169 raft_consensus.cc:2258] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.108304 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:03.109133 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 8af3e74889b64c408c9ed9c54916f890 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8af3e74889b64c408c9ed9c54916f890 already in progress: creating tablet
I20260811 18:57:03.109716 4169 ts_tablet_manager.cc:1905] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.114639 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 (DEFAULT_TABLE table=default.Table_1fbb53e0b2d44010bbb9da4dc739a78d [id=8931a7530e1b483d864b7c8a64612f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.114940 3862 tablet_service.cc:1467] Processing CreateTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 (DEFAULT_TABLE table=default.Table_524727faeeb3483298947337811873bf [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.115171 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c8443df21949ff9bdc63fa6aa34108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.115342 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ab2eee21264b1aa5ba00528cae3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.120138 4011 tablet_service.cc:1467] Processing CreateTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 (DEFAULT_TABLE table=default.Table_524727faeeb3483298947337811873bf [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.120363 4163 tablet_bootstrap.cc:492] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.120550 4163 ts_tablet_manager.cc:1397] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.000s
I20260811 18:57:03.120844 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ab2eee21264b1aa5ba00528cae3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.121423 4163 raft_consensus.cc:348] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.121658 4163 raft_consensus.cc:374] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.121755 4163 raft_consensus.cc:729] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.122018 4163 consensus_queue.cc:260] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.122609 4163 ts_tablet_manager.cc:1428] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:03.122931 4163 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.124831 4163 tablet_bootstrap.cc:654] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.126679 4169 ts_tablet_manager.cc:1918] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.126824 4169 log.cc:1196] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/4ae2221b54a44f6db10f6c98ee2344a3
I20260811 18:57:03.127200 4169 ts_tablet_manager.cc:1939] T 4ae2221b54a44f6db10f6c98ee2344a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:03.127701 4169 tablet_replica.cc:330] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:03.127925 4169 raft_consensus.cc:2229] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.128131 4169 raft_consensus.cc:2258] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.128361 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 4ae2221b54a44f6db10f6c98ee2344a3 (table default.Table_ac6f106b51c54467b6bf3f075a7d9422 [id=b557383bcdf44c0c98286725320837e6]) successfully deleted
I20260811 18:57:03.132827 4163 tablet_bootstrap.cc:492] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.133023 4163 ts_tablet_manager.cc:1397] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.003s
I20260811 18:57:03.133836 4163 raft_consensus.cc:348] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.134104 4163 raft_consensus.cc:374] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.134193 4163 raft_consensus.cc:729] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.134986 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 60274115fe164d85badcf2e72d640f54 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.135596 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60274115fe164d85badcf2e72d640f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.135725 4169 ts_tablet_manager.cc:1905] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.137879 3723 tablet_service.cc:1467] Processing CreateTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 (DEFAULT_TABLE table=default.Table_524727faeeb3483298947337811873bf [id=64fc5489749347609383603760522f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.138448 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ab2eee21264b1aa5ba00528cae3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.144796 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 60274115fe164d85badcf2e72d640f54 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.145473 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60274115fe164d85badcf2e72d640f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.153661 4163 consensus_queue.cc:260] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.154510 4163 ts_tablet_manager.cc:1428] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.021s user 0.001s sys 0.002s
I20260811 18:57:03.155424 4163 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.172866 4163 tablet_bootstrap.cc:654] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.175160 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 60274115fe164d85badcf2e72d640f54 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.175450 4234 raft_consensus.cc:504] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.180833 4169 ts_tablet_manager.cc:1918] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.182684 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60274115fe164d85badcf2e72d640f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.189013 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
I20260811 18:57:03.189258 4259 mvcc.cc:205] Tried to move back new op lower bound from 7317400055598817280 to 7317400054907678720. Current Snapshot: MvccSnapshot[applied={T|T < 7317400055598817280}]
I20260811 18:57:03.189426 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb530181d7b4d2baf5391808246c74b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.189749 3892 raft_consensus.cc:3037] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.190980 4183 raft_consensus.cc:482] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.191175 4183 raft_consensus.cc:504] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.192303 4183 leader_election.cc:288] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.193044 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.193246 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.193420 3892 raft_consensus.cc:2454] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
W20260811 18:57:03.194041 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 already in progress: creating tablet
I20260811 18:57:03.194239 3965 leader_election.cc:302] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
I20260811 18:57:03.194423 4169 log.cc:1196] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:03.194594 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
W20260811 18:57:03.194840 3965 leader_election.cc:341] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.194967 4169 ts_tablet_manager.cc:1939] T 637bb23ea30c4de0a6fe23f7aaf70c25 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:03.195204 4183 raft_consensus.cc:2781] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.195533 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65f9d97ef0514e40893e87f1885ad5f2" } new_table_name: "default.Table_314cf78c221448d1a5fa8708b3adf8b7" modify_external_catalogs: true
I20260811 18:57:03.196441 4162 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.196696 4162 ts_tablet_manager.cc:1397] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.245s user 0.005s sys 0.004s
I20260811 18:57:03.197681 4162 raft_consensus.cc:348] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.197837 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f30cf1eb759b4a7ab689610c645803e5" } new_table_name: "default.Table_d816b59355a94c8698108ac81bf1ab89" modify_external_catalogs: true
I20260811 18:57:03.198084 4162 raft_consensus.cc:374] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.198230 4162 raft_consensus.cc:729] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.198661 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_27837256c8254b8c9b6493a7ca044751"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.198904 4169 tablet_replica.cc:330] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:03.199115 4169 raft_consensus.cc:2229] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.199361 4169 raft_consensus.cc:2258] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:03.199419 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27837256c8254b8c9b6493a7ca044751 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.200273 4169 ts_tablet_manager.cc:1905] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.201011 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab16863006cb44f5825c317ad3ecdea2" } new_table_name: "default.Table_4a945f1a824043b6b99704ee0994c6c6" modify_external_catalogs: true
I20260811 18:57:03.203289 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 1 got 0
I20260811 18:57:03.203487 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:03.204186 4196 raft_consensus.cc:482] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.204463 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3444887e982747ff9c397d48da9ae8c0" } modify_external_catalogs: true
I20260811 18:57:03.204401 4196 raft_consensus.cc:504] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.204684 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3444887e982747ff9c397d48da9ae8c0" } modify_external_catalogs: true
I20260811 18:57:03.205317 4196 leader_election.cc:288] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:03.206727 3678 leader_election.cc:341] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 1f43a7646be343cda0e8f5ad1cfb76cd
I20260811 18:57:03.206291 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1fd04f00945749c89bd1277cfca402f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.207290 4169 ts_tablet_manager.cc:1918] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.207450 4169 log.cc:1196] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3b1bd859a80441068cccd993263dc4e1
I20260811 18:57:03.207572 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f43a7646be343cda0e8f5ad1cfb76cd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:03.207664 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fd04f00945749c89bd1277cfca402f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.207907 3891 raft_consensus.cc:2454] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:03.208349 3676 leader_election.cc:302] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.209081 4196 raft_consensus.cc:2781] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:03.209218 4196 raft_consensus.cc:482] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.209408 4196 raft_consensus.cc:3037] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.209329 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_03c613798a09459d8d50569ba4639330"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.210014 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03c613798a09459d8d50569ba4639330 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.212600 3551 catalog_manager.cc:5495] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67). New cstate: current_term: 1 leader_uuid: "59811bdf82234ef7ac5865dbecfcc388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:03.213593 4196 raft_consensus.cc:504] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.214776 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.215008 4196 leader_election.cc:288] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:03.215044 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 637bb23ea30c4de0a6fe23f7aaf70c25 (table default.Table_425ac46cd0ab4d87bca0094367567ae8 [id=5fed4973b7224d1881e80a7f44d7297d]) successfully deleted
W20260811 18:57:03.216041 3678 leader_election.cc:341] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 1f43a7646be343cda0e8f5ad1cfb76cd
I20260811 18:57:03.216534 4169 ts_tablet_manager.cc:1939] T 3b1bd859a80441068cccd993263dc4e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:03.217015 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f43a7646be343cda0e8f5ad1cfb76cd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.217263 3891 raft_consensus.cc:3037] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.217760 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 3b1bd859a80441068cccd993263dc4e1 (table default.Table_e2ce8f5ac51a49eea0edef37b3dbdd2c [id=115cebb7c7fa490f82d30693d7bf06d4]) successfully deleted
I20260811 18:57:03.218528 4258 mvcc.cc:205] Tried to move back new op lower bound from 7317400055598817280 to 7317400054907678720. Current Snapshot: MvccSnapshot[applied={T|T < 7317400055598817280}]
W20260811 18:57:03.220822 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8af3e74889b64c408c9ed9c54916f890 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8af3e74889b64c408c9ed9c54916f890 already in progress: creating tablet
I20260811 18:57:03.220620 4272 tablet.cc:1722] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260811 18:57:03.222365 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5df7a07983174dd1aaf48951c9ff1a1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df7a07983174dd1aaf48951c9ff1a1b already in progress: creating tablet
W20260811 18:57:03.223603 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 already in progress: creating tablet
I20260811 18:57:03.224776 3891 raft_consensus.cc:2454] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:03.226074 3676 leader_election.cc:302] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.226874 4196 raft_consensus.cc:2781] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.227360 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:03.228504 4196 raft_consensus.cc:686] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:03.229115 4196 consensus_queue.cc:237] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.229142 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:03.231009 3967 leader_election.cc:341] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.231415 3967 leader_election.cc:302] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:03.232607 4265 raft_consensus.cc:2726] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.236375 4234 leader_election.cc:288] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.238214 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb530181d7b4d2baf5391808246c74b" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
W20260811 18:57:03.238420 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 consensus state has no leader
I20260811 18:57:03.238539 3745 raft_consensus.cc:3037] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.238917 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_738a85e77f9e49efa9899428ec31a9ab" } modify_external_catalogs: true
I20260811 18:57:03.239159 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_738a85e77f9e49efa9899428ec31a9ab" } modify_external_catalogs: true
I20260811 18:57:03.243626 3745 raft_consensus.cc:2454] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:03.237917 4183 raft_consensus.cc:686] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:03.244941 4183 consensus_queue.cc:237] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.249720 4273 tablet.cc:1722] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:03.255458 4165 tablet_bootstrap.cc:492] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.255692 4165 ts_tablet_manager.cc:1397] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.224s user 0.042s sys 0.077s
I20260811 18:57:03.256608 4165 raft_consensus.cc:348] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.256883 4165 raft_consensus.cc:374] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.256959 4165 raft_consensus.cc:729] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.257236 4165 consensus_queue.cc:260] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.260303 4165 ts_tablet_manager.cc:1428] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:03.260264 4270 tablet.cc:1722] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:03.265159 4165 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:03.267187 4162 consensus_queue.cc:260] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.267810 4178 raft_consensus.cc:482] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.267959 4162 ts_tablet_manager.cc:1428] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.071s user 0.003s sys 0.000s
I20260811 18:57:03.268049 4178 raft_consensus.cc:504] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.268371 4162 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:03.268832 4178 leader_election.cc:288] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.269546 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:03.270045 3745 raft_consensus.cc:2379] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate 59811bdf82234ef7ac5865dbecfcc388 in this term.
I20260811 18:57:03.268442 4165 tablet_bootstrap.cc:654] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.269389 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41558efe0344434db6e755804442d2de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:03.271950 3824 leader_election.cc:302] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:03.272541 4167 ts_tablet_manager.cc:1918] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.275535 4167 log.cc:1196] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:03.276479 4167 ts_tablet_manager.cc:1939] T dd465168aecf43a1b08b9c11d241f898 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:03.279533 4163 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.279877 4163 ts_tablet_manager.cc:1397] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.125s user 0.010s sys 0.033s
I20260811 18:57:03.280113 4178 raft_consensus.cc:3037] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.291628 4163 raft_consensus.cc:348] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.292313 4163 raft_consensus.cc:374] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.292622 4163 raft_consensus.cc:729] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.293133 4178 raft_consensus.cc:2726] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.293020 4163 consensus_queue.cc:260] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.294062 4163 ts_tablet_manager.cc:1428] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260811 18:57:03.294862 4163 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:03.302675 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d106e0e1f4b845429b10f4e100f69a8b consensus state has no leader
I20260811 18:57:03.304068 4163 tablet_bootstrap.cc:654] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:03.304613 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6d1d58835ad24749a00ce27b4bc73c59 consensus state has no leader
I20260811 18:57:03.305912 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:03.306874 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8efb663060ef457cbece962fce701eb1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.309932 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 82d1339366a24a87b74bd1d411a3f6fe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.310811 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 494c09652d9e408880ca606ae0b30773 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:03.312439 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7b3a14979e84b1e8ec6ed3f49b93b00 consensus state has no leader
I20260811 18:57:03.312861 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b1bd859a80441068cccd993263dc4e1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.313597 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ae2221b54a44f6db10f6c98ee2344a3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.314471 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e159333f7174424eaf413e0a07c149e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.315254 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e159333f7174424eaf413e0a07c149e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.317530 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d35faa013d8c45e29cddf10c7c62e27d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.319953 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d35faa013d8c45e29cddf10c7c62e27d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.321085 4162 tablet_bootstrap.cc:654] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.324275 4167 tablet_replica.cc:330] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:03.324584 4167 raft_consensus.cc:2229] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.324846 4163 tablet_bootstrap.cc:492] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.324954 4167 raft_consensus.cc:2258] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.325140 4163 ts_tablet_manager.cc:1397] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.003s
I20260811 18:57:03.325681 4167 ts_tablet_manager.cc:1905] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.326541 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet dd465168aecf43a1b08b9c11d241f898 (table default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]) successfully deleted
I20260811 18:57:03.326545 4163 raft_consensus.cc:348] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.327011 4163 raft_consensus.cc:374] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.327183 4163 raft_consensus.cc:729] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:03.327394 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 637bb23ea30c4de0a6fe23f7aaf70c25 because the tablet was not found. No further retry: Not found: Tablet not found: 637bb23ea30c4de0a6fe23f7aaf70c25
I20260811 18:57:03.327567 4163 consensus_queue.cc:260] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.328534 4163 ts_tablet_manager.cc:1428] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:03.341234 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:03.344583 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.345283 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:03.346554 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 08b8dcf7a56043efbbd40bb888616492 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08b8dcf7a56043efbbd40bb888616492 already in progress: creating tablet
I20260811 18:57:03.346762 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:03.347672 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 08b8dcf7a56043efbbd40bb888616492 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08b8dcf7a56043efbbd40bb888616492 already in progress: creating tablet
W20260811 18:57:03.354844 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2852f9e5f67c4025836018af553388a5 consensus state has no leader
I20260811 18:57:03.355393 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_29c83b98881f4edaba5330f38e7a67ed" } modify_external_catalogs: true
I20260811 18:57:03.355744 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_75948600dcf347f89478127c9f9b96fd" } modify_external_catalogs: true
I20260811 18:57:03.355798 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_29c83b98881f4edaba5330f38e7a67ed" } modify_external_catalogs: true
I20260811 18:57:03.356026 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_75948600dcf347f89478127c9f9b96fd" } modify_external_catalogs: true
I20260811 18:57:03.355412 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_57bac5d4e666472fbab036524cbbed7d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.359169 4165 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.359383 4165 ts_tablet_manager.cc:1397] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.099s user 0.023s sys 0.046s
W20260811 18:57:03.359431 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57bac5d4e666472fbab036524cbbed7d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.360741 4165 raft_consensus.cc:348] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.361232 4165 raft_consensus.cc:374] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.361492 4165 raft_consensus.cc:729] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.361873 4165 consensus_queue.cc:260] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.364390 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]). Expected version 4 got 0
I20260811 18:57:03.368948 4165 ts_tablet_manager.cc:1428] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:57:03.370791 3893 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.371996 3745 raft_consensus.cc:1261] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 59811bdf82234ef7ac5865dbecfcc388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:03.372562 4165 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:03.374944 4163 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.375720 4271 consensus_queue.cc:1035] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:03.380388 4163 tablet_bootstrap.cc:654] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.380934 3893 raft_consensus.cc:1261] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer 59811bdf82234ef7ac5865dbecfcc388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:03.383581 4271 consensus_queue.cc:1035] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:03.386082 4165 tablet_bootstrap.cc:654] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.390095 4271 raft_consensus.cc:482] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.390378 4271 raft_consensus.cc:504] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.392109 4271 leader_election.cc:288] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.392493 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d1339366a24a87b74bd1d411a3f6fe" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.392916 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d1339366a24a87b74bd1d411a3f6fe" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:03.393272 3965 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:03.394107 3967 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.394454 3967 leader_election.cc:302] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:03.394898 4162 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.395161 4162 ts_tablet_manager.cc:1397] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.127s user 0.007s sys 0.002s
I20260811 18:57:03.396660 4162 raft_consensus.cc:348] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.397073 4162 raft_consensus.cc:374] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.397271 4162 raft_consensus.cc:729] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.398669 4162 consensus_queue.cc:260] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.398916 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 494c09652d9e408880ca606ae0b30773
I20260811 18:57:03.399572 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 494c09652d9e408880ca606ae0b30773 deleted and replaced by tablet a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:03.401409 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.401556 4167 ts_tablet_manager.cc:1918] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.401997 4167 log.cc:1196] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7d1b5cc176f54784ad79ad1ebd385acc
I20260811 18:57:03.402244 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:03.402259 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 494c09652d9e408880ca606ae0b30773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 494c09652d9e408880ca606ae0b30773 already in progress: creating tablet
I20260811 18:57:03.402807 4167 ts_tablet_manager.cc:1939] T 7d1b5cc176f54784ad79ad1ebd385acc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:03.403309 4163 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.403960 4162 ts_tablet_manager.cc:1428] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:57:03.404070 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.404413 4162 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:03.405916 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 494c09652d9e408880ca606ae0b30773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 494c09652d9e408880ca606ae0b30773 already in progress: creating tablet
I20260811 18:57:03.406884 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_420484ea1b7f491bb6f1c10bfffed8f3" } modify_external_catalogs: true
I20260811 18:57:03.407159 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_420484ea1b7f491bb6f1c10bfffed8f3" } modify_external_catalogs: true
I20260811 18:57:03.409085 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ac944e858133489cb57a21e726976f92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.409823 4163 ts_tablet_manager.cc:1397] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.006s
W20260811 18:57:03.410389 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac944e858133489cb57a21e726976f92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.410662 4271 raft_consensus.cc:2726] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.411057 4162 tablet_bootstrap.cc:654] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.412096 4163 raft_consensus.cc:348] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.412607 4163 raft_consensus.cc:374] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.412873 4163 raft_consensus.cc:729] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.413132 4163 consensus_queue.cc:260] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.413968 4163 ts_tablet_manager.cc:1428] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260811 18:57:03.415302 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e54e73ee78a4aeca60980925f339f11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.416023 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e54e73ee78a4aeca60980925f339f11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.418883 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd862d7d75f24a05aa54d761e710cd50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.419661 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd862d7d75f24a05aa54d761e710cd50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.420105 4167 tablet_replica.cc:330] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:03.420123 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7d1b5cc176f54784ad79ad1ebd385acc (table default.Table_0a4ee223798a47db804a96036f506e0c [id=f41c0cf4f9b349868da9c90deb5b1972]) successfully deleted
I20260811 18:57:03.420503 4167 raft_consensus.cc:2229] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.420748 4167 raft_consensus.cc:2258] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:03.422284 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e355634612f64c14965d7af2fa442cb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e355634612f64c14965d7af2fa442cb7 already in progress: creating tablet
I20260811 18:57:03.422642 4270 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.423743 4273 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.423795 4272 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.425400 4163 tablet_bootstrap.cc:492] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.426595 4252 mvcc.cc:205] Tried to move back new op lower bound from 7317400057497415680 to 7317400054907678720. Current Snapshot: MvccSnapshot[applied={T|T < 7317400057497415680}]
I20260811 18:57:03.429641 4272 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P 59811bdf82234ef7ac5865dbecfcc388: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.430459 4270 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.432619 4273 tablet.cc:1717] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Skipping requested alter to schema version 4, tablet already version 4
I20260811 18:57:03.434528 4167 ts_tablet_manager.cc:1905] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.439944 4273 tablet.cc:1722] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:03.441787 4272 tablet.cc:1722] T 7b227bc7305f4b2dae28f04e7001553b P 59811bdf82234ef7ac5865dbecfcc388: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:03.446173 4163 tablet_bootstrap.cc:654] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.452764 4270 tablet.cc:1722] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:03.465348 4162 tablet_bootstrap.cc:492] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.465739 4162 ts_tablet_manager.cc:1397] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.005s sys 0.001s
I20260811 18:57:03.467607 4162 raft_consensus.cc:348] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.468022 4162 raft_consensus.cc:374] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.468314 4162 raft_consensus.cc:729] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.468804 4162 consensus_queue.cc:260] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.469945 4162 ts_tablet_manager.cc:1428] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:57:03.473241 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 927fb981490b4cecabe40314d6f71048
I20260811 18:57:03.475615 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 1 got 0
I20260811 18:57:03.476006 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47ce181398cc4821841119e0793fa1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.476687 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47ce181398cc4821841119e0793fa1a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.477175 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.477824 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:03.478480 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 927fb981490b4cecabe40314d6f71048 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 927fb981490b4cecabe40314d6f71048 already in progress: creating tablet
W20260811 18:57:03.480082 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 927fb981490b4cecabe40314d6f71048 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 927fb981490b4cecabe40314d6f71048 already in progress: creating tablet
I20260811 18:57:03.481554 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_52e96fad41744bc5bd6efbbb18e5f692" } new_table_name: "default.Table_7e851b9ff216483bb8768c88e4b6e1b9" modify_external_catalogs: true
I20260811 18:57:03.482896 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.484391 4162 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:03.493443 4165 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.493822 4165 ts_tablet_manager.cc:1397] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.124s user 0.022s sys 0.036s
I20260811 18:57:03.494192 4163 tablet_bootstrap.cc:492] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.494525 4163 ts_tablet_manager.cc:1397] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.005s
I20260811 18:57:03.495527 4165 raft_consensus.cc:348] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.495807 4163 raft_consensus.cc:348] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.495978 4165 raft_consensus.cc:374] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.496194 4163 raft_consensus.cc:374] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.496205 4165 raft_consensus.cc:729] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.496307 4163 raft_consensus.cc:729] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.496658 4165 consensus_queue.cc:260] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.497535 4165 ts_tablet_manager.cc:1428] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:03.497970 4165 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:03.500020 4183 raft_consensus.cc:482] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.500253 4183 raft_consensus.cc:504] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.500804 4167 ts_tablet_manager.cc:1918] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.501137 4167 log.cc:1196] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d24799ded256403e9fad5c1a22b8aa80
I20260811 18:57:03.501310 4183 leader_election.cc:288] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.502095 4165 tablet_bootstrap.cc:654] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.502421 4162 tablet_bootstrap.cc:654] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.503332 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34aeab84310d49c79805ea898b4f87ce" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.503302 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34aeab84310d49c79805ea898b4f87ce" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:03.503827 3891 raft_consensus.cc:2454] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:03.503929 3745 raft_consensus.cc:2454] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:03.504752 3967 leader_election.cc:302] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:03.506100 4183 raft_consensus.cc:2781] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:03.506446 4183 raft_consensus.cc:482] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.506624 4183 raft_consensus.cc:3037] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.511585 4183 raft_consensus.cc:504] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.513650 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34aeab84310d49c79805ea898b4f87ce" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.514115 3891 raft_consensus.cc:3037] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.514055 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34aeab84310d49c79805ea898b4f87ce" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:03.514400 3745 raft_consensus.cc:3037] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.517022 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 34aeab84310d49c79805ea898b4f87ce
I20260811 18:57:03.517863 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet abb530181d7b4d2baf5391808246c74b
I20260811 18:57:03.518994 3891 raft_consensus.cc:2454] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:03.520088 3965 leader_election.cc:302] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
I20260811 18:57:03.520776 4271 raft_consensus.cc:2781] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.521071 4271 raft_consensus.cc:686] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:03.521684 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 (DEFAULT_TABLE table=default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.521756 4271 consensus_queue.cc:237] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.522274 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8426ccbe530446fcbaa162a03cbb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.526702 3745 raft_consensus.cc:2454] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:03.528470 4183 leader_election.cc:288] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.529668 4165 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.529938 4165 ts_tablet_manager.cc:1397] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.010s
I20260811 18:57:03.531467 4165 raft_consensus.cc:348] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.531790 4165 raft_consensus.cc:374] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.526314 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 80cf2375b4c64dbfae4e20003545da60 (DEFAULT_TABLE table=default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.532553 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 807cee88c18340dbb23c51cad54ab56d (DEFAULT_TABLE table=default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.532832 4163 consensus_queue.cc:260] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.533206 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807cee88c18340dbb23c51cad54ab56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.533595 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 80cf2375b4c64dbfae4e20003545da60 (DEFAULT_TABLE table=default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.534050 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cf2375b4c64dbfae4e20003545da60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.533947 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac944e858133489cb57a21e726976f92" } new_table_name: "default.Table_7fa5c52d66624877a73a99af0be2381a" modify_external_catalogs: true
I20260811 18:57:03.534173 4163 ts_tablet_manager.cc:1428] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.039s user 0.003s sys 0.000s
I20260811 18:57:03.534631 4163 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.534780 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 508f0b39cd654aa991a2928a4b067762 (DEFAULT_TABLE table=default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.535326 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508f0b39cd654aa991a2928a4b067762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.535746 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet abb530181d7b4d2baf5391808246c74b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.536240 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03c613798a09459d8d50569ba4639330" } new_table_name: "default.Table_a1cacf3713764528a169a58c4f2812b6" modify_external_catalogs: true
I20260811 18:57:03.537786 4163 tablet_bootstrap.cc:654] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.538929 3722 tablet_service.cc:1467] Processing CreateTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 (DEFAULT_TABLE table=default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.539445 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ce1042ae974f4dbac964bd7ea16a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.539417 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 34aeab84310d49c79805ea898b4f87ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.539812 4169 tablet_replica.cc:330] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:03.540151 4169 raft_consensus.cc:2229] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.540441 4169 raft_consensus.cc:2258] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.540899 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d0d75fef740443968800749bb76bfd2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.541482 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0d75fef740443968800749bb76bfd2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.541522 4169 ts_tablet_manager.cc:1905] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.542860 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1fd04f00945749c89bd1277cfca402f8" } new_table_name: "default.Table_6d55543667e9439eb0420ca7ec6e8e74" modify_external_catalogs: true
I20260811 18:57:03.542941 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet abb530181d7b4d2baf5391808246c74b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.533118 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 (DEFAULT_TABLE table=default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:03.545300 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7f640c60e81b4a369458b1497a269c9c consensus state has no leader
I20260811 18:57:03.545225 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 34aeab84310d49c79805ea898b4f87ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.545882 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8426ccbe530446fcbaa162a03cbb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.546804 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cf2375b4c64dbfae4e20003545da60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.551291 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 508f0b39cd654aa991a2928a4b067762 (DEFAULT_TABLE table=default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.551447 4165 raft_consensus.cc:729] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.551537 4012 tablet_service.cc:1467] Processing CreateTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 (DEFAULT_TABLE table=default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.552042 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508f0b39cd654aa991a2928a4b067762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.551983 4014 tablet_service.cc:1467] Processing CreateTablet for tablet 80cf2375b4c64dbfae4e20003545da60 (DEFAULT_TABLE table=default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.551929 4165 consensus_queue.cc:260] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.552484 4014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80cf2375b4c64dbfae4e20003545da60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.553009 4165 ts_tablet_manager.cc:1428] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.023s user 0.000s sys 0.003s
I20260811 18:57:03.553504 4165 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:03.555029 4167 ts_tablet_manager.cc:1939] T d24799ded256403e9fad5c1a22b8aa80 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:03.556319 4167 tablet_replica.cc:330] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:03.556692 4167 raft_consensus.cc:2229] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:03.560143 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 508f0b39cd654aa991a2928a4b067762 (DEFAULT_TABLE table=default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.562656 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8c0444ec2242299482736a73551f80 (DEFAULT_TABLE table=default.Table_233cfa4d8fe948dbac05d78693cd6d9e [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.563396 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d24799ded256403e9fad5c1a22b8aa80 (table default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]) successfully deleted
I20260811 18:57:03.563942 3859 tablet_service.cc:1467] Processing CreateTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 (DEFAULT_TABLE table=default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.564435 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8426ccbe530446fcbaa162a03cbb1892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.568188 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8c0444ec2242299482736a73551f80 (DEFAULT_TABLE table=default.Table_233cfa4d8fe948dbac05d78693cd6d9e [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.574502 3866 tablet_service.cc:1467] Processing CreateTablet for tablet ed3346bd12d04c45a511308c427856e5 (DEFAULT_TABLE table=default.Table_14ea27ae9206400f82d35d43e532e99b [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.575104 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3346bd12d04c45a511308c427856e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.576066 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 807cee88c18340dbb23c51cad54ab56d (DEFAULT_TABLE table=default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.578766 4196 raft_consensus.cc:482] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.579437 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 33979de5b48c4674a46c8da3b7151753 (DEFAULT_TABLE table=default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.580062 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8c0444ec2242299482736a73551f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.582118 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807cee88c18340dbb23c51cad54ab56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.583189 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8c0444ec2242299482736a73551f80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:03.585646 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 34aeab84310d49c79805ea898b4f87ce (table default.Table_738a85e77f9e49efa9899428ec31a9ab [id=7cda514ae5f24bf785720ec89d44b923], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 34aeab84310d49c79805ea898b4f87ce consensus state has no leader
I20260811 18:57:03.587671 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 508f0b39cd654aa991a2928a4b067762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.589725 4162 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.593932 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 807cee88c18340dbb23c51cad54ab56d (DEFAULT_TABLE table=default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.594023 4167 pending_rounds.cc:62] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:03.594630 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807cee88c18340dbb23c51cad54ab56d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.595664 4007 tablet_service.cc:1514] Processing DeleteTablet for tablet abb530181d7b4d2baf5391808246c74b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.596025 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 33979de5b48c4674a46c8da3b7151753 (DEFAULT_TABLE table=default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.596328 4168 tablet_replica.cc:330] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:03.596203 4196 raft_consensus.cc:504] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.596800 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33979de5b48c4674a46c8da3b7151753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.596724 4009 tablet_service.cc:1467] Processing CreateTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 (DEFAULT_TABLE table=default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.597383 4196 leader_election.cc:288] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:03.596900 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet 34aeab84310d49c79805ea898b4f87ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.597158 4168 raft_consensus.cc:2229] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:03.601285 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8c0444ec2242299482736a73551f80 (DEFAULT_TABLE table=default.Table_233cfa4d8fe948dbac05d78693cd6d9e [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.591017 3712 tablet_service.cc:1467] Processing CreateTablet for tablet ed3346bd12d04c45a511308c427856e5 (DEFAULT_TABLE table=default.Table_14ea27ae9206400f82d35d43e532e99b [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.604259 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5c59d98631a74a07a2d35f6b4bc209a3" } modify_external_catalogs: true
I20260811 18:57:03.604704 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5c59d98631a74a07a2d35f6b4bc209a3" } modify_external_catalogs: true
I20260811 18:57:03.604662 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_59cd4e34981541759a761fe10bf57a2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.605621 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59cd4e34981541759a761fe10bf57a2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.607488 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 33979de5b48c4674a46c8da3b7151753 (DEFAULT_TABLE table=default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.610046 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3346bd12d04c45a511308c427856e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.613510 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ce1042ae974f4dbac964bd7ea16a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.596028 4167 pending_rounds.cc:68] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400055020150784 op_type: NO_OP noop_request { }
I20260811 18:57:03.618109 4167 raft_consensus.cc:2866] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:03.618743 4167 raft_consensus.cc:2258] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.618700 4005 tablet_service.cc:1467] Processing CreateTablet for tablet ed3346bd12d04c45a511308c427856e5 (DEFAULT_TABLE table=default.Table_14ea27ae9206400f82d35d43e532e99b [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.619908 4005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed3346bd12d04c45a511308c427856e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.621708 4167 ts_tablet_manager.cc:1905] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.621697 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3085d74b0bf7447d9c247b7e8872b215" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:03.623255 3678 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.624624 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8c0444ec2242299482736a73551f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.632442 4162 ts_tablet_manager.cc:1397] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.120s user 0.000s sys 0.007s
I20260811 18:57:03.632807 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33979de5b48c4674a46c8da3b7151753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.641386 4168 pending_rounds.cc:62] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Trying to abort 1 pending ops.
I20260811 18:57:03.653658 4168 pending_rounds.cc:68] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400056816119808 op_type: NO_OP noop_request { }
I20260811 18:57:03.654093 4168 raft_consensus.cc:2866] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:03.655112 4168 raft_consensus.cc:2258] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.657400 4165 tablet_bootstrap.cc:654] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.659716 4168 ts_tablet_manager.cc:1905] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.660990 4271 raft_consensus.cc:482] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.666839 4162 raft_consensus.cc:348] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.668947 4162 raft_consensus.cc:374] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.669301 4162 raft_consensus.cc:729] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.670485 4271 raft_consensus.cc:504] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.671345 4271 leader_election.cc:288] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.672538 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3414c134374aa59ac23d276a018ec9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.673298 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3414c134374aa59ac23d276a018ec9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:03.676134 3965 leader_election.cc:341] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.680401 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3085d74b0bf7447d9c247b7e8872b215" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.680838 3891 raft_consensus.cc:2454] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:03.680934 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ce1042ae974f4dbac964bd7ea16a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.682722 3676 leader_election.cc:302] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.670058 4162 consensus_queue.cc:260] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.685382 4178 raft_consensus.cc:482] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.686671 4196 raft_consensus.cc:2781] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:03.686970 4196 raft_consensus.cc:482] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.689376 4293 raft_consensus.cc:482] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.690797 4196 raft_consensus.cc:3037] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.695009 3745 raft_consensus.cc:2454] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:03.696437 3967 leader_election.cc:302] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
W20260811 18:57:03.696449 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 91761.3us
I20260811 18:57:03.698851 4271 raft_consensus.cc:2781] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:03.699110 4271 raft_consensus.cc:482] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.699965 4196 raft_consensus.cc:504] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.701105 4271 raft_consensus.cc:3037] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.705134 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b70decfaca9c4a9f87915d80838c6061"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.706028 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b70decfaca9c4a9f87915d80838c6061 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.710868 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3085d74b0bf7447d9c247b7e8872b215" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:03.711681 3678 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.715497 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_233cfa4d8fe948dbac05d78693cd6d9e" } new_table_name: "default.Table_4eaba0dd4bfc4e45878b027d222974fd" modify_external_catalogs: true
I20260811 18:57:03.701980 4196 leader_election.cc:288] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:03.717764 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 807cee88c18340dbb23c51cad54ab56d
W20260811 18:57:03.719060 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e1efffade0f486d8d627e56e0be6ff6 has no consensus state
I20260811 18:57:03.717432 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8] alter complete (version 4)
I20260811 18:57:03.726963 4162 ts_tablet_manager.cc:1428] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.092s user 0.004s sys 0.001s
I20260811 18:57:03.728395 4162 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:03.731945 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ae200f3814e431a8288c30fe94bffcf has no consensus state
W20260811 18:57:03.732326 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ce8db230c7e41b6a4792273a0f07e6e has no consensus state
I20260811 18:57:03.736392 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:03.723649 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_542b2f198c2a42abb286bbdcf948584b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.739339 4163 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.739583 4163 ts_tablet_manager.cc:1397] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.205s user 0.017s sys 0.051s
I20260811 18:57:03.740033 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33979de5b48c4674a46c8da3b7151753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.739941 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ed6982fd3114b80993a67546347cfb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.740751 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ed6982fd3114b80993a67546347cfb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.740724 4178 raft_consensus.cc:504] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.743160 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f43a7646be343cda0e8f5ad1cfb76cd with delete_type TABLET_DATA_DELETED (replaced by tablet ed3346bd12d04c45a511308c427856e5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.744139 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_89b7704fd6264ef8bd2289a13f9a488d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.744728 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89b7704fd6264ef8bd2289a13f9a488d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.751345 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_de899ce498b24cd5b270adf6fbb81b92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.754093 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3085d74b0bf7447d9c247b7e8872b215" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.754417 3891 raft_consensus.cc:3037] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:03.756394 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de899ce498b24cd5b270adf6fbb81b92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.690615 4293 raft_consensus.cc:504] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.758914 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d35faa013d8c45e29cddf10c7c62e27d" } modify_external_catalogs: true
I20260811 18:57:03.760368 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d35faa013d8c45e29cddf10c7c62e27d" } modify_external_catalogs: true
I20260811 18:57:03.760635 4178 leader_election.cc:288] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:03.761436 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_542b2f198c2a42abb286bbdcf948584b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.761857 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8397339a34974af9b4e8f20778beb8c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.762692 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8397339a34974af9b4e8f20778beb8c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.763522 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.773277 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.774623 4293 leader_election.cc:288] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.776150 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.780351 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:03.780701 4036 raft_consensus.cc:2454] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:03.784286 3824 leader_election.cc:341] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:03.784596 3824 leader_election.cc:302] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
I20260811 18:57:03.789505 4271 raft_consensus.cc:504] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.790434 4163 raft_consensus.cc:348] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.790783 4163 raft_consensus.cc:374] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.790997 4163 raft_consensus.cc:729] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.791491 4163 consensus_queue.cc:260] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.791786 4271 leader_election.cc:288] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.791725 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3414c134374aa59ac23d276a018ec9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:03.789714 4293 raft_consensus.cc:2781] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260811 18:57:03.793078 3965 leader_election.cc:341] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.798136 4196 raft_consensus.cc:482] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.800257 4196 raft_consensus.cc:504] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.801359 4293 raft_consensus.cc:482] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:03.801594 4293 raft_consensus.cc:3037] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.803112 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:03.803398 3824 leader_election.cc:341] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:03.803485 3746 raft_consensus.cc:2454] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:03.803593 3824 leader_election.cc:302] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:03.804201 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3414c134374aa59ac23d276a018ec9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:03.804888 3745 raft_consensus.cc:3037] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.806453 4196 leader_election.cc:288] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:03.809381 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634c06d76eca404a88f326e905882930" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:03.809573 4162 tablet_bootstrap.cc:654] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:03.810225 3676 leader_election.cc:341] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.817914 4178 raft_consensus.cc:2726] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.818084 3745 raft_consensus.cc:2454] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:03.820050 4167 ts_tablet_manager.cc:1918] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:03.820320 4167 log.cc:1196] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3b1bd859a80441068cccd993263dc4e1
I20260811 18:57:03.820878 4167 ts_tablet_manager.cc:1939] T 3b1bd859a80441068cccd993263dc4e1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:03.821818 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 53) took 1035 ms. Trace:
I20260811 18:57:03.822023 4167 rpcz_store.cc:276] 0811 18:57:02.786151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:02.792638 (+ 6487us) service_pool.cc:227] Handling call
0811 18:57:03.556209 (+763571us) ts_tablet_manager.cc:1145] Deleting tablet 3b1bd859a80441068cccd993263dc4e1
0811 18:57:03.556221 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:03.814491 (+258270us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.819902 (+ 5411us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.821733 (+ 1831us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":762629}
I20260811 18:57:03.822923 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 54) took 1036 ms. Trace:
I20260811 18:57:03.823128 4167 rpcz_store.cc:276] 0811 18:57:02.786266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:02.792684 (+ 6418us) service_pool.cc:227] Handling call
0811 18:57:03.822608 (+1029924us) ts_tablet_manager.cc:1145] Deleting tablet 8964f92255884b3aae0cac60f019fdb7
0811 18:57:03.822620 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:03.822781 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1028837}
W20260811 18:57:03.824124 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8964f92255884b3aae0cac60f019fdb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8964f92255884b3aae0cac60f019fdb7 already in progress: creating tablet
W20260811 18:57:03.824512 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 82d1339366a24a87b74bd1d411a3f6fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82d1339366a24a87b74bd1d411a3f6fe already in progress: creating tablet
W20260811 18:57:03.824846 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8efb663060ef457cbece962fce701eb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8efb663060ef457cbece962fce701eb1 already in progress: creating tablet
I20260811 18:57:03.824993 4167 tablet_replica.cc:330] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:03.825201 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 39a040ea35974f709a7369080925d470 because the tablet was not found. No further retry: Not found: Tablet not found: 39a040ea35974f709a7369080925d470
I20260811 18:57:03.825253 4167 raft_consensus.cc:2229] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:03.825649 4167 pending_rounds.cc:62] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:03.825798 4167 pending_rounds.cc:68] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400056752476160 op_type: NO_OP noop_request { }
I20260811 18:57:03.826211 4167 raft_consensus.cc:2866] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:03.826380 4167 raft_consensus.cc:2258] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.827431 4167 ts_tablet_manager.cc:1905] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.834357 4163 ts_tablet_manager.cc:1428] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.083s user 0.000s sys 0.004s
I20260811 18:57:03.835243 4163 tablet_bootstrap.cc:492] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:03.841099 3967 leader_election.cc:302] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:03.843624 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634c06d76eca404a88f326e905882930" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:03.846292 4162 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.846650 4162 ts_tablet_manager.cc:1397] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.003s sys 0.006s
I20260811 18:57:03.848076 4162 raft_consensus.cc:348] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.848968 4162 raft_consensus.cc:374] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:03.849220 4162 raft_consensus.cc:729] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:03.849639 4162 consensus_queue.cc:260] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.852608 4183 raft_consensus.cc:2781] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:03.851374 3678 leader_election.cc:341] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.852901 4183 raft_consensus.cc:686] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:03.852980 3678 leader_election.cc:302] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.853397 4183 consensus_queue.cc:237] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.853704 4196 raft_consensus.cc:2726] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:03.854936 4234 consensus_queue.cc:1035] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:03.859119 3891 raft_consensus.cc:2454] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:03.860590 3676 leader_election.cc:302] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:03.861106 4196 raft_consensus.cc:2781] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.865918 4162 ts_tablet_manager.cc:1428] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:03.866766 4163 tablet_bootstrap.cc:654] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.867333 4162 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:03.880035 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 47219us
I20260811 18:57:03.880679 4196 raft_consensus.cc:686] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:03.881400 4196 consensus_queue.cc:237] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.882586 4162 tablet_bootstrap.cc:654] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:03.884635 4293 raft_consensus.cc:504] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.889894 4169 ts_tablet_manager.cc:1918] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:03.903184 4168 ts_tablet_manager.cc:1918] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:03.904743 4168 log.cc:1196] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/abb530181d7b4d2baf5391808246c74b
I20260811 18:57:03.906616 4169 log.cc:1196] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/34aeab84310d49c79805ea898b4f87ce
I20260811 18:57:03.907285 4169 ts_tablet_manager.cc:1939] T 34aeab84310d49c79805ea898b4f87ce P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:03.911028 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:03.911984 4293 leader_election.cc:288] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:03.913664 4036 raft_consensus.cc:3037] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.914829 4165 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:03.915242 4165 ts_tablet_manager.cc:1397] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.362s user 0.010s sys 0.018s
I20260811 18:57:03.911636 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e355634612f64c14965d7af2fa442cb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:03.916591 3746 raft_consensus.cc:3037] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:03.919392 4036 raft_consensus.cc:2454] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:03.920009 4168 ts_tablet_manager.cc:1939] T abb530181d7b4d2baf5391808246c74b P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:03.921478 3746 raft_consensus.cc:2454] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:03.922324 3824 leader_election.cc:302] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:03.923005 4293 raft_consensus.cc:2781] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:03.924096 4293 raft_consensus.cc:686] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:03.924458 4167 ts_tablet_manager.cc:1918] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:03.924777 4167 log.cc:1196] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/1f43a7646be343cda0e8f5ad1cfb76cd
I20260811 18:57:03.924809 4293 consensus_queue.cc:237] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:03.927780 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9f8c0444ec2242299482736a73551f80 consensus state has no leader
I20260811 18:57:03.933334 4167 ts_tablet_manager.cc:1939] T 1f43a7646be343cda0e8f5ad1cfb76cd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:03.933656 4169 tablet_replica.cc:330] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:03.933471 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_593bb155f3de4ed9a2913e3114d49e5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:03.933945 4169 raft_consensus.cc:2229] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:03.934530 4169 raft_consensus.cc:2258] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.934698 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 59) took 1021 ms. Trace:
I20260811 18:57:03.934907 4167 rpcz_store.cc:276] 0811 18:57:02.913079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:02.913305 (+ 226us) service_pool.cc:227] Handling call
0811 18:57:03.824789 (+911484us) ts_tablet_manager.cc:1145] Deleting tablet 1f43a7646be343cda0e8f5ad1cfb76cd
0811 18:57:03.824803 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:03.831439 (+ 6636us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.924214 (+ 92775us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.934504 (+ 10290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":911160}
I20260811 18:57:03.935621 4167 tablet_replica.cc:330] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:03.935554 4169 ts_tablet_manager.cc:1905] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:03.935910 4167 raft_consensus.cc:2229] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:03.942114 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_593bb155f3de4ed9a2913e3114d49e5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.945816 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 1f43a7646be343cda0e8f5ad1cfb76cd (table default.Table_14ea27ae9206400f82d35d43e532e99b [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
I20260811 18:57:03.936276 4167 raft_consensus.cc:2258] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:03.949504 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_84792678a95b4b7a9ceffcde9ddf3fdf" } new_table_name: "default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db" modify_external_catalogs: true
I20260811 18:57:03.952191 4162 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:03.952486 4162 ts_tablet_manager.cc:1397] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.085s user 0.010s sys 0.010s
I20260811 18:57:03.953383 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c56b0614a7843bea868d1e3fd95de68" } modify_external_catalogs: true
I20260811 18:57:03.953713 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c56b0614a7843bea868d1e3fd95de68" } modify_external_catalogs: true
I20260811 18:57:03.954880 4162 raft_consensus.cc:348] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.957015 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_174aeae5133245c3ba894064a8d08dd6" } modify_external_catalogs: true
I20260811 18:57:03.957247 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_174aeae5133245c3ba894064a8d08dd6" } modify_external_catalogs: true
I20260811 18:57:03.957978 4163 tablet_bootstrap.cc:492] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:03.958709 4178 raft_consensus.cc:482] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:03.958981 4178 raft_consensus.cc:504] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.960319 4178 leader_election.cc:288] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:03.962154 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8964f92255884b3aae0cac60f019fdb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:03.963049 3824 leader_election.cc:341] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:03.963289 4163 ts_tablet_manager.cc:1397] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.123s user 0.003s sys 0.003s
I20260811 18:57:03.966537 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e (DEFAULT_TABLE table=default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.935322 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9fe9862d3266414295d433fd5b136638"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.967767 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fe9862d3266414295d433fd5b136638 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.969425 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 34aeab84310d49c79805ea898b4f87ce (table default.Table_738a85e77f9e49efa9899428ec31a9ab [id=7cda514ae5f24bf785720ec89d44b923]) successfully deleted
I20260811 18:57:03.975371 4163 raft_consensus.cc:348] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:03.979264 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 80cf2375b4c64dbfae4e20003545da60
I20260811 18:57:03.980499 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 2852f9e5f67c4025836018af553388a5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.982164 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:03.982863 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 494c09652d9e408880ca606ae0b30773 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.990571 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cf285cdf5235451bb1531cb294017101
I20260811 18:57:03.991392 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.991966 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 82d1339366a24a87b74bd1d411a3f6fe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:03.994437 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_14ea27ae9206400f82d35d43e532e99b" } new_table_name: "default.Table_9d18dc31cde54577b8bcc329191b74ef" modify_external_catalogs: true
I20260811 18:57:03.995604 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 60274115fe164d85badcf2e72d640f54 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.006547 4163 raft_consensus.cc:374] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.010149 4163 raft_consensus.cc:729] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.011137 4163 consensus_queue.cc:260] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.012122 4163 ts_tablet_manager.cc:1428] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.049s user 0.004s sys 0.000s
W20260811 18:57:04.013947 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a300a7aca2814da6b999eb4d865fea66 consensus state has no leader
I20260811 18:57:04.014026 4163 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.022226 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_178c66b5e0f1479a9a97248ecab4e779 [id=76fa1a8be6ed4f28932466dba0a991db] alter complete (version 1)
I20260811 18:57:04.022624 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:04.023661 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_457be7b012ba40deab01ec6480b82ede"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.024519 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_457be7b012ba40deab01ec6480b82ede in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.026489 4169 ts_tablet_manager.cc:1918] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.026822 4169 log.cc:1196] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/abb530181d7b4d2baf5391808246c74b
I20260811 18:57:04.026775 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47ce181398cc4821841119e0793fa1a8" } new_table_name: "default.Table_5c143dbecb50403790cf8e126be5b651" modify_external_catalogs: true
I20260811 18:57:04.027968 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e (DEFAULT_TABLE table=default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.030874 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de7a638d5c564c84b20b870f9c3a97e5" } modify_external_catalogs: true
I20260811 18:57:04.031172 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de7a638d5c564c84b20b870f9c3a97e5" } modify_external_catalogs: true
I20260811 18:57:04.033717 4169 ts_tablet_manager.cc:1939] T abb530181d7b4d2baf5391808246c74b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.034688 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f924af711624a8298197fae459c7e15" } new_table_name: "default.Table_d62ce6ce67e045c3a28f6c71419187b5" modify_external_catalogs: true
I20260811 18:57:04.035223 4169 tablet_replica.cc:330] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.037473 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_79396737f1a343c98184686b7eda44ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.038311 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79396737f1a343c98184686b7eda44ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.036334 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce8db230c7e41b6a4792273a0f07e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.037016 3857 tablet_service.cc:1467] Processing CreateTablet for tablet dde27d8cc3eb4d09879e717fc90305fd (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.038502 3856 tablet_service.cc:1467] Processing CreateTablet for tablet cf285cdf5235451bb1531cb294017101 (DEFAULT_TABLE table=default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.038959 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 3c70d9db064e45af91fd9901d85e211f (DEFAULT_TABLE table=default.Table_27837256c8254b8c9b6493a7ca044751 [id=730987527e1749bebaa10e860079be4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.039348 3863 tablet_service.cc:1467] Processing CreateTablet for tablet a30bb384136f4a169a5751655437d5a3 (DEFAULT_TABLE table=default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.040176 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.040237 3870 tablet_service.cc:1467] Processing CreateTablet for tablet a300a7aca2814da6b999eb4d865fea66 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.040993 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.041040 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.044145 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.041303 4163 tablet_bootstrap.cc:654] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.044832 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 (DEFAULT_TABLE table=default.Table_6d55543667e9439eb0420ca7ec6e8e74 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.045382 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 151df87806694f46be5013592c68c01a (DEFAULT_TABLE table=default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:04.046635 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ed3346bd12d04c45a511308c427856e5 consensus state has no leader
I20260811 18:57:04.052731 3855 tablet_service.cc:1467] Processing CreateTablet for tablet d3d6daff4b7540309ff58755758e9752 (DEFAULT_TABLE table=default.Table_47ce181398cc4821841119e0793fa1a8 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.056663 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.058511 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae200f3814e431a8288c30fe94bffcf (DEFAULT_TABLE table=default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.059379 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde27d8cc3eb4d09879e717fc90305fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.060163 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf285cdf5235451bb1531cb294017101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.946681 4165 raft_consensus.cc:348] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.963315 4004 tablet_service.cc:1467] Processing CreateTablet for tablet dde27d8cc3eb4d09879e717fc90305fd (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.060702 4165 raft_consensus.cc:374] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.060885 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce8db230c7e41b6a4792273a0f07e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.060879 4165 raft_consensus.cc:729] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.061038 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde27d8cc3eb4d09879e717fc90305fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.963687 4013 tablet_service.cc:1467] Processing CreateTablet for tablet cf285cdf5235451bb1531cb294017101 (DEFAULT_TABLE table=default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.964452 4009 tablet_service.cc:1467] Processing CreateTablet for tablet a30bb384136f4a169a5751655437d5a3 (DEFAULT_TABLE table=default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.061362 4165 consensus_queue.cc:260] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:03.970983 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 3c70d9db064e45af91fd9901d85e211f (DEFAULT_TABLE table=default.Table_27837256c8254b8c9b6493a7ca044751 [id=730987527e1749bebaa10e860079be4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.971582 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.062202 4165 ts_tablet_manager.cc:1428] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.147s user 0.002s sys 0.002s
I20260811 18:57:04.062729 4165 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:04.067137 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a30bb384136f4a169a5751655437d5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.068928 4165 tablet_bootstrap.cc:654] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.070600 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf285cdf5235451bb1531cb294017101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.071735 4169 raft_consensus.cc:2229] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.073359 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a300a7aca2814da6b999eb4d865fea66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.074479 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae200f3814e431a8288c30fe94bffcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.074636 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c70d9db064e45af91fd9901d85e211f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.077981 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d6daff4b7540309ff58755758e9752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.078673 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9fa42654cf4ec18220e5fb173ddef7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:04.077541 4303 log.cc:924] Time spent T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Append to log took a long time: real 0.084s user 0.001s sys 0.000s
I20260811 18:57:04.079464 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]). Expected version 1 got 0
I20260811 18:57:04.082754 3723 tablet_service.cc:1467] Processing CreateTablet for tablet dde27d8cc3eb4d09879e717fc90305fd (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.083508 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde27d8cc3eb4d09879e717fc90305fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.084270 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.090513 4169 raft_consensus.cc:2258] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.094170 4169 ts_tablet_manager.cc:1905] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.095584 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.096048 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 8964f92255884b3aae0cac60f019fdb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:03.971900 4005 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e (DEFAULT_TABLE table=default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.982009 4012 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.984270 4014 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:03.986925 4010 tablet_service.cc:1467] Processing CreateTablet for tablet a300a7aca2814da6b999eb4d865fea66 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.988485 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 (DEFAULT_TABLE table=default.Table_6d55543667e9439eb0420ca7ec6e8e74 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.103328 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]) successfully deleted
I20260811 18:57:04.103514 4005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce8db230c7e41b6a4792273a0f07e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.990603 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae200f3814e431a8288c30fe94bffcf (DEFAULT_TABLE table=default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.104065 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae200f3814e431a8288c30fe94bffcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.105791 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.106132 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9fa42654cf4ec18220e5fb173ddef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.993337 4002 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.001080 4001 tablet_service.cc:1467] Processing CreateTablet for tablet 151df87806694f46be5013592c68c01a (DEFAULT_TABLE table=default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.107018 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151df87806694f46be5013592c68c01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.107484 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151df87806694f46be5013592c68c01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.003973 4000 tablet_service.cc:1467] Processing CreateTablet for tablet d3d6daff4b7540309ff58755758e9752 (DEFAULT_TABLE table=default.Table_47ce181398cc4821841119e0793fa1a8 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.110991 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d6daff4b7540309ff58755758e9752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.114650 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a300a7aca2814da6b999eb4d865fea66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.118678 4162 raft_consensus.cc:374] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.118820 4234 raft_consensus.cc:482] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.118899 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1efffade0f486d8d627e56e0be6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.119062 4234 raft_consensus.cc:504] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.120091 4234 leader_election.cc:288] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:04.123574 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet dd465168aecf43a1b08b9c11d241f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.104243 3712 tablet_service.cc:1467] Processing CreateTablet for tablet cf285cdf5235451bb1531cb294017101 (DEFAULT_TABLE table=default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.124204 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 151df87806694f46be5013592c68c01a (DEFAULT_TABLE table=default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.124887 4167 ts_tablet_manager.cc:1905] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.125663 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d24799ded256403e9fad5c1a22b8aa80" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:04.126516 3965 leader_election.cc:341] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:04.130539 3722 tablet_service.cc:1467] Processing CreateTablet for tablet a300a7aca2814da6b999eb4d865fea66 (DEFAULT_TABLE table=default.Table_84792678a95b4b7a9ceffcde9ddf3fdf [id=7818472e23a14febb1990a619b485823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.130858 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.131588 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 (DEFAULT_TABLE table=default.Table_6d55543667e9439eb0420ca7ec6e8e74 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.132930 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.133453 4162 raft_consensus.cc:729] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.136157 3719 tablet_service.cc:1467] Processing CreateTablet for tablet a30bb384136f4a169a5751655437d5a3 (DEFAULT_TABLE table=default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.136785 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a30bb384136f4a169a5751655437d5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.138383 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1efffade0f486d8d627e56e0be6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.138594 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 3c70d9db064e45af91fd9901d85e211f (DEFAULT_TABLE table=default.Table_27837256c8254b8c9b6493a7ca044751 [id=730987527e1749bebaa10e860079be4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.139088 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.141964 3720 tablet_service.cc:1467] Processing CreateTablet for tablet d3d6daff4b7540309ff58755758e9752 (DEFAULT_TABLE table=default.Table_47ce181398cc4821841119e0793fa1a8 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.142563 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c70d9db064e45af91fd9901d85e211f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.142660 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae200f3814e431a8288c30fe94bffcf (DEFAULT_TABLE table=default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.143064 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf285cdf5235451bb1531cb294017101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.143393 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9fa42654cf4ec18220e5fb173ddef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.143713 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151df87806694f46be5013592c68c01a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:04.145737 3967 leader_election.cc:341] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d24799ded256403e9fad5c1a22b8aa80
I20260811 18:57:04.146669 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3fe670d2d1474485834592fa8f239e62" } new_table_name: "default.Table_0d00075c1e094c1cb6a03b6760496fb5" modify_external_catalogs: true
I20260811 18:57:04.150245 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ace95690ebf4f619a57d3d7bd47bec5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:04.156960 3967 leader_election.cc:302] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:04.158027 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.158411 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1efffade0f486d8d627e56e0be6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.159441 4234 raft_consensus.cc:2726] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:04.005627 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8964f92255884b3aae0cac60f019fdb7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:04.030920 3999 tablet_service.cc:1514] Processing DeleteTablet for tablet 3085d74b0bf7447d9c247b7e8872b215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.042485 3997 tablet_service.cc:1514] Processing DeleteTablet for tablet d24799ded256403e9fad5c1a22b8aa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.042835 3998 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.043557 3996 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3414c134374aa59ac23d276a018ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.060184 4183 consensus_queue.cc:1035] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:04.161427 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ace95690ebf4f619a57d3d7bd47bec5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:04.168866 3824 leader_election.cc:341] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:04.169144 3824 leader_election.cc:302] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:04.169301 4162 consensus_queue.cc:260] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.169549 4293 raft_consensus.cc:2726] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:04.170225 4162 ts_tablet_manager.cc:1428] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.217s user 0.004s sys 0.000s
I20260811 18:57:04.176064 4165 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:04.176405 4165 ts_tablet_manager.cc:1397] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.114s user 0.007s sys 0.000s
I20260811 18:57:04.180562 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a300a7aca2814da6b999eb4d865fea66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.181898 4165 raft_consensus.cc:348] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.182427 4165 raft_consensus.cc:374] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.183334 4162 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:04.172499 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet e355634612f64c14965d7af2fa442cb7 with delete_type TABLET_DATA_DELETED (replaced by tablet b1ab2eee21264b1aa5ba00528cae3843) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.185245 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a30bb384136f4a169a5751655437d5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.187000 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d6daff4b7540309ff58755758e9752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.188179 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae200f3814e431a8288c30fe94bffcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.195611 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c70d9db064e45af91fd9901d85e211f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.196768 4165 raft_consensus.cc:729] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.197302 4165 consensus_queue.cc:260] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.206678 4183 raft_consensus.cc:482] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.206871 4183 raft_consensus.cc:504] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.208377 4183 leader_election.cc:288] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:04.211019 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fb981490b4cecabe40314d6f71048" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:04.214627 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fb981490b4cecabe40314d6f71048" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:04.215183 3746 raft_consensus.cc:2454] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:04.218571 4169 ts_tablet_manager.cc:1918] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:04.219800 3965 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:04.220906 3967 leader_election.cc:302] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:04.221498 4183 raft_consensus.cc:2781] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:04.222276 4183 raft_consensus.cc:482] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:04.222478 4183 raft_consensus.cc:3037] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.235168 3370 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111),
I20260811 18:57:04.235404 3370 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:04.237087 4162 tablet_bootstrap.cc:654] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.241223 4169 log.cc:1196] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/1f43a7646be343cda0e8f5ad1cfb76cd
W20260811 18:57:04.252112 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889): Couldn't send request to peer d85eb97df7824bc9bad9dca349b0c180. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:04.252573 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 51672.8us
I20260811 18:57:04.252971 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]). Expected version 3 got 0
I20260811 18:57:04.260284 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]) successfully deleted
I20260811 18:57:04.281603 4165 ts_tablet_manager.cc:1428] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.049s user 0.000s sys 0.003s
I20260811 18:57:04.286996 4169 ts_tablet_manager.cc:1939] T 1f43a7646be343cda0e8f5ad1cfb76cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.288815 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 1f43a7646be343cda0e8f5ad1cfb76cd (table default.Table_9d18dc31cde54577b8bcc329191b74ef [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:04.289862 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 807cee88c18340dbb23c51cad54ab56d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 807cee88c18340dbb23c51cad54ab56d already in progress: creating tablet
I20260811 18:57:04.296865 4168 tablet_replica.cc:330] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.299088 4165 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:04.306213 4234 raft_consensus.cc:482] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.306502 4234 raft_consensus.cc:504] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.308081 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:04.308709 3891 raft_consensus.cc:2454] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 0.
I20260811 18:57:04.311610 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:04.313728 4234 leader_election.cc:288] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:04.313999 3965 leader_election.cc:302] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
W20260811 18:57:04.315258 3967 leader_election.cc:341] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:04.320513 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b1ce1042ae974f4dbac964bd7ea16a94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1ce1042ae974f4dbac964bd7ea16a94 already in progress: creating tablet
W20260811 18:57:04.321992 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 80cf2375b4c64dbfae4e20003545da60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80cf2375b4c64dbfae4e20003545da60 already in progress: creating tablet
W20260811 18:57:04.322923 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cf285cdf5235451bb1531cb294017101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf285cdf5235451bb1531cb294017101 already in progress: creating tablet
I20260811 18:57:04.323395 4169 tablet_replica.cc:330] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.323671 4169 raft_consensus.cc:2229] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.324002 4169 raft_consensus.cc:2258] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.324939 4169 ts_tablet_manager.cc:1905] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.330924 4234 raft_consensus.cc:2781] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:04.315544 4183 raft_consensus.cc:504] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.335421 4183 leader_election.cc:288] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:04.339535 4234 raft_consensus.cc:482] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:04.339888 4234 raft_consensus.cc:3037] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.346819 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fb981490b4cecabe40314d6f71048" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:04.348527 4168 raft_consensus.cc:2229] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:04.349251 3965 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:04.351227 3553 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]). Expected version 1 got 0
I20260811 18:57:04.357851 4234 raft_consensus.cc:504] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.359608 4162 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:04.360064 4162 ts_tablet_manager.cc:1397] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.188s user 0.011s sys 0.015s
I20260811 18:57:04.361631 4162 raft_consensus.cc:348] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.362186 4162 raft_consensus.cc:374] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.362514 4162 raft_consensus.cc:729] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.362978 4162 consensus_queue.cc:260] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.365355 4169 ts_tablet_manager.cc:1918] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.367686 4168 pending_rounds.cc:62] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Trying to abort 1 pending ops.
I20260811 18:57:04.368420 4162 ts_tablet_manager.cc:1428] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:04.369971 4162 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:04.374732 4169 log.cc:1196] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:04.376014 4169 ts_tablet_manager.cc:1939] T 3085d74b0bf7447d9c247b7e8872b215 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.376478 4162 tablet_bootstrap.cc:654] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.370663 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:04.381539 3891 raft_consensus.cc:3037] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.384075 4169 tablet_replica.cc:330] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.384994 4169 raft_consensus.cc:2229] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:04.389683 4168 pending_rounds.cc:68] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400057951428608 op_type: NO_OP noop_request { }
I20260811 18:57:04.390308 4168 raft_consensus.cc:2866] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:04.383277 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 3085d74b0bf7447d9c247b7e8872b215 (table default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]) successfully deleted
I20260811 18:57:04.388171 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 60274115fe164d85badcf2e72d640f54
I20260811 18:57:04.393618 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 60274115fe164d85badcf2e72d640f54 deleted and replaced by tablet 9ad616256b9949ceaf8b300f660d00c5
I20260811 18:57:04.394320 4162 tablet_bootstrap.cc:492] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:04.394738 4162 ts_tablet_manager.cc:1397] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.010s
I20260811 18:57:04.395442 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]). Expected version 3 got 0
I20260811 18:57:04.396571 4162 raft_consensus.cc:348] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.397239 4162 raft_consensus.cc:374] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.397397 4162 raft_consensus.cc:729] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.398046 4162 consensus_queue.cc:260] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.398978 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef02aae7a42c46678973fed0d27e7421"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.400549 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef02aae7a42c46678973fed0d27e7421 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.401425 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.404140 4162 ts_tablet_manager.cc:1428] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260811 18:57:04.409118 3891 raft_consensus.cc:2454] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:04.411268 4168 raft_consensus.cc:2258] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.415262 4162 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:04.418689 4162 tablet_bootstrap.cc:654] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.421458 4167 ts_tablet_manager.cc:1918] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:04.424494 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d3d6daff4b7540309ff58755758e9752 consensus state has no leader
I20260811 18:57:04.425662 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 634c06d76eca404a88f326e905882930
I20260811 18:57:04.427888 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2852f9e5f67c4025836018af553388a5
I20260811 18:57:04.428905 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 2852f9e5f67c4025836018af553388a5 deleted and replaced by tablet 987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:04.422508 4234 leader_election.cc:288] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:04.444684 4168 ts_tablet_manager.cc:1905] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.446727 4165 tablet_bootstrap.cc:654] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.459756 4165 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:04.460457 4165 ts_tablet_manager.cc:1397] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.005s
I20260811 18:57:04.462486 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_219170f2f267485bb7b6551dc4e03d1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:04.463804 4165 raft_consensus.cc:348] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:04.464102 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_219170f2f267485bb7b6551dc4e03d1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.464848 4165 raft_consensus.cc:374] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.465052 4165 raft_consensus.cc:729] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.465564 4165 consensus_queue.cc:260] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.467664 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cb0316f2e1db4353b9919edf3d8e1c90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.468623 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb0316f2e1db4353b9919edf3d8e1c90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.470561 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_11582479665b4ae58e3bfc2103bddde9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.471627 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11582479665b4ae58e3bfc2103bddde9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.473627 4165 ts_tablet_manager.cc:1428] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.013s user 0.000s sys 0.003s
I20260811 18:57:04.473958 4168 ts_tablet_manager.cc:1918] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:04.475322 4168 log.cc:1196] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/34aeab84310d49c79805ea898b4f87ce
W20260811 18:57:04.475322 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 58476.4us
I20260811 18:57:04.411916 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.412040 4169 pending_rounds.cc:62] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:04.475786 4169 pending_rounds.cc:68] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400059601604608 op_type: NO_OP noop_request { }
I20260811 18:57:04.418057 4163 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.427659 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 634c06d76eca404a88f326e905882930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.428944 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.476125 4169 raft_consensus.cc:2866] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:04.476827 4169 raft_consensus.cc:2258] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.476509 4163 ts_tablet_manager.cc:1397] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.464s user 0.019s sys 0.029s
I20260811 18:57:04.477651 4165 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:04.479400 4163 raft_consensus.cc:348] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.479759 4163 raft_consensus.cc:374] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.480940 4163 raft_consensus.cc:729] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.481421 4163 consensus_queue.cc:260] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.482576 4163 ts_tablet_manager.cc:1428] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:57:04.483328 4163 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.482877 4169 ts_tablet_manager.cc:1905] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.492307 4168 ts_tablet_manager.cc:1939] T 34aeab84310d49c79805ea898b4f87ce P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:04.493176 4163 tablet_bootstrap.cc:654] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.493992 4168 tablet_replica.cc:330] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.494457 4168 raft_consensus.cc:2229] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.430145 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 634c06d76eca404a88f326e905882930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.434719 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.497253 3522 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: Network error: Client connection negotiation failed: client connection to 127.3.72.125:38889: connect: Connection refused (error 111),
I20260811 18:57:04.497443 4167 log.cc:1196] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:04.450865 4162 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:04.497781 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:04.497864 4162 ts_tablet_manager.cc:1397] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.002s
I20260811 18:57:04.498142 4168 raft_consensus.cc:2258] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.499224 4168 ts_tablet_manager.cc:1905] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.501931 4162 raft_consensus.cc:348] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.502410 4162 raft_consensus.cc:374] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.502662 4162 raft_consensus.cc:729] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.503072 4162 consensus_queue.cc:260] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.494591 4165 tablet_bootstrap.cc:654] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.505539 4162 ts_tablet_manager.cc:1428] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260811 18:57:04.505960 4162 tablet_bootstrap.cc:492] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:04.509148 4162 tablet_bootstrap.cc:654] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:04.496599 3967 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 174855us
I20260811 18:57:04.512836 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fb981490b4cecabe40314d6f71048" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:04.513276 3746 raft_consensus.cc:3037] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.514127 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 34aeab84310d49c79805ea898b4f87ce (table default.Table_738a85e77f9e49efa9899428ec31a9ab [id=7cda514ae5f24bf785720ec89d44b923]) successfully deleted
I20260811 18:57:04.515820 4167 ts_tablet_manager.cc:1939] T 0f3414c134374aa59ac23d276a018ec9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:04.515925 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 634c06d76eca404a88f326e905882930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.516886 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 60) took 1578 ms. Trace:
I20260811 18:57:04.517068 4167 rpcz_store.cc:276] 0811 18:57:02.937999 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:02.947023 (+ 9024us) service_pool.cc:227] Handling call
0811 18:57:03.935437 (+988414us) ts_tablet_manager.cc:1145] Deleting tablet 0f3414c134374aa59ac23d276a018ec9
0811 18:57:03.935453 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.289650 (+354197us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:04.421206 (+131556us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:04.516840 (+ 95634us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":985298}
I20260811 18:57:04.517726 3746 raft_consensus.cc:2454] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:04.517720 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 66) took 1458 ms. Trace:
I20260811 18:57:04.518044 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.518261 4167 rpcz_store.cc:276] 0811 18:57:03.059001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.059059 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:04.517416 (+1458357us) ts_tablet_manager.cc:1145] Deleting tablet 8af3e74889b64c408c9ed9c54916f890
0811 18:57:04.517430 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.517591 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1458008}
I20260811 18:57:04.518853 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 67) took 1457 ms. Trace:
I20260811 18:57:04.519323 4167 rpcz_store.cc:276] 0811 18:57:03.061354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.061424 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:04.518538 (+1457114us) ts_tablet_manager.cc:1145] Deleting tablet 5df7a07983174dd1aaf48951c9ff1a1b
0811 18:57:04.518548 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.518729 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1456722}
I20260811 18:57:04.519821 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 74) took 1292 ms. Trace:
I20260811 18:57:04.519964 4169 ts_tablet_manager.cc:1918] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:04.520244 4169 log.cc:1196] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e355634612f64c14965d7af2fa442cb7
I20260811 18:57:04.521055 4169 ts_tablet_manager.cc:1939] T e355634612f64c14965d7af2fa442cb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.522537 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e355634612f64c14965d7af2fa442cb7 (table default.Table_3fe670d2d1474485834592fa8f239e62 [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:04.524302 4169 tablet_replica.cc:330] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.524371 3965 leader_election.cc:302] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388; no voters:
I20260811 18:57:04.524633 4169 raft_consensus.cc:2229] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.524756 4271 raft_consensus.cc:2781] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:04.524907 4169 raft_consensus.cc:2258] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.524986 4271 raft_consensus.cc:686] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:04.525309 4271 consensus_queue.cc:237] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.525667 4169 ts_tablet_manager.cc:1905] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.527845 4168 ts_tablet_manager.cc:1918] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.528403 4168 log.cc:1196] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/e355634612f64c14965d7af2fa442cb7
I20260811 18:57:04.528980 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0f3414c134374aa59ac23d276a018ec9 (table default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]) successfully deleted
I20260811 18:57:04.529385 4168 ts_tablet_manager.cc:1939] T e355634612f64c14965d7af2fa442cb7 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
W20260811 18:57:04.530005 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8af3e74889b64c408c9ed9c54916f890 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8af3e74889b64c408c9ed9c54916f890 already in progress: creating tablet
W20260811 18:57:04.531618 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5df7a07983174dd1aaf48951c9ff1a1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5df7a07983174dd1aaf48951c9ff1a1b already in progress: creating tablet
W20260811 18:57:04.533071 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c18120da2d245f495c52a670047760d consensus state has no leader
I20260811 18:57:04.534134 4163 tablet_bootstrap.cc:492] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.534507 4163 ts_tablet_manager.cc:1397] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.000s
I20260811 18:57:04.536229 4163 raft_consensus.cc:348] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.536679 4163 raft_consensus.cc:374] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.536855 4163 raft_consensus.cc:729] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.537305 4163 consensus_queue.cc:260] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.538151 4163 ts_tablet_manager.cc:1428] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260811 18:57:04.538002 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.537716 3967 leader_election.cc:302] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24
I20260811 18:57:04.541061 4234 raft_consensus.cc:2781] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:04.541285 4234 raft_consensus.cc:686] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Becoming Leader. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Running, Role: LEADER
I20260811 18:57:04.541945 4163 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.541675 4234 consensus_queue.cc:237] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.546222 4163 tablet_bootstrap.cc:654] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.535974 4168 tablet_replica.cc:330] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.547768 4168 raft_consensus.cc:2229] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.548601 4168 raft_consensus.cc:2258] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.550410 4168 ts_tablet_manager.cc:1905] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.553382 4163 tablet_bootstrap.cc:492] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.553848 4163 ts_tablet_manager.cc:1397] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 18:57:04.556644 4163 raft_consensus.cc:348] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.557070 4163 raft_consensus.cc:374] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.557405 4163 raft_consensus.cc:729] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.557993 4163 consensus_queue.cc:260] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.559602 4163 ts_tablet_manager.cc:1428] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260811 18:57:04.560055 4163 tablet_bootstrap.cc:492] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.569110 4163 tablet_bootstrap.cc:654] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.576054 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet e355634612f64c14965d7af2fa442cb7 (table default.Table_3fe670d2d1474485834592fa8f239e62 [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:04.579357 4004 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:04.579838 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 807cee88c18340dbb23c51cad54ab56d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 807cee88c18340dbb23c51cad54ab56d already in progress: creating tablet
W20260811 18:57:04.583871 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 consensus state has no leader
I20260811 18:57:04.592610 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9ccfd97c9c604a7b9f786709962c107c" } modify_external_catalogs: true
I20260811 18:57:04.592932 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9ccfd97c9c604a7b9f786709962c107c" } modify_external_catalogs: true
I20260811 18:57:04.589460 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65d16e5449864d18a0a01e39d25ad8dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.594378 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65d16e5449864d18a0a01e39d25ad8dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.594866 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1cacf3713764528a169a58c4f2812b6" } modify_external_catalogs: true
I20260811 18:57:04.595242 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1cacf3713764528a169a58c4f2812b6" } modify_external_catalogs: true
I20260811 18:57:04.590117 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1f27ea0d4f084ac081cca10255b06c11" } modify_external_catalogs: true
I20260811 18:57:04.596164 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1f27ea0d4f084ac081cca10255b06c11" } modify_external_catalogs: true
I20260811 18:57:04.596619 4163 tablet_bootstrap.cc:492] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.597034 4163 ts_tablet_manager.cc:1397] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.014s sys 0.006s
I20260811 18:57:04.598557 4163 raft_consensus.cc:348] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.599668 4163 raft_consensus.cc:374] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:04.599725 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 80cf2375b4c64dbfae4e20003545da60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80cf2375b4c64dbfae4e20003545da60 already in progress: creating tablet
I20260811 18:57:04.581485 4168 ts_tablet_manager.cc:1918] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.599936 4163 raft_consensus.cc:729] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.600270 4168 log.cc:1196] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/d24799ded256403e9fad5c1a22b8aa80
I20260811 18:57:04.600456 4163 consensus_queue.cc:260] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.601505 4163 ts_tablet_manager.cc:1428] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:04.601711 4168 ts_tablet_manager.cc:1939] T d24799ded256403e9fad5c1a22b8aa80 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:04.602077 4163 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.604347 4168 tablet_replica.cc:330] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.604924 4168 raft_consensus.cc:2229] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:04.605459 4168 pending_rounds.cc:62] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Trying to abort 1 pending ops.
I20260811 18:57:04.606101 4163 tablet_bootstrap.cc:654] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.606330 4168 pending_rounds.cc:68] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400059306893312 op_type: NO_OP noop_request { }
I20260811 18:57:04.607415 4168 raft_consensus.cc:2866] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:04.608369 4168 raft_consensus.cc:2258] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.610023 4168 ts_tablet_manager.cc:1905] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:04.613194 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 consensus state has no leader
W20260811 18:57:04.619527 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet b1ce1042ae974f4dbac964bd7ea16a94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1ce1042ae974f4dbac964bd7ea16a94 already in progress: creating tablet
I20260811 18:57:04.624529 4168 ts_tablet_manager.cc:1918] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:04.624818 4168 log.cc:1196] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:04.625720 4168 ts_tablet_manager.cc:1939] T 0f3414c134374aa59ac23d276a018ec9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:04.627089 4168 tablet_replica.cc:330] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.627681 4168 raft_consensus.cc:2229] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.628039 4168 raft_consensus.cc:2258] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.628962 4168 ts_tablet_manager.cc:1905] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.631237 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa (DEFAULT_TABLE table=default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.632151 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a55a42d1e64f04ad7f433614a3a1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.635133 3855 tablet_service.cc:1467] Processing CreateTablet for tablet d9421bc220774f66b90bf537a47f1e13 (DEFAULT_TABLE table=default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.635797 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9421bc220774f66b90bf537a47f1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.639118 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe (DEFAULT_TABLE table=default.Table_89b7704fd6264ef8bd2289a13f9a488d [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.640075 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732c700ff79432aaf1f0d1c1e90aabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.642583 4168 ts_tablet_manager.cc:1918] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.643087 4168 log.cc:1196] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:04.644717 4168 ts_tablet_manager.cc:1939] T 82d1339366a24a87b74bd1d411a3f6fe P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:04.646229 4168 tablet_replica.cc:330] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
W20260811 18:57:04.646981 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet cf285cdf5235451bb1531cb294017101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf285cdf5235451bb1531cb294017101 already in progress: creating tablet
I20260811 18:57:04.647149 4168 raft_consensus.cc:2229] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:04.647541 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 3085d74b0bf7447d9c247b7e8872b215 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3085d74b0bf7447d9c247b7e8872b215 already in progress: creating tablet
I20260811 18:57:04.647821 4168 raft_consensus.cc:2258] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.648943 4168 ts_tablet_manager.cc:1905] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.620559 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d816b59355a94c8698108ac81bf1ab89" } new_table_name: "default.Table_8af423b4350249d6bae77e7e3c7d9fa4" modify_external_catalogs: true
I20260811 18:57:04.628432 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8e4b85a9b7d5460b83544a3677cde81a" } modify_external_catalogs: true
I20260811 18:57:04.650468 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8e4b85a9b7d5460b83544a3677cde81a" } modify_external_catalogs: true
I20260811 18:57:04.651850 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]). Expected version 3 got 0
I20260811 18:57:04.629180 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dcfccd3951dc4875a1d70530eb514fb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.653111 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dcfccd3951dc4875a1d70530eb514fb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.660352 4168 ts_tablet_manager.cc:1918] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.660807 4168 log.cc:1196] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/634c06d76eca404a88f326e905882930
I20260811 18:57:04.665853 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 0190cbd859274fcfa09c6fab899c4843 (DEFAULT_TABLE table=default.Table_d0d75fef740443968800749bb76bfd2b [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.666306 3860 tablet_service.cc:1467] Processing CreateTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 (DEFAULT_TABLE table=default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.665910 3866 tablet_service.cc:1467] Processing CreateTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.667354 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 6381a951a966473c80670b0dd1aff062 (DEFAULT_TABLE table=default.Table_2ed6982fd3114b80993a67546347cfb3 [id=263eb849fd614011a9dd641aeb9126d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.669400 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 866ed614511649459b22a9fce55220b2 (DEFAULT_TABLE table=default.Table_59cd4e34981541759a761fe10bf57a2c [id=58d4d7c05ba44fc2b4c4a71746d340dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.669853 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25ff8a9e3a944269e9799995c73ebe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.670557 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e215759d48c24d8cb99bbc3fc9009a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.671298 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866ed614511649459b22a9fce55220b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.671831 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6381a951a966473c80670b0dd1aff062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.673416 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet d24799ded256403e9fad5c1a22b8aa80 (table default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]) successfully deleted
I20260811 18:57:04.673806 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0190cbd859274fcfa09c6fab899c4843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.674172 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 0f3414c134374aa59ac23d276a018ec9 (table default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]) successfully deleted
I20260811 18:57:04.674590 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 82d1339366a24a87b74bd1d411a3f6fe (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:04.675611 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_97097723c5264ab781135d328c9be026"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.680037 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97097723c5264ab781135d328c9be026 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.687886 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_75b82b1226924777a76c6b0f311c9fe0" } new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" modify_external_catalogs: true
I20260811 18:57:04.696084 3721 tablet_service.cc:1467] Processing CreateTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe (DEFAULT_TABLE table=default.Table_89b7704fd6264ef8bd2289a13f9a488d [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.697381 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 0190cbd859274fcfa09c6fab899c4843 (DEFAULT_TABLE table=default.Table_d0d75fef740443968800749bb76bfd2b [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.701247 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6381a951a966473c80670b0dd1aff062 (DEFAULT_TABLE table=default.Table_2ed6982fd3114b80993a67546347cfb3 [id=263eb849fd614011a9dd641aeb9126d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.702055 3720 tablet_service.cc:1467] Processing CreateTablet for tablet d9421bc220774f66b90bf537a47f1e13 (DEFAULT_TABLE table=default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.705557 4004 tablet_service.cc:1467] Processing CreateTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa (DEFAULT_TABLE table=default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.706805 4168 ts_tablet_manager.cc:1939] T 634c06d76eca404a88f326e905882930 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:04.708359 4168 tablet_replica.cc:330] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:04.710870 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5ae200f3814e431a8288c30fe94bffcf
I20260811 18:57:04.710959 4011 tablet_service.cc:1467] Processing CreateTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe (DEFAULT_TABLE table=default.Table_89b7704fd6264ef8bd2289a13f9a488d [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.711560 4168 raft_consensus.cc:2229] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:04.712013 4168 pending_rounds.cc:62] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Trying to abort 1 pending ops.
I20260811 18:57:04.712219 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet dd465168aecf43a1b08b9c11d241f898 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.713742 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 508f0b39cd654aa991a2928a4b067762
I20260811 18:57:04.714599 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet cd3b82603ec340ed9cabe9b2ff62a8b3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.718175 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 41558efe0344434db6e755804442d2de
I20260811 18:57:04.718658 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 807cee88c18340dbb23c51cad54ab56d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.718330 3548 catalog_manager.cc:5495] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67). New cstate: current_term: 1 leader_uuid: "59811bdf82234ef7ac5865dbecfcc388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:04.719476 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 9f8c0444ec2242299482736a73551f80 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.719440 3548 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]). Expected version 1 got 0
I20260811 18:57:04.719688 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_11582479665b4ae58e3bfc2103bddde9" } modify_external_catalogs: true
I20260811 18:57:04.720208 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_11582479665b4ae58e3bfc2103bddde9" } modify_external_catalogs: true
I20260811 18:57:04.721179 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.721185 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5c143dbecb50403790cf8e126be5b651" } new_table_name: "default.Table_923c4369260d436fafb1aff17a2ec8d4" modify_external_catalogs: true
I20260811 18:57:04.721796 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.723053 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 41558efe0344434db6e755804442d2de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.723569 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 6381a951a966473c80670b0dd1aff062 (DEFAULT_TABLE table=default.Table_2ed6982fd3114b80993a67546347cfb3 [id=263eb849fd614011a9dd641aeb9126d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.733340 4169 ts_tablet_manager.cc:1918] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.735517 4168 pending_rounds.cc:68] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062126866432 op_type: NO_OP noop_request { }
I20260811 18:57:04.735948 4001 tablet_service.cc:1467] Processing CreateTablet for tablet 866ed614511649459b22a9fce55220b2 (DEFAULT_TABLE table=default.Table_59cd4e34981541759a761fe10bf57a2c [id=58d4d7c05ba44fc2b4c4a71746d340dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.736553 4009 tablet_service.cc:1467] Processing CreateTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 (DEFAULT_TABLE table=default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.737097 4168 raft_consensus.cc:2866] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:04.737504 4168 raft_consensus.cc:2258] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.738294 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a55a42d1e64f04ad7f433614a3a1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.738484 4005 tablet_service.cc:1467] Processing CreateTablet for tablet d9421bc220774f66b90bf537a47f1e13 (DEFAULT_TABLE table=default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.738993 4005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9421bc220774f66b90bf537a47f1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.739547 4168 ts_tablet_manager.cc:1905] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.741760 4013 tablet_service.cc:1467] Processing CreateTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.743425 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 0190cbd859274fcfa09c6fab899c4843 (DEFAULT_TABLE table=default.Table_d0d75fef740443968800749bb76bfd2b [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.743945 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25ff8a9e3a944269e9799995c73ebe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.746228 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6381a951a966473c80670b0dd1aff062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.746531 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0190cbd859274fcfa09c6fab899c4843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.746893 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e215759d48c24d8cb99bbc3fc9009a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.747470 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866ed614511649459b22a9fce55220b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.751359 4163 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.751730 4163 ts_tablet_manager.cc:1397] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.150s user 0.008s sys 0.004s
I20260811 18:57:04.754510 4293 raft_consensus.cc:482] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.754805 4169 log.cc:1196] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8964f92255884b3aae0cac60f019fdb7
I20260811 18:57:04.759696 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732c700ff79432aaf1f0d1c1e90aabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.763439 4163 raft_consensus.cc:348] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.764362 4163 raft_consensus.cc:374] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.764057 4293 raft_consensus.cc:504] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.766393 4293 leader_election.cc:288] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:04.768137 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efb663060ef457cbece962fce701eb1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:04.770388 3824 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:04.772383 4163 raft_consensus.cc:729] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.773151 4163 consensus_queue.cc:260] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.788076 4163 ts_tablet_manager.cc:1428] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.034s user 0.004s sys 0.001s
I20260811 18:57:04.798524 4169 ts_tablet_manager.cc:1939] T 8964f92255884b3aae0cac60f019fdb7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.800710 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8964f92255884b3aae0cac60f019fdb7 (table default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]) successfully deleted
I20260811 18:57:04.802718 4169 tablet_replica.cc:330] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.803277 4169 raft_consensus.cc:2229] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.805546 4169 raft_consensus.cc:2258] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.806794 4169 ts_tablet_manager.cc:1905] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.807632 4163 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.830560 4299 raft_consensus.cc:482] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.831046 4299 raft_consensus.cc:504] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.832465 4299 leader_election.cc:288] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:04.833422 4299 raft_consensus.cc:482] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.833786 4299 raft_consensus.cc:504] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:04.837093 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d1339366a24a87b74bd1d411a3f6fe" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:04.837107 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:04.837525 3746 raft_consensus.cc:3037] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.840337 4163 tablet_bootstrap.cc:654] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.840631 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa (DEFAULT_TABLE table=default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:04.842136 3676 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:04.842900 3676 consensus_peers.cc:487] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3085d74b0bf7447d9c247b7e8872b215. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:04.843809 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 (DEFAULT_TABLE table=default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.846194 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 866ed614511649459b22a9fce55220b2 (DEFAULT_TABLE table=default.Table_59cd4e34981541759a761fe10bf57a2c [id=58d4d7c05ba44fc2b4c4a71746d340dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.846470 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:04.847383 3893 raft_consensus.cc:2379] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd8c4bdbd811417ab011a7131f17d283 in current term 1: Already voted for candidate 59811bdf82234ef7ac5865dbecfcc388 in this term.
I20260811 18:57:04.848935 4330 raft_consensus.cc:482] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.848861 3713 tablet_service.cc:1467] Processing CreateTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.849757 4299 leader_election.cc:288] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:04.850010 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a732c700ff79432aaf1f0d1c1e90aabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.857316 4293 raft_consensus.cc:482] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.857749 4293 raft_consensus.cc:504] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.859040 4293 leader_election.cc:288] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:04.849375 4330 raft_consensus.cc:504] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.864979 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a55a42d1e64f04ad7f433614a3a1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.873780 4163 tablet_bootstrap.cc:492] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.874290 4163 ts_tablet_manager.cc:1397] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.081s user 0.001s sys 0.009s
I20260811 18:57:04.876168 4163 raft_consensus.cc:348] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.876720 4163 raft_consensus.cc:374] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.877643 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:04.878000 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f25ff8a9e3a944269e9799995c73ebe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.878167 3893 raft_consensus.cc:2454] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:04.878902 4163 raft_consensus.cc:729] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.879114 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866ed614511649459b22a9fce55220b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.879527 4163 consensus_queue.cc:260] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.883258 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e215759d48c24d8cb99bbc3fc9009a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.884759 4163 ts_tablet_manager.cc:1428] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260811 18:57:04.887393 4163 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.895105 4163 tablet_bootstrap.cc:654] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.903821 4330 leader_election.cc:288] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:04.904422 3676 leader_election.cc:302] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:04.904975 4330 raft_consensus.cc:2781] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:04.895548 4169 ts_tablet_manager.cc:1918] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:04.905438 4330 raft_consensus.cc:482] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:04.905561 4169 log.cc:1196] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8efb663060ef457cbece962fce701eb1
I20260811 18:57:04.905743 4330 raft_consensus.cc:3037] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.906378 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9421bc220774f66b90bf537a47f1e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.907065 4169 ts_tablet_manager.cc:1939] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.908428 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8efb663060ef457cbece962fce701eb1 (table default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]) successfully deleted
I20260811 18:57:04.909083 4169 tablet_replica.cc:330] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:04.909619 4169 raft_consensus.cc:2229] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:04.909991 4169 raft_consensus.cc:2258] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:04.910759 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0190cbd859274fcfa09c6fab899c4843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.911111 4169 ts_tablet_manager.cc:1905] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:04.915318 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6381a951a966473c80670b0dd1aff062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.918396 3746 raft_consensus.cc:2454] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59811bdf82234ef7ac5865dbecfcc388 in term 1.
I20260811 18:57:04.925136 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:04.926259 4167 rpcz_store.cc:276] 0811 18:57:03.227253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.228966 (+ 1713us) service_pool.cc:227] Handling call
0811 18:57:04.519629 (+1290663us) ts_tablet_manager.cc:1145] Deleting tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
0811 18:57:04.519641 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.519769 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1289948}
I20260811 18:57:04.927498 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 75) took 1582 ms. Trace:
I20260811 18:57:04.927945 4167 rpcz_store.cc:276] 0811 18:57:03.344753 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.345069 (+ 316us) service_pool.cc:227] Handling call
0811 18:57:04.927091 (+1582022us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:04.927110 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.927349 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1581627}
I20260811 18:57:04.928858 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 76) took 1527 ms. Trace:
I20260811 18:57:04.939937 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:04.941768 3824 leader_election.cc:341] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:04.944168 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 already in progress: creating tablet
I20260811 18:57:04.953976 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.951900 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:04.955951 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet 41558efe0344434db6e755804442d2de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.951730 4035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:04.960126 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:04.961961 4163 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:04.962409 4163 ts_tablet_manager.cc:1397] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.075s user 0.006s sys 0.002s
I20260811 18:57:04.965163 4163 raft_consensus.cc:348] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.965801 4163 raft_consensus.cc:374] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.966182 4163 raft_consensus.cc:729] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.966809 4163 consensus_queue.cc:260] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.929677 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efb663060ef457cbece962fce701eb1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:04.945158 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.945397 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:04.945916 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 41558efe0344434db6e755804442d2de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:04.950766 3678 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:04.970840 4163 ts_tablet_manager.cc:1428] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.002s sys 0.002s
I20260811 18:57:04.956697 4330 raft_consensus.cc:504] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:04.963491 4162 tablet_bootstrap.cc:492] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:04.971480 4163 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:04.973737 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d106e0e1f4b845429b10f4e100f69a8b
W20260811 18:57:04.976910 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 08b8dcf7a56043efbbd40bb888616492 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 08b8dcf7a56043efbbd40bb888616492 already in progress: creating tablet
I20260811 18:57:04.977166 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:04.980959 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 80cf2375b4c64dbfae4e20003545da60 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:04.982762 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_219170f2f267485bb7b6551dc4e03d1a" } new_table_name: "default.Table_176c852b84c14616bc0ea0f110671639" modify_external_catalogs: true
W20260811 18:57:04.981706 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 987b9ac6859444e3a86c56af786cc84e has no consensus state
I20260811 18:57:04.986183 4163 tablet_bootstrap.cc:654] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.986322 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e394e4379aae4ac8b877d52273a6bf37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:04.987574 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e394e4379aae4ac8b877d52273a6bf37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:04.977505 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]) successfully deleted
I20260811 18:57:04.987612 4169 ts_tablet_manager.cc:1918] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:04.990073 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 60274115fe164d85badcf2e72d640f54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60274115fe164d85badcf2e72d640f54 already in progress: creating tablet
I20260811 18:57:04.990497 4169 log.cc:1196] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/dd465168aecf43a1b08b9c11d241f898
I20260811 18:57:04.993587 3745 raft_consensus.cc:2379] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
I20260811 18:57:04.993836 3678 leader_election.cc:302] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:04.996747 4169 ts_tablet_manager.cc:1939] T dd465168aecf43a1b08b9c11d241f898 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:04.996770 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:04.997337 3893 raft_consensus.cc:3037] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.999679 4167 rpcz_store.cc:276] 0811 18:57:03.401801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.401973 (+ 172us) service_pool.cc:227] Handling call
0811 18:57:04.928532 (+1526559us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:04.928548 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:04.928724 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1526107}
W20260811 18:57:05.002956 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 43748.4us
W20260811 18:57:05.003832 3678 leader_election.cc:341] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.004156 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:05.004446 4330 leader_election.cc:288] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.004999 3678 leader_election.cc:302] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:05.005577 4330 raft_consensus.cc:2726] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.005949 4162 ts_tablet_manager.cc:1397] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.466s user 0.006s sys 0.010s
I20260811 18:57:05.006197 3893 raft_consensus.cc:2454] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:05.006443 4167 tablet_replica.cc:330] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:05.006633 3824 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.007321 3824 leader_election.cc:302] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.007463 3676 leader_election.cc:302] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.007725 4333 raft_consensus.cc:2726] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:05.007854 3824 raft_consensus.cc:2670] T 8efb663060ef457cbece962fce701eb1 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:05.007980 4168 ts_tablet_manager.cc:1918] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:05.008054 4333 raft_consensus.cc:2781] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:05.008302 4168 log.cc:1196] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/927fb981490b4cecabe40314d6f71048
I20260811 18:57:05.009160 3824 leader_election.cc:302] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:04.998471 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet dd465168aecf43a1b08b9c11d241f898 (table default.Table_efd83594c1f04b33b212055bafe30b88 [id=4bf1be6d59ba4db9a5f7f0f8cdf8d571]) successfully deleted
I20260811 18:57:05.010196 4293 raft_consensus.cc:2726] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:05.010191 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 60274115fe164d85badcf2e72d640f54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60274115fe164d85badcf2e72d640f54 already in progress: creating tablet
I20260811 18:57:05.010864 4169 tablet_replica.cc:330] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.011323 4169 raft_consensus.cc:2229] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.011847 4169 raft_consensus.cc:2258] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.013114 4169 ts_tablet_manager.cc:1905] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.014370 4162 raft_consensus.cc:348] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.015077 4333 raft_consensus.cc:686] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:05.015817 4333 consensus_queue.cc:237] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.019080 4167 raft_consensus.cc:2229] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.019361 4163 tablet_bootstrap.cc:492] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.019563 4167 raft_consensus.cc:2258] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.020406 4163 ts_tablet_manager.cc:1397] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.001s
W20260811 18:57:05.021739 3678 leader_election.cc:341] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.025295 4167 ts_tablet_manager.cc:1905] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:05.026715 4165 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.071s sys 0.130s
W20260811 18:57:05.026979 4165 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.503s user 0.071s sys 0.130s
I20260811 18:57:05.028833 4165 tablet_bootstrap.cc:492] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:05.029141 4165 ts_tablet_manager.cc:1397] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.552s user 0.072s sys 0.135s
I20260811 18:57:05.030685 4162 raft_consensus.cc:374] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.031024 4162 raft_consensus.cc:729] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.031345 4163 raft_consensus.cc:348] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.032110 4163 raft_consensus.cc:374] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.032455 4163 raft_consensus.cc:729] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.033110 4163 consensus_queue.cc:260] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.040246 4162 consensus_queue.cc:260] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:05.042354 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 494c09652d9e408880ca606ae0b30773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 494c09652d9e408880ca606ae0b30773 already in progress: creating tablet
I20260811 18:57:05.043332 4163 ts_tablet_manager.cc:1428] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.023s user 0.002s sys 0.002s
I20260811 18:57:05.044340 4163 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.050529 4163 tablet_bootstrap.cc:654] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.050720 4168 ts_tablet_manager.cc:1939] T 927fb981490b4cecabe40314d6f71048 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:05.052218 4165 raft_consensus.cc:348] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.052824 4165 raft_consensus.cc:374] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.053911 4165 raft_consensus.cc:729] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.055927 4165 consensus_queue.cc:260] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.056422 4169 ts_tablet_manager.cc:1918] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:05.056936 3678 consensus_peers.cc:487] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:05.059338 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.057657 4168 tablet_replica.cc:330] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:05.060408 4169 log.cc:1196] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/634c06d76eca404a88f326e905882930
I20260811 18:57:05.061290 4168 raft_consensus.cc:2229] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:05.061538 4169 ts_tablet_manager.cc:1939] T 634c06d76eca404a88f326e905882930 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.061496 4165 ts_tablet_manager.cc:1428] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.030s user 0.004s sys 0.001s
I20260811 18:57:05.065016 4165 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:05.066105 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]) successfully deleted
I20260811 18:57:05.066850 4162 ts_tablet_manager.cc:1428] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.061s user 0.000s sys 0.004s
I20260811 18:57:05.067471 4168 raft_consensus.cc:2258] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:05.067833 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2852f9e5f67c4025836018af553388a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2852f9e5f67c4025836018af553388a5 already in progress: creating tablet
I20260811 18:57:05.071777 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.072041 4168 ts_tablet_manager.cc:1905] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.072722 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 927fb981490b4cecabe40314d6f71048 (table default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]) successfully deleted
W20260811 18:57:05.073432 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 2852f9e5f67c4025836018af553388a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2852f9e5f67c4025836018af553388a5 already in progress: creating tablet
W20260811 18:57:05.073907 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 5ae200f3814e431a8288c30fe94bffcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ae200f3814e431a8288c30fe94bffcf already in progress: creating tablet
I20260811 18:57:05.077283 4165 tablet_bootstrap.cc:654] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.077706 4162 tablet_bootstrap.cc:492] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.079986 4293 raft_consensus.cc:482] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.083680 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:05.084205 4293 raft_consensus.cc:504] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:05.084654 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d3d6daff4b7540309ff58755758e9752 consensus state has no leader
I20260811 18:57:05.085281 4293 leader_election.cc:288] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.085296 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c281ac8dd4a444c08084edb70426f847"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.086799 3824 leader_election.cc:341] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: d24799ded256403e9fad5c1a22b8aa80
W20260811 18:57:05.087371 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c281ac8dd4a444c08084edb70426f847 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.087715 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 36724a83bf9844b9a27a88ca36a9bf50) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:05.088694 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5ae200f3814e431a8288c30fe94bffcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ae200f3814e431a8288c30fe94bffcf already in progress: creating tablet
I20260811 18:57:05.090104 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9f8c0444ec2242299482736a73551f80
I20260811 18:57:05.090271 4163 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.090607 4163 ts_tablet_manager.cc:1397] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.004s
I20260811 18:57:05.092077 4163 raft_consensus.cc:348] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.092486 4163 raft_consensus.cc:374] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.092747 4163 raft_consensus.cc:729] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.093227 4163 consensus_queue.cc:260] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:05.097865 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 508f0b39cd654aa991a2928a4b067762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 508f0b39cd654aa991a2928a4b067762 already in progress: creating tablet
I20260811 18:57:05.098067 4163 ts_tablet_manager.cc:1428] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 18:57:05.098495 4163 tablet_bootstrap.cc:492] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.098592 3550 catalog_manager.cc:6565] ReplaceTablet: tablet 9f8c0444ec2242299482736a73551f80 deleted and replaced by tablet e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:05.099030 3552 catalog_manager.cc:6565] ReplaceTablet: tablet cd3b82603ec340ed9cabe9b2ff62a8b3 deleted and replaced by tablet 36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:05.099864 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.100814 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d22535c5f02a4044aad07546138fca10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.101516 4169 tablet_replica.cc:330] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.101890 4169 raft_consensus.cc:2229] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:05.101948 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d22535c5f02a4044aad07546138fca10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.102191 4169 raft_consensus.cc:2258] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.104112 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef02aae7a42c46678973fed0d27e7421" } new_table_name: "default.Table_095898ad95614d4d95ef7babb8104578" modify_external_catalogs: true
I20260811 18:57:05.104564 4169 ts_tablet_manager.cc:1905] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:05.113626 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 01d564ad80064912a8ce8e31c1b6c4d2 has no consensus state
I20260811 18:57:05.114416 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6460e311a6da4deaba4efce4c6dbd883"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.114890 3824 leader_election.cc:341] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d24799ded256403e9fad5c1a22b8aa80
W20260811 18:57:05.115304 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6460e311a6da4deaba4efce4c6dbd883 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.115295 3824 leader_election.cc:302] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:05.116199 4230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:05.116698 4230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260811 18:57:05.116775 4293 raft_consensus.cc:2726] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:05.116984 4230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:57:05.120288 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_815d74faaf044b2f9000790283e989c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.120525 4293 raft_consensus.cc:482] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.120999 4293 raft_consensus.cc:504] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:05.121167 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_815d74faaf044b2f9000790283e989c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.122097 4293 leader_election.cc:288] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:05.123687 3824 leader_election.cc:341] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:05.127713 4163 tablet_bootstrap.cc:654] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.131260 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 36724a83bf9844b9a27a88ca36a9bf50) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.132649 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.134837 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet cd3b82603ec340ed9cabe9b2ff62a8b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 36724a83bf9844b9a27a88ca36a9bf50) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.135403 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.137293 4163 tablet_bootstrap.cc:492] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.137712 4163 ts_tablet_manager.cc:1397] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.003s
W20260811 18:57:05.139029 3824 leader_election.cc:341] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:05.139014 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d8171ab03efd442a9bf4e1176f04cb24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.139271 3824 leader_election.cc:302] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.139436 4163 raft_consensus.cc:348] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:05.139847 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8171ab03efd442a9bf4e1176f04cb24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.139882 4163 raft_consensus.cc:374] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.140112 4293 raft_consensus.cc:2726] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.146482 4165 tablet_bootstrap.cc:492] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:05.146886 4165 ts_tablet_manager.cc:1397] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.083s user 0.003s sys 0.004s
I20260811 18:57:05.148490 4165 raft_consensus.cc:348] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.149225 4165 raft_consensus.cc:374] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.149403 4165 raft_consensus.cc:729] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.149883 4165 consensus_queue.cc:260] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.153263 4169 ts_tablet_manager.cc:1918] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:05.154376 4167 ts_tablet_manager.cc:1918] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.154959 4167 log.cc:1196] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/927fb981490b4cecabe40314d6f71048
I20260811 18:57:05.156111 4167 ts_tablet_manager.cc:1939] T 927fb981490b4cecabe40314d6f71048 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.158821 4165 ts_tablet_manager.cc:1428] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.012s user 0.001s sys 0.003s
I20260811 18:57:05.159476 4165 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:05.163446 4165 tablet_bootstrap.cc:654] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.155292 4162 tablet_bootstrap.cc:654] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.140112 4163 raft_consensus.cc:729] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.164822 4163 consensus_queue.cc:260] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.168629 4163 ts_tablet_manager.cc:1428] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.031s user 0.004s sys 0.000s
I20260811 18:57:05.169294 4163 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:05.169341 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 has no consensus state
I20260811 18:57:05.170749 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 77) took 1688 ms. Trace:
I20260811 18:57:05.170966 4167 rpcz_store.cc:276] 0811 18:57:03.482570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.482651 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:05.000173 (+1517522us) ts_tablet_manager.cc:1145] Deleting tablet 927fb981490b4cecabe40314d6f71048
0811 18:57:05.000186 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.120634 (+120448us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.154025 (+ 33391us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.170533 (+ 16508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1517058}
I20260811 18:57:05.171880 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 927fb981490b4cecabe40314d6f71048 (table default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]) successfully deleted
I20260811 18:57:05.172700 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 6d1d58835ad24749a00ce27b4bc73c59 (table default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]). Expected version 1 got 0
I20260811 18:57:05.174775 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_75b82b1226924777a76c6b0f311c9fe0 [id=7ef504f80694466a804cb34703c396fe]). Expected version 1 got 0
I20260811 18:57:05.174829 4169 log.cc:1196] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/41558efe0344434db6e755804442d2de
I20260811 18:57:05.175832 4169 ts_tablet_manager.cc:1939] T 41558efe0344434db6e755804442d2de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.179288 4167 tablet_replica.cc:330] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.179662 4167 raft_consensus.cc:2229] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.179735 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet d24799ded256403e9fad5c1a22b8aa80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.180044 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3414c134374aa59ac23d276a018ec9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.181543 4169 tablet_replica.cc:330] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.183660 4167 raft_consensus.cc:2258] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.184499 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 331396c9962f48ed9ea6dc66f1660902 (DEFAULT_TABLE table=default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.184994 4167 ts_tablet_manager.cc:1905] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.185108 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331396c9962f48ed9ea6dc66f1660902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.186811 4169 raft_consensus.cc:2229] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.193995 4333 raft_consensus.cc:482] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.194321 4333 raft_consensus.cc:504] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.196632 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 41558efe0344434db6e755804442d2de (table default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]) successfully deleted
I20260811 18:57:05.188408 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 331396c9962f48ed9ea6dc66f1660902 (DEFAULT_TABLE table=default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.199033 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331396c9962f48ed9ea6dc66f1660902. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.200148 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d106e0e1f4b845429b10f4e100f69a8b already in progress: creating tablet
I20260811 18:57:05.201815 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 405773849fb94800be61ad8a7a6055a4 (DEFAULT_TABLE table=default.Table_095898ad95614d4d95ef7babb8104578 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.201870 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (DEFAULT_TABLE table=default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.202453 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405773849fb94800be61ad8a7a6055a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.203284 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d564ad80064912a8ce8e31c1b6c4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.206813 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 55c1a7d44717415ca76a2d0b7e94394e (DEFAULT_TABLE table=default.Table_457be7b012ba40deab01ec6480b82ede [id=1f86b4e3748c4cb7b34b2a24524dd1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.207548 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c1a7d44717415ca76a2d0b7e94394e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.209321 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad616256b9949ceaf8b300f660d00c5 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.209854 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad616256b9949ceaf8b300f660d00c5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.210724 4230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260811 18:57:05.211036 4162 tablet_bootstrap.cc:492] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.211396 4162 ts_tablet_manager.cc:1397] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.144s user 0.005s sys 0.005s
I20260811 18:57:05.212368 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 (DEFAULT_TABLE table=default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.212833 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d995d72f8142afaeb57a2c29910fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.212678 4333 leader_election.cc:288] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:05.213265 4230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:05.213479 4230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:05.213703 4230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:05.215144 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:05.215502 3893 raft_consensus.cc:2454] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:05.215926 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (DEFAULT_TABLE table=default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.216717 3863 tablet_service.cc:1467] Processing CreateTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 (DEFAULT_TABLE table=default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.216972 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d564ad80064912a8ce8e31c1b6c4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.217444 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7259cf84f5c4c89a1b4510f743c2bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.217849 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 405773849fb94800be61ad8a7a6055a4 (DEFAULT_TABLE table=default.Table_095898ad95614d4d95ef7babb8104578 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.217861 4162 raft_consensus.cc:348] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.219233 4162 raft_consensus.cc:374] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.221603 3676 leader_election.cc:302] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.221751 4162 raft_consensus.cc:729] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.222177 4333 raft_consensus.cc:2781] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:05.222204 4162 consensus_queue.cc:260] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.222837 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405773849fb94800be61ad8a7a6055a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.224061 4333 raft_consensus.cc:482] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:05.224406 4333 raft_consensus.cc:3037] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.225675 4162 ts_tablet_manager.cc:1428] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.004s
I20260811 18:57:05.226239 4162 tablet_bootstrap.cc:492] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.227142 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 55c1a7d44717415ca76a2d0b7e94394e (DEFAULT_TABLE table=default.Table_457be7b012ba40deab01ec6480b82ede [id=1f86b4e3748c4cb7b34b2a24524dd1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.231478 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 (DEFAULT_TABLE table=default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.230085 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.233183 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 (DEFAULT_TABLE table=default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.237412 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad616256b9949ceaf8b300f660d00c5 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.239265 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c1a7d44717415ca76a2d0b7e94394e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.242081 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d995d72f8142afaeb57a2c29910fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.244635 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f8eb041f3d704762868a048faec0e0fc (DEFAULT_TABLE table=default.Table_79396737f1a343c98184686b7eda44ab [id=32914b6ff485486f843da54878e2d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.246958 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7259cf84f5c4c89a1b4510f743c2bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.247179 3859 tablet_service.cc:1467] Processing CreateTablet for tablet f8eb041f3d704762868a048faec0e0fc (DEFAULT_TABLE table=default.Table_79396737f1a343c98184686b7eda44ab [id=32914b6ff485486f843da54878e2d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.249195 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 987b9ac6859444e3a86c56af786cc84e (DEFAULT_TABLE table=default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.249869 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad616256b9949ceaf8b300f660d00c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.252720 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8eb041f3d704762868a048faec0e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.255524 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8eb041f3d704762868a048faec0e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.186303 4163 tablet_bootstrap.cc:654] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.262797 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987b9ac6859444e3a86c56af786cc84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.264533 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 987b9ac6859444e3a86c56af786cc84e (DEFAULT_TABLE table=default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.266975 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987b9ac6859444e3a86c56af786cc84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.267917 4169 raft_consensus.cc:2258] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.269274 4293 raft_consensus.cc:482] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.271042 4271 raft_consensus.cc:482] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.271342 4271 raft_consensus.cc:504] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.270787 4293 raft_consensus.cc:504] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.274644 4293 leader_election.cc:288] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.276365 4013 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.276866 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (DEFAULT_TABLE table=default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.279695 4035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.271688 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.280409 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 634c06d76eca404a88f326e905882930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.281522 4169 ts_tablet_manager.cc:1905] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.281790 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:05.282200 3824 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.282546 4036 raft_consensus.cc:2454] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:05.282845 3824 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.283401 3824 leader_election.cc:302] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.286743 4293 raft_consensus.cc:2726] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.288422 4168 ts_tablet_manager.cc:1918] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:05.290656 4168 log.cc:1196] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/41558efe0344434db6e755804442d2de
I20260811 18:57:05.292520 4168 ts_tablet_manager.cc:1939] T 41558efe0344434db6e755804442d2de P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:05.294013 4333 raft_consensus.cc:504] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.294756 4001 tablet_service.cc:1467] Processing CreateTablet for tablet 331396c9962f48ed9ea6dc66f1660902 (DEFAULT_TABLE table=default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.297144 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d564ad80064912a8ce8e31c1b6c4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.298767 4011 tablet_service.cc:1467] Processing CreateTablet for tablet 405773849fb94800be61ad8a7a6055a4 (DEFAULT_TABLE table=default.Table_095898ad95614d4d95ef7babb8104578 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.301590 4271 leader_election.cc:288] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:05.302309 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad616256b9949ceaf8b300f660d00c5 (DEFAULT_TABLE table=default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.304633 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 55c1a7d44717415ca76a2d0b7e94394e (DEFAULT_TABLE table=default.Table_457be7b012ba40deab01ec6480b82ede [id=1f86b4e3748c4cb7b34b2a24524dd1ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.305519 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331396c9962f48ed9ea6dc66f1660902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.307457 4010 tablet_service.cc:1467] Processing CreateTablet for tablet f8eb041f3d704762868a048faec0e0fc (DEFAULT_TABLE table=default.Table_79396737f1a343c98184686b7eda44ab [id=32914b6ff485486f843da54878e2d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.308367 4003 tablet_service.cc:1467] Processing CreateTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 (DEFAULT_TABLE table=default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.308876 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 (DEFAULT_TABLE table=default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.310617 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad616256b9949ceaf8b300f660d00c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.315541 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:05.316810 4036 raft_consensus.cc:3037] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.316900 4293 raft_consensus.cc:482] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.311137 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:05.321067 4162 tablet_bootstrap.cc:654] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.322614 4293 raft_consensus.cc:504] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.322970 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8eb041f3d704762868a048faec0e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.323415 4293 leader_election.cc:288] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.323565 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:05.324025 4333 leader_election.cc:288] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.324522 3891 raft_consensus.cc:3037] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.333102 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 987b9ac6859444e3a86c56af786cc84e (DEFAULT_TABLE table=default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.338591 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7259cf84f5c4c89a1b4510f743c2bc5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.338712 3824 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:05.340931 3891 raft_consensus.cc:2454] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:05.342249 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987b9ac6859444e3a86c56af786cc84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.344543 4168 tablet_replica.cc:330] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
W20260811 18:57:05.345907 3965 leader_election.cc:341] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: Tablet not RUNNING: STOPPED
I20260811 18:57:05.348245 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c1a7d44717415ca76a2d0b7e94394e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.349990 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d995d72f8142afaeb57a2c29910fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.357131 4163 tablet_bootstrap.cc:492] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.357409 4163 ts_tablet_manager.cc:1397] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.188s user 0.015s sys 0.014s
I20260811 18:57:05.357558 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405773849fb94800be61ad8a7a6055a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.362790 4169 ts_tablet_manager.cc:1918] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.363564 4169 log.cc:1196] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:05.363047 4163 raft_consensus.cc:348] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.365151 4163 raft_consensus.cc:374] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.366595 4163 raft_consensus.cc:729] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.367003 4163 consensus_queue.cc:260] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.369325 4169 ts_tablet_manager.cc:1939] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.372550 4163 ts_tablet_manager.cc:1428] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260811 18:57:05.372893 4230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=46243
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:05.375430 4230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:05.375988 4163 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.378391 4169 tablet_replica.cc:330] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.379650 4230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:57:05.380076 4169 raft_consensus.cc:2229] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.380463 4169 raft_consensus.cc:2258] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.381459 4169 ts_tablet_manager.cc:1905] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.383210 4036 raft_consensus.cc:2454] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:05.383569 4163 tablet_bootstrap.cc:654] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.396265 3676 leader_election.cc:302] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.396783 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd3b82603ec340ed9cabe9b2ff62a8b3" candidate_uuid: "59811bdf82234ef7ac5865dbecfcc388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.398073 4167 ts_tablet_manager.cc:1918] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.398880 3967 leader_election.cc:302] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.399000 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d1339366a24a87b74bd1d411a3f6fe" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.399354 3746 raft_consensus.cc:2454] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:05.400669 3824 leader_election.cc:302] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:05.401454 4293 raft_consensus.cc:2781] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:05.401686 4293 raft_consensus.cc:482] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:05.401974 4293 raft_consensus.cc:3037] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.404544 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 41558efe0344434db6e755804442d2de (table default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]) successfully deleted
I20260811 18:57:05.404680 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_593bb155f3de4ed9a2913e3114d49e5b" } modify_external_catalogs: true
I20260811 18:57:05.407955 4293 raft_consensus.cc:504] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.409958 4293 leader_election.cc:288] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.411374 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:05.414888 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 508f0b39cd654aa991a2928a4b067762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 508f0b39cd654aa991a2928a4b067762 already in progress: creating tablet
I20260811 18:57:05.414958 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_593bb155f3de4ed9a2913e3114d49e5b" } modify_external_catalogs: true
I20260811 18:57:05.418211 4163 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.418722 4163 ts_tablet_manager.cc:1397] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.008s
W20260811 18:57:05.419157 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d106e0e1f4b845429b10f4e100f69a8b already in progress: creating tablet
I20260811 18:57:05.420503 4163 raft_consensus.cc:348] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.421016 4163 raft_consensus.cc:374] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.421231 4163 raft_consensus.cc:729] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.421663 4163 consensus_queue.cc:260] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.422549 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_153f40da23a34fd4a2cba7e46cb4f96b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.423183 4163 ts_tablet_manager.cc:1428] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
W20260811 18:57:05.423273 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153f40da23a34fd4a2cba7e46cb4f96b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.423751 4163 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.427065 4169 ts_tablet_manager.cc:1918] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.427327 4169 log.cc:1196] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d24799ded256403e9fad5c1a22b8aa80
I20260811 18:57:05.427946 4169 ts_tablet_manager.cc:1939] T d24799ded256403e9fad5c1a22b8aa80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.429018 4163 tablet_bootstrap.cc:654] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:05.431969 4352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:05.431993 4169 tablet_replica.cc:330] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.433279 4169 raft_consensus.cc:2229] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.433555 4169 raft_consensus.cc:2258] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.434330 4169 ts_tablet_manager.cc:1905] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:05.435870 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 55206.9us
I20260811 18:57:05.436959 4230 server_base.cc:1034] running on GCE node
W20260811 18:57:05.438019 4354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:05.437906 4167 log.cc:1196] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/abb530181d7b4d2baf5391808246c74b
I20260811 18:57:05.438282 4333 raft_consensus.cc:482] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.438402 4330 raft_consensus.cc:2781] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:05.443683 4169 ts_tablet_manager.cc:1918] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.444012 4169 log.cc:1196] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0f3414c134374aa59ac23d276a018ec9
I20260811 18:57:05.444576 4169 ts_tablet_manager.cc:1939] T 0f3414c134374aa59ac23d276a018ec9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.444665 4163 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.444866 4163 ts_tablet_manager.cc:1397] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.003s
I20260811 18:57:05.445369 4330 raft_consensus.cc:686] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:05.445779 4330 consensus_queue.cc:237] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.445940 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82d1339366a24a87b74bd1d411a3f6fe" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:05.446187 3744 raft_consensus.cc:3037] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.447544 4163 raft_consensus.cc:348] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.447847 4163 raft_consensus.cc:374] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.447999 4163 raft_consensus.cc:729] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.448913 4163 consensus_queue.cc:260] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.448146 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db39f5cc5ace43a49931b2da095c6037"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.452029 4163 ts_tablet_manager.cc:1428] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260811 18:57:05.452253 3722 tablet_service.cc:1467] Processing CreateTablet for tablet fb51e419dd24459caf3ba3adfbbd4517 (DEFAULT_TABLE table=default.Table_9fe9862d3266414295d433fd5b136638 [id=0fee6d656a1b486cb1fb77abf4042dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.452786 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb51e419dd24459caf3ba3adfbbd4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.454162 4168 raft_consensus.cc:2229] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.455163 4163 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.455744 4000 tablet_service.cc:1467] Processing CreateTablet for tablet fb51e419dd24459caf3ba3adfbbd4517 (DEFAULT_TABLE table=default.Table_9fe9862d3266414295d433fd5b136638 [id=0fee6d656a1b486cb1fb77abf4042dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.456291 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb51e419dd24459caf3ba3adfbbd4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.452667 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ace95690ebf4f619a57d3d7bd47bec5" } modify_external_catalogs: true
I20260811 18:57:05.446578 4333 raft_consensus.cc:504] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.458794 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b092ff08e7a340bfb2b5ad86895ca81d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.459900 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b092ff08e7a340bfb2b5ad86895ca81d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.459944 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c09652d9e408880ca606ae0b30773" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:05.460280 3891 raft_consensus.cc:2454] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:05.462796 3744 raft_consensus.cc:2454] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:05.463949 3824 leader_election.cc:302] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.464414 4293 raft_consensus.cc:2781] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:05.464922 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 6ef0a376939d4d57b0d95db644f25dd9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:05.466034 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd619a3970764d0fb914fcc6f59ea7a5" } modify_external_catalogs: true
I20260811 18:57:05.466295 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd619a3970764d0fb914fcc6f59ea7a5" } modify_external_catalogs: true
I20260811 18:57:05.466593 4167 ts_tablet_manager.cc:1939] T abb530181d7b4d2baf5391808246c74b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:05.468287 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db39f5cc5ace43a49931b2da095c6037 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.469524 4163 tablet_bootstrap.cc:654] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:05.469991 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9f8c0444ec2242299482736a73551f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f8c0444ec2242299482736a73551f80 already in progress: creating tablet
I20260811 18:57:05.470553 3553 catalog_manager.cc:5587] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027) does not have the latest schema for tablet 9c18120da2d245f495c52a670047760d (table default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137]). Expected version 1 got 0
I20260811 18:57:05.471818 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet d9421bc220774f66b90bf537a47f1e13 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:05.473349 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ace95690ebf4f619a57d3d7bd47bec5" } modify_external_catalogs: true
I20260811 18:57:05.475651 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9c18120da2d245f495c52a670047760d (table default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137]). Expected version 1 got 0
W20260811 18:57:05.477110 3539 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 58551.5us
I20260811 18:57:05.477775 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d24799ded256403e9fad5c1a22b8aa80 (table default.Table_318043c37396470886587403377a976f [id=3f698f2e9594498ba8e1146deb471cf3]) successfully deleted
I20260811 18:57:05.478037 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0f3414c134374aa59ac23d276a018ec9 (table default.Table_4579e89d4503429badc8868ce44d947a [id=44ef867690364cf5a40525f13dbf0608]) successfully deleted
I20260811 18:57:05.478645 3871 tablet_service.cc:1467] Processing CreateTablet for tablet fb51e419dd24459caf3ba3adfbbd4517 (DEFAULT_TABLE table=default.Table_9fe9862d3266414295d433fd5b136638 [id=0fee6d656a1b486cb1fb77abf4042dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.479410 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb51e419dd24459caf3ba3adfbbd4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.480480 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.480906 4169 tablet_replica.cc:330] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.481267 4169 raft_consensus.cc:2229] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.481547 4169 raft_consensus.cc:2258] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.483918 4169 ts_tablet_manager.cc:1905] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.484146 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e159333f7174424eaf413e0a07c149e9" } modify_external_catalogs: true
I20260811 18:57:05.484592 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e159333f7174424eaf413e0a07c149e9" } modify_external_catalogs: true
I20260811 18:57:05.486904 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 82d1339366a24a87b74bd1d411a3f6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 7f640c60e81b4a369458b1497a269c9c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.487893 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c09652d9e408880ca606ae0b30773" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.488392 4271 raft_consensus.cc:2684] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260811 18:57:05.493858 4333 leader_election.cc:288] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.499799 4168 raft_consensus.cc:2258] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.501737 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 8964f92255884b3aae0cac60f019fdb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.494208 4331 raft_consensus.cc:482] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.506258 4331 raft_consensus.cc:504] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.507741 4331 leader_election.cc:288] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.510965 4168 ts_tablet_manager.cc:1905] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.500919 3676 leader_election.cc:302] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.468087 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 84) took 1935 ms. Trace:
W20260811 18:57:05.519479 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 54243.7us
W20260811 18:57:05.520136 3678 leader_election.cc:341] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.522776 4167 rpcz_store.cc:276] 0811 18:57:03.532585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.535531 (+ 2946us) service_pool.cc:227] Handling call
0811 18:57:05.178445 (+1642914us) ts_tablet_manager.cc:1145] Deleting tablet abb530181d7b4d2baf5391808246c74b
0811 18:57:05.178471 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.242864 (+ 64393us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.323900 (+ 81036us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.467998 (+144098us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1642503}
I20260811 18:57:05.523737 4167 tablet_replica.cc:330] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.524531 4167 raft_consensus.cc:2229] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.525236 4167 raft_consensus.cc:2258] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.526499 4167 ts_tablet_manager.cc:1905] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.526633 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8964f92255884b3aae0cac60f019fdb7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.526996 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet abb530181d7b4d2baf5391808246c74b (table default.Table_29c83b98881f4edaba5330f38e7a67ed [id=f89b3f14c6e74c52a901945c5bfd97a1]) successfully deleted
W20260811 18:57:05.527630 3678 leader_election.cc:341] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.532435 4331 raft_consensus.cc:2781] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:05.532738 4331 raft_consensus.cc:482] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:05.532984 4331 raft_consensus.cc:3037] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:05.535781 4351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:05.537510 4230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:05.546607 4293 raft_consensus.cc:686] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:05.547282 4293 consensus_queue.cc:237] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.556246 4230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:05.559785 4230 hybrid_clock.cc:648] HybridClock initialized: now 1786474625559628 us; error 53 us; skew 500 ppm
I20260811 18:57:05.559839 4331 raft_consensus.cc:504] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.560433 4230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:05.560721 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.562762 4331 leader_election.cc:288] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:05.563428 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 331396c9962f48ed9ea6dc66f1660902
I20260811 18:57:05.568341 3546 catalog_manager.cc:5495] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:05.570230 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9c18120da2d245f495c52a670047760d (table default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137]). Expected version 1 got 0
W20260811 18:57:05.573531 3676 leader_election.cc:341] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8964f92255884b3aae0cac60f019fdb7
I20260811 18:57:05.573789 3676 leader_election.cc:302] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:05.573789 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c09652d9e408880ca606ae0b30773" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:05.574186 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.574772 4331 raft_consensus.cc:2726] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:05.574779 3824 leader_election.cc:341] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:05.575824 3892 raft_consensus.cc:3037] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:05.578487 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fb981490b4cecabe40314d6f71048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.578863 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.587255 3892 raft_consensus.cc:2454] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:05.587895 3891 raft_consensus.cc:1261] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer 59811bdf82234ef7ac5865dbecfcc388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:05.588840 3676 leader_election.cc:302] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:05.589735 4331 raft_consensus.cc:2781] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:05.590062 4331 raft_consensus.cc:686] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:05.590729 4162 tablet_bootstrap.cc:492] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.590986 4162 ts_tablet_manager.cc:1397] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.365s user 0.007s sys 0.007s
I20260811 18:57:05.591374 4331 consensus_queue.cc:237] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:05.592124 3967 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 125143us
I20260811 18:57:05.592201 4162 raft_consensus.cc:348] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.592597 4162 raft_consensus.cc:374] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.592711 4162 raft_consensus.cc:729] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.593150 4162 consensus_queue.cc:260] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.592532 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c09652d9e408880ca606ae0b30773" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:05.592897 4271 consensus_queue.cc:1035] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260811 18:57:05.594208 4169 ts_tablet_manager.cc:1918] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.594583 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:05.595222 3678 leader_election.cc:341] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.593190 4168 ts_tablet_manager.cc:1918] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.596097 4168 log.cc:1196] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:05.596804 4168 ts_tablet_manager.cc:1939] T cd3b82603ec340ed9cabe9b2ff62a8b3 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:05.596794 4162 ts_tablet_manager.cc:1428] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260811 18:57:05.603006 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.607765 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_66613580530e4ae8989a795dd5d5ef69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.608515 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66613580530e4ae8989a795dd5d5ef69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.609225 4167 ts_tablet_manager.cc:1918] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.609477 4167 log.cc:1196] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/34aeab84310d49c79805ea898b4f87ce
I20260811 18:57:05.610003 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e7259cf84f5c4c89a1b4510f743c2bc5
I20260811 18:57:05.610419 4162 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.610412 4271 consensus_queue.cc:1035] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:05.611088 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_095898ad95614d4d95ef7babb8104578" } new_table_name: "default.Table_365efecf4f8e4f4993ba5579614a3049" modify_external_catalogs: true
I20260811 18:57:05.613135 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 2852f9e5f67c4025836018af553388a5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:05.614009 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ed6982fd3114b80993a67546347cfb3" } new_table_name: "default.Table_86432f719d4b48c7ae9f1e0fd8095fcc" modify_external_catalogs: true
I20260811 18:57:05.614784 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 33979de5b48c4674a46c8da3b7151753
I20260811 18:57:05.615643 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1fbb53e0b2d44010bbb9da4dc739a78d" } new_table_name: "default.Table_bcaa2e21fe064cbea9a28f034c63eae6" modify_external_catalogs: true
I20260811 18:57:05.617674 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:05.618230 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 6ef0a376939d4d57b0d95db644f25dd9 deleted and replaced by tablet f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:05.618851 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c6dd21c795924a47a8a2435b5de1afb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.619707 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6dd21c795924a47a8a2435b5de1afb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.620435 4230 webserver.cc:460] Webserver started at http://127.3.72.125:46243/ using document root <none> and password file <none>
I20260811 18:57:05.621668 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d9421bc220774f66b90bf537a47f1e13
I20260811 18:57:05.621829 4230 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:05.622126 4230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:05.622143 3553 catalog_manager.cc:6565] ReplaceTablet: tablet d9421bc220774f66b90bf537a47f1e13 deleted and replaced by tablet d19675f757674450aceb5942e1a51c99
I20260811 18:57:05.622622 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 33d995d72f8142afaeb57a2c29910fa3
I20260811 18:57:05.623221 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 6d1d58835ad24749a00ce27b4bc73c59 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:05.624135 4162 tablet_bootstrap.cc:654] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.624863 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a30bb384136f4a169a5751655437d5a3
I20260811 18:57:05.625572 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 82d1339366a24a87b74bd1d411a3f6fe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:05.626137 4167 ts_tablet_manager.cc:1939] T 34aeab84310d49c79805ea898b4f87ce P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:05.629235 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_095898ad95614d4d95ef7babb8104578 [id=22478156dcb4450d9c8b30abaed1bc9a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 consensus state has no leader
I20260811 18:57:05.629590 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 83) took 2097 ms. Trace:
I20260811 18:57:05.629750 4167 rpcz_store.cc:276] 0811 18:57:03.532439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.542417 (+ 9978us) service_pool.cc:227] Handling call
0811 18:57:05.523513 (+1981096us) ts_tablet_manager.cc:1145] Deleting tablet 34aeab84310d49c79805ea898b4f87ce
0811 18:57:05.523534 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.586602 (+ 63068us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.609062 (+ 22460us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.629484 (+ 20422us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1978090}
I20260811 18:57:05.633502 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d0d75fef740443968800749bb76bfd2b" } new_table_name: "default.Table_be1e5fa6b9304fd6bcb4304038be24f3" modify_external_catalogs: true
I20260811 18:57:05.634764 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3441d374bdb04681b2bd464f6ed910e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.635533 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3441d374bdb04681b2bd464f6ed910e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.637384 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9bb5acd56644427f9aec05c1ab8e46ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.638167 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bb5acd56644427f9aec05c1ab8e46ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.640508 4230 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.000s
I20260811 18:57:05.643563 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 88) took 1920 ms. Trace:
I20260811 18:57:05.643882 4167 rpcz_store.cc:276] 0811 18:57:03.722057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.728010 (+ 5953us) service_pool.cc:227] Handling call
0811 18:57:05.634682 (+1906672us) ts_tablet_manager.cc:1145] Deleting tablet 807cee88c18340dbb23c51cad54ab56d
0811 18:57:05.634698 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.642824 (+ 8126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1897888}
I20260811 18:57:05.646819 4162 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.647053 4162 ts_tablet_manager.cc:1397] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.002s
I20260811 18:57:05.648269 4162 raft_consensus.cc:348] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.648635 4162 raft_consensus.cc:374] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.648816 4162 raft_consensus.cc:729] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.649212 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 99) took 1560 ms. Trace:
I20260811 18:57:05.649394 4167 rpcz_store.cc:276] 0811 18:57:04.084048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.084146 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:05.644312 (+1560166us) ts_tablet_manager.cc:1145] Deleting tablet cf285cdf5235451bb1531cb294017101
0811 18:57:05.644335 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.644681 (+ 346us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1559877}
I20260811 18:57:05.650629 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 96) took 1566 ms. Trace:
I20260811 18:57:05.650784 4167 rpcz_store.cc:276] 0811 18:57:04.083580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.084877 (+ 1297us) service_pool.cc:227] Handling call
0811 18:57:05.650159 (+1565282us) ts_tablet_manager.cc:1145] Deleting tablet 80cf2375b4c64dbfae4e20003545da60
0811 18:57:05.650178 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.650471 (+ 293us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1517055}
I20260811 18:57:05.651386 4162 consensus_queue.cc:260] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.653479 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 97) took 1569 ms. Trace:
I20260811 18:57:05.653693 4167 rpcz_store.cc:276] 0811 18:57:04.083731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.085022 (+ 1291us) service_pool.cc:227] Handling call
0811 18:57:05.653145 (+1568123us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:05.653162 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.653362 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1513883}
I20260811 18:57:05.654393 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.654831 4162 ts_tablet_manager.cc:1428] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:05.655985 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 34aeab84310d49c79805ea898b4f87ce (table default.Table_738a85e77f9e49efa9899428ec31a9ab [id=7cda514ae5f24bf785720ec89d44b923]) successfully deleted
I20260811 18:57:05.656378 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:05.656415 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 807cee88c18340dbb23c51cad54ab56d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 807cee88c18340dbb23c51cad54ab56d already in progress: creating tablet
W20260811 18:57:05.656745 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cf285cdf5235451bb1531cb294017101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf285cdf5235451bb1531cb294017101 already in progress: creating tablet
I20260811 18:57:05.657034 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:05.656999 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 80cf2375b4c64dbfae4e20003545da60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80cf2375b4c64dbfae4e20003545da60 already in progress: creating tablet
I20260811 18:57:05.657320 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 with delete_type TABLET_DATA_DELETED (replaced by tablet f61d3822fc2d4b569f223238a6464e0a) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:05.657547 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b1ce1042ae974f4dbac964bd7ea16a94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1ce1042ae974f4dbac964bd7ea16a94 already in progress: creating tablet
I20260811 18:57:05.657692 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.657984 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.658190 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.658537 4377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:05.658653 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.659013 4011 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:05.659090 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 57486.9us
I20260811 18:57:05.659291 4003 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.659827 4167 tablet_replica.cc:330] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.660143 4167 raft_consensus.cc:2229] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:05.660682 4162 tablet_bootstrap.cc:492] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.661198 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 with delete_type TABLET_DATA_DELETED (replaced by tablet f61d3822fc2d4b569f223238a6464e0a) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.662145 4167 pending_rounds.cc:62] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:05.662294 4168 tablet_replica.cc:330] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: stopping tablet replica
I20260811 18:57:05.662266 4167 pending_rounds.cc:68] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400059424886784 op_type: NO_OP noop_request { }
I20260811 18:57:05.662506 4167 raft_consensus.cc:2866] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:05.662663 4167 raft_consensus.cc:2258] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.662626 4168 raft_consensus.cc:2229] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:05.662783 4230 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.003s
I20260811 18:57:05.662900 4168 pending_rounds.cc:62] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Trying to abort 2 pending ops.
I20260811 18:57:05.663031 4168 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400066380767232 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" new_extra_config { } }
I20260811 18:57:05.663342 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:05.663566 4168 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:55448: Aborted: Op aborted
I20260811 18:57:05.664841 4168 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062060187648 op_type: NO_OP noop_request { }
I20260811 18:57:05.665014 4168 raft_consensus.cc:2866] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:05.665061 4162 tablet_bootstrap.cc:654] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.665180 4168 raft_consensus.cc:2258] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.663039 4230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "d85eb97df7824bc9bad9dca349b0c180"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-zpfg"
I20260811 18:57:05.666064 4167 ts_tablet_manager.cc:1905] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.667723 4168 ts_tablet_manager.cc:1905] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.667706 4230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 18:57:05.677520 4359 log.cc:924] Time spent T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.125s user 0.000s sys 0.000s
=================================================================
==3947==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190000bf988 at pc 0x000000967ea4 bp 0x7fc79187f570 sp 0x7fc79187f568
READ of size 4 at 0x6190000bf988 thread T54 (rpc worker-4005)
I20260811 18:57:05.681866 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:05.690676 4162 tablet_bootstrap.cc:492] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.690936 4162 ts_tablet_manager.cc:1397] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.003s
I20260811 18:57:05.692020 4162 raft_consensus.cc:348] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.692552 4162 raft_consensus.cc:374] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.692665 4162 raft_consensus.cc:729] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.693018 4162 consensus_queue.cc:260] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.693789 4162 ts_tablet_manager.cc:1428] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260811 18:57:05.694322 4162 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.697446 4167 ts_tablet_manager.cc:1918] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:05.697849 4167 log.cc:1196] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:05.699116 4168 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:05.700865 4168 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-2/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:05.707552 4168 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P 59811bdf82234ef7ac5865dbecfcc388: Deleting consensus metadata
I20260811 18:57:05.708313 4167 ts_tablet_manager.cc:1939] T 3085d74b0bf7447d9c247b7e8872b215 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.709568 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 103) took 1624 ms. Trace:
I20260811 18:57:05.709802 4167 rpcz_store.cc:276] 0811 18:57:04.084645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.084700 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:05.659586 (+1574886us) ts_tablet_manager.cc:1145] Deleting tablet 3085d74b0bf7447d9c247b7e8872b215
0811 18:57:05.659604 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.682058 (+ 22454us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.697272 (+ 15214us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.709489 (+ 12217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1501326}
I20260811 18:57:05.697554 4162 tablet_bootstrap.cc:654] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.712311 4167 tablet_replica.cc:330] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.712730 4167 raft_consensus.cc:2229] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.713092 4167 raft_consensus.cc:2258] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.714063 4167 ts_tablet_manager.cc:1905] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.690984 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:05.716815 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 6d1d58835ad24749a00ce27b4bc73c59 deleted and replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
W20260811 18:57:05.692198 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6381a951a966473c80670b0dd1aff062 consensus state has no leader
I20260811 18:57:05.693181 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc0e1659d31645b28fe322618e22cb6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.693949 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61c8443df21949ff9bdc63fa6aa34108 consensus state has no leader
W20260811 18:57:05.694329 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 consensus state has no leader
I20260811 18:57:05.695228 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a945f1a824043b6b99704ee0994c6c6" } new_table_name: "default.Table_44e9cbdec47545899467895fd869b389" modify_external_catalogs: true
W20260811 18:57:05.720510 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc0e1659d31645b28fe322618e22cb6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.713681 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5345e1fe57284913bcf778715a827515"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.714715 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_897c0f5693354801afbbaa4de2ad5252"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.723017 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_897c0f5693354801afbbaa4de2ad5252 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:05.725068 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5345e1fe57284913bcf778715a827515 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:05.726899 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44991.8us
W20260811 18:57:05.727439 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 9f8c0444ec2242299482736a73551f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f8c0444ec2242299482736a73551f80 already in progress: creating tablet
W20260811 18:57:05.727957 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 927fb981490b4cecabe40314d6f71048 because the tablet was not found. No further retry: Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
W20260811 18:57:05.728386 3541 catalog_manager.cc:4808] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 634c06d76eca404a88f326e905882930 because the tablet was not found. No further retry: Not found: Tablet not found: 634c06d76eca404a88f326e905882930
I20260811 18:57:05.745002 4165 tablet_bootstrap.cc:492] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: No bootstrap required, opened a new log
I20260811 18:57:05.745405 4165 ts_tablet_manager.cc:1397] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Time spent bootstrapping tablet: real 0.586s user 0.008s sys 0.004s
I20260811 18:57:05.746659 4165 raft_consensus.cc:348] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.747061 4165 raft_consensus.cc:374] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.747267 4165 raft_consensus.cc:729] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59811bdf82234ef7ac5865dbecfcc388, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.747720 4165 consensus_queue.cc:260] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.748982 4165 ts_tablet_manager.cc:1428] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:05.749811 4165 tablet_bootstrap.cc:492] T 5df7a07983174dd1aaf48951c9ff1a1b P 59811bdf82234ef7ac5865dbecfcc388: Bootstrap starting.
I20260811 18:57:05.754689 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.756181 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.756213 4167 ts_tablet_manager.cc:1918] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.756481 4167 log.cc:1196] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e355634612f64c14965d7af2fa442cb7
I20260811 18:57:05.757139 4167 ts_tablet_manager.cc:1939] T e355634612f64c14965d7af2fa442cb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.758082 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.758745 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 104) took 1664 ms. Trace:
I20260811 18:57:05.758952 4167 rpcz_store.cc:276] 0811 18:57:04.094316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.102510 (+ 8194us) service_pool.cc:227] Handling call
0811 18:57:05.712159 (+1609649us) ts_tablet_manager.cc:1145] Deleting tablet e355634612f64c14965d7af2fa442cb7
0811 18:57:05.712175 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.723874 (+ 11699us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.755916 (+ 32042us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.758687 (+ 2771us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527175}
W20260811 18:57:05.760650 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
W20260811 18:57:05.761179 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:05.761255 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 105) took 1360 ms. Trace:
I20260811 18:57:05.761406 4167 rpcz_store.cc:276] 0811 18:57:04.400675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.401139 (+ 464us) service_pool.cc:227] Handling call
0811 18:57:05.759427 (+1358288us) ts_tablet_manager.cc:1145] Deleting tablet 60274115fe164d85badcf2e72d640f54
0811 18:57:05.759441 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.761199 (+ 1758us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1357839}
W20260811 18:57:05.761651 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
I20260811 18:57:05.762090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 107) took 1330 ms. Trace:
W20260811 18:57:05.762043 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d9421bc220774f66b90bf537a47f1e13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9421bc220774f66b90bf537a47f1e13 already in progress: creating tablet
I20260811 18:57:05.762284 4167 rpcz_store.cc:276] 0811 18:57:04.431171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.434430 (+ 3259us) service_pool.cc:227] Handling call
0811 18:57:05.761813 (+1327383us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:05.761826 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.762015 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1264275}
W20260811 18:57:05.762553 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
I20260811 18:57:05.762825 4167 tablet_replica.cc:330] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:05.762887 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
W20260811 18:57:05.763334 3541 catalog_manager.cc:4920] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]): Aborted: Op aborted
I20260811 18:57:05.763600 3541 catalog_manager.cc:4841] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:05.771917 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.789765 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.790685 4167 raft_consensus.cc:2229] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.791879 4167 raft_consensus.cc:2258] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.793298 4167 ts_tablet_manager.cc:1905] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.800308 4293 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.800654 4293 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.801261 4006 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d1d58835ad24749a00ce27b4bc73c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 67e63ee9b9fd4b12ae791b9d5bc2e648) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:05.801910 4293 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.803069 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef0a376939d4d57b0d95db644f25dd9 with delete_type TABLET_DATA_DELETED (replaced by tablet f61d3822fc2d4b569f223238a6464e0a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.803676 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.804136 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.804675 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:05.804962 4169 log.cc:1196] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:05.806217 4361 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:05.807459 4169 ts_tablet_manager.cc:1939] T 5df7a07983174dd1aaf48951c9ff1a1b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:05.808109 4169 tablet_replica.cc:330] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:05.808348 4169 raft_consensus.cc:2229] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:05.808501 4162 tablet_bootstrap.cc:492] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.808657 4169 pending_rounds.cc:62] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:05.808686 4162 ts_tablet_manager.cc:1397] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.025s
I20260811 18:57:05.808728 4169 pending_rounds.cc:68] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400066246193152 op_type: NO_OP noop_request { }
I20260811 18:57:05.808895 4169 raft_consensus.cc:2866] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:05.809022 4169 raft_consensus.cc:2258] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.810199 4162 raft_consensus.cc:348] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.810496 4162 raft_consensus.cc:374] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.810648 4162 raft_consensus.cc:729] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.810912 4162 consensus_queue.cc:260] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:05.811551 4162 ts_tablet_manager.cc:1428] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:05.811920 4162 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:05.813735 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b70decfaca9c4a9f87915d80838c6061" } modify_external_catalogs: true
I20260811 18:57:05.813961 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b70decfaca9c4a9f87915d80838c6061" } modify_external_catalogs: true
I20260811 18:57:05.814060 4162 tablet_bootstrap.cc:654] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.802270 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:05.815470 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0190cbd859274fcfa09c6fab899c4843 consensus state has no leader
W20260811 18:57:05.815656 3824 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.816035 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5df7a07983174dd1aaf48951c9ff1a1b (table default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]) successfully deleted
I20260811 18:57:05.806510 4361 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.816797 4163 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:05.819335 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:05.833860 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7b3a14979e84b1e8ec6ed3f49b93b00 consensus state has no leader
I20260811 18:57:05.835023 4163 ts_tablet_manager.cc:1397] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.362s user 0.051s sys 0.111s
W20260811 18:57:05.836998 3824 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.839376 4163 raft_consensus.cc:348] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.840581 3824 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.841813 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:05.846468 4163 raft_consensus.cc:374] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.846694 4163 raft_consensus.cc:729] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.847237 4165 tablet_bootstrap.cc:654] T 5df7a07983174dd1aaf48951c9ff1a1b P 59811bdf82234ef7ac5865dbecfcc388: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:05.847242 4163 consensus_queue.cc:260] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.848321 4293 raft_consensus.cc:2726] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.848657 4169 ts_tablet_manager.cc:1905] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:05.849243 3824 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.850051 4036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:05.850824 4361 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:05.852511 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.853904 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.855125 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 3085d74b0bf7447d9c247b7e8872b215 (table default.Table_5c59d98631a74a07a2d35f6b4bc209a3 [id=445215db3b154d81ab69f783cf110258]) successfully deleted
I20260811 18:57:05.855394 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e355634612f64c14965d7af2fa442cb7 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]) successfully deleted
W20260811 18:57:05.855660 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 60274115fe164d85badcf2e72d640f54 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60274115fe164d85badcf2e72d640f54 already in progress: creating tablet
W20260811 18:57:05.855899 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2852f9e5f67c4025836018af553388a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2852f9e5f67c4025836018af553388a5 already in progress: creating tablet
W20260811 18:57:05.857264 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 6d1d58835ad24749a00ce27b4bc73c59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d1d58835ad24749a00ce27b4bc73c59 already in progress: creating tablet
I20260811 18:57:05.863099 4003 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.863719 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df7a07983174dd1aaf48951c9ff1a1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.863950 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.865381 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efb663060ef457cbece962fce701eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.865515 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:05.866580 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.867216 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.868259 4011 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.869450 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.871428 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.871469 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 828993fb34bb4c62bce2a9322d7c992a (DEFAULT_TABLE table=default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.872066 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.872123 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828993fb34bb4c62bce2a9322d7c992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.873111 4230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:05.874279 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.874934 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.875030 4230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:05.875680 4230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:05.875626 4000 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.876298 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.874114 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.879062 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.879623 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.881070 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 494939a4eb5a45d6a123d174ad513bcb (DEFAULT_TABLE table=default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.881732 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.883306 4001 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.883769 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494939a4eb5a45d6a123d174ad513bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.883880 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.885380 4163 ts_tablet_manager.cc:1428] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.001s sys 0.004s
I20260811 18:57:05.887112 4013 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.887954 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.893831 3824 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:05.895037 3824 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:05.895191 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.895515 4383 raft_consensus.cc:2726] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:05.896454 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.896688 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 (DEFAULT_TABLE table=default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.897257 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36724a83bf9844b9a27a88ca36a9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.907204 3867 tablet_service.cc:1467] Processing CreateTablet for tablet e3021c51525846e199f279f988248af0 (DEFAULT_TABLE table=default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.908439 3998 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.908874 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:05.910063 3541 catalog_manager.cc:4914] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]),no further retry: Not found: Tablet not found: 6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:05.910423 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b591f65f304846aab2374c6ada0dd0ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:05.910794 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.911376 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b591f65f304846aab2374c6ada0dd0ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.912631 3996 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.914638 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:05.915521 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:05.917433 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.921696 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.921799 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.922777 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.924325 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.929227 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.930433 4167 ts_tablet_manager.cc:1918] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:05.933758 4167 log.cc:1196] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/634c06d76eca404a88f326e905882930
I20260811 18:57:05.934602 4167 ts_tablet_manager.cc:1939] T 634c06d76eca404a88f326e905882930 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:05.936340 4036 raft_consensus.cc:1261] T 9c18120da2d245f495c52a670047760d P 59811bdf82234ef7ac5865dbecfcc388 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:05.936673 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 106) took 1506 ms. Trace:
I20260811 18:57:05.936842 4167 rpcz_store.cc:276] 0811 18:57:04.429880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.429949 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:05.762713 (+1332764us) ts_tablet_manager.cc:1145] Deleting tablet 634c06d76eca404a88f326e905882930
0811 18:57:05.762725 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:05.868673 (+105948us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.874398 (+ 5725us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:05.936171 (+ 61773us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55,"tablet-delete.queue_time_us":1265169}
I20260811 18:57:05.938740 4167 tablet_replica.cc:330] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:05.939134 4167 raft_consensus.cc:2229] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:05.940948 4330 consensus_queue.cc:1035] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:05.945011 4163 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:05.947678 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.948354 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.956204 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.957475 4167 raft_consensus.cc:2258] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:05.960567 4162 tablet_bootstrap.cc:492] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:05.962468 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.963029 4162 ts_tablet_manager.cc:1397] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.149s user 0.024s sys 0.035s
I20260811 18:57:05.963850 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3021c51525846e199f279f988248af0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:05.965016 3676 consensus_peers.cc:487] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd3b82603ec340ed9cabe9b2ff62a8b3. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:05.965317 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.962829 4167 ts_tablet_manager.cc:1905] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:05.966702 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.967572 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.969127 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.969841 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.971382 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.972699 3859 tablet_service.cc:1467] Processing CreateTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 (DEFAULT_TABLE table=default.Table_dcfccd3951dc4875a1d70530eb514fb2 [id=3c7daa88956747719fb78e9860e7039f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.973729 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 (DEFAULT_TABLE table=default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.973726 4162 raft_consensus.cc:348] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.974395 4162 raft_consensus.cc:374] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:05.974606 4162 raft_consensus.cc:729] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:05.976161 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.977972 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:05.978642 3892 raft_consensus.cc:1261] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:05.980640 4162 consensus_queue.cc:260] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:05.981122 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:05.982858 4162 ts_tablet_manager.cc:1428] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.020s user 0.000s sys 0.004s
I20260811 18:57:05.982683 4330 consensus_queue.cc:1035] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:05.984994 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 634c06d76eca404a88f326e905882930 (table default.Table_de7a638d5c564c84b20b870f9c3a97e5 [id=93a135134c1542eb8b5031fc7f223526]) successfully deleted
W20260811 18:57:05.985445 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5ae200f3814e431a8288c30fe94bffcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ae200f3814e431a8288c30fe94bffcf already in progress: creating tablet
I20260811 18:57:05.985698 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_923c4369260d436fafb1aff17a2ec8d4" } modify_external_catalogs: true
I20260811 18:57:05.986123 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_923c4369260d436fafb1aff17a2ec8d4" } modify_external_catalogs: true
W20260811 18:57:05.985762 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 508f0b39cd654aa991a2928a4b067762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 508f0b39cd654aa991a2928a4b067762 already in progress: creating tablet
I20260811 18:57:05.994525 3866 tablet_service.cc:1467] Processing CreateTablet for tablet e5fd9fce095e4a769dbb38c7903359cc (DEFAULT_TABLE table=default.Table_4eaba0dd4bfc4e45878b027d222974fd [id=b3ed273027a946aab2041419fc3c11e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.016705 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ad5f39f59b8447cb3f15b9646e4b852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.017720 4162 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.028331 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fd9fce095e4a769dbb38c7903359cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.030262 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 125757b8c1a6448b841c63339f392339 (DEFAULT_TABLE table=default.Table_6460e311a6da4deaba4efce4c6dbd883 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.030961 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 125757b8c1a6448b841c63339f392339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.036207 4383 raft_consensus.cc:482] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.036752 4383 raft_consensus.cc:504] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.037345 4361 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.037642 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a08e6dbf04e3fa58689ee0cd45db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.038144 4361 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.039094 4361 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.039611 4330 raft_consensus.cc:482] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.039887 4330 raft_consensus.cc:504] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.042038 4330 leader_election.cc:288] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.046414 4383 leader_election.cc:288] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:06.052639 3676 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8efb663060ef457cbece962fce701eb1
I20260811 18:57:06.069773 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db" } modify_external_catalogs: true
I20260811 18:57:06.070166 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db" } modify_external_catalogs: true
W20260811 18:57:06.070314 3678 consensus_peers.cc:487] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd3b82603ec340ed9cabe9b2ff62a8b3. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:06.070830 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 58245us
I20260811 18:57:06.070564 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.071270 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91346cfabb664d0ebfc876dacc125c50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.072400 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91346cfabb664d0ebfc876dacc125c50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.073751 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d22535c5f02a4044aad07546138fca10" } modify_external_catalogs: true
I20260811 18:57:06.074139 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d22535c5f02a4044aad07546138fca10" } modify_external_catalogs: true
I20260811 18:57:06.075070 4330 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.075449 4330 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.080137 4330 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.085486 4162 tablet_bootstrap.cc:654] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.095043 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8397339a34974af9b4e8f20778beb8c7" } modify_external_catalogs: true
I20260811 18:57:06.095407 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8397339a34974af9b4e8f20778beb8c7" } modify_external_catalogs: true
I20260811 18:57:06.098496 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:06.099354 3893 raft_consensus.cc:2454] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.099704 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 3744b3d608754de68c4c57b072d1dc15 (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.104689 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3744b3d608754de68c4c57b072d1dc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.104722 4035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efb663060ef457cbece962fce701eb1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.112593 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.113306 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f53cc0009ca74325a65d63adf0744d37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:06.113862 3857 tablet_service.cc:1467] Processing CreateTablet for tablet ef47211aa8864d13a55551da47804c19 (DEFAULT_TABLE table=default.Table_c281ac8dd4a444c08084edb70426f847 [id=59e59153732d407d94b287a5588e974c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.114264 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f53cc0009ca74325a65d63adf0744d37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.115325 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef47211aa8864d13a55551da47804c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.119138 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.119303 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb (DEFAULT_TABLE table=default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.124693 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94943cfbe41a47b3a36c81957a3adaeb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:06.126528 3824 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
W20260811 18:57:06.127118 3824 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.137518 4330 raft_consensus.cc:482] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.137777 4330 raft_consensus.cc:504] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.138669 4330 leader_election.cc:288] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.138947 3676 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.139667 4330 raft_consensus.cc:2781] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.139832 4330 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.139940 4330 raft_consensus.cc:3037] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.144912 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:06.147769 3893 raft_consensus.cc:2454] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.162649 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.163208 3743 raft_consensus.cc:2454] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:06.164959 3676 leader_election.cc:302] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.165971 4163 tablet_bootstrap.cc:654] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:06.172581 3678 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.172935 3678 leader_election.cc:302] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.173807 4331 raft_consensus.cc:2726] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:06.174474 3678 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.175990 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.184255 3824 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.184926 4331 raft_consensus.cc:2781] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.185660 4383 raft_consensus.cc:2781] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.185902 4383 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:06.186657 3678 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.187047 4383 raft_consensus.cc:3037] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.192780 4331 raft_consensus.cc:482] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.195338 4383 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.197324 4167 ts_tablet_manager.cc:1918] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:06.197702 4167 log.cc:1196] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/41558efe0344434db6e755804442d2de
I20260811 18:57:06.197706 4383 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.198735 4331 raft_consensus.cc:3037] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.201105 4167 ts_tablet_manager.cc:1939] T 41558efe0344434db6e755804442d2de P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.202100 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 118) took 1256 ms. Trace:
I20260811 18:57:06.202965 4167 rpcz_store.cc:276] 0811 18:57:04.945685 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:04.945739 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:05.937896 (+992157us) ts_tablet_manager.cc:1145] Deleting tablet 41558efe0344434db6e755804442d2de
0811 18:57:05.937907 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.083745 (+145838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.195291 (+111546us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.201997 (+ 6706us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":967008}
W20260811 18:57:06.203127 3824 leader_election.cc:341] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 927fb981490b4cecabe40314d6f71048
I20260811 18:57:06.203626 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 41558efe0344434db6e755804442d2de (table default.Table_9ccfd97c9c604a7b9f786709962c107c [id=3180baea9cf54ee2bfe3e67fe6197df3]) successfully deleted
I20260811 18:57:06.206404 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 119) took 1171 ms. Trace:
I20260811 18:57:06.212512 3824 leader_election.cc:302] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.215566 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:06.220250 3824 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.221069 4383 raft_consensus.cc:2726] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.221104 4167 rpcz_store.cc:276] 0811 18:57:05.034411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.041946 (+ 7535us) service_pool.cc:227] Handling call
0811 18:57:06.206012 (+1164066us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:06.206030 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.206239 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1145408}
W20260811 18:57:06.222077 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d106e0e1f4b845429b10f4e100f69a8b already in progress: creating tablet
I20260811 18:57:06.222865 4162 tablet_bootstrap.cc:492] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.222856 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08b8dcf7a56043efbbd40bb888616492" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:06.223249 3743 raft_consensus.cc:3037] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.223268 4162 ts_tablet_manager.cc:1397] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.240s user 0.009s sys 0.014s
I20260811 18:57:06.223661 4330 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.227701 4330 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.228402 4162 raft_consensus.cc:348] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.229046 4162 raft_consensus.cc:374] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.229205 4162 raft_consensus.cc:729] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.229590 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.229559 4162 consensus_queue.cc:260] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.233726 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.238274 3893 raft_consensus.cc:3037] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.247538 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 23f41a2638e74ba19c1c527b9fc796df (DEFAULT_TABLE table=default.Table_65d16e5449864d18a0a01e39d25ad8dd [id=29a0d1e06bea499bacc5851fd6d0db5c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.257014 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 133501us
I20260811 18:57:06.260535 4162 ts_tablet_manager.cc:1428] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260811 18:57:06.262519 4167 tablet_replica.cc:330] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:06.263005 4167 raft_consensus.cc:2229] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:06.263157 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet d19675f757674450aceb5942e1a51c99
I20260811 18:57:06.263561 4167 pending_rounds.cc:62] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:06.263619 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d3d6daff4b7540309ff58755758e9752
I20260811 18:57:06.263855 4167 pending_rounds.cc:68] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400064111112192 op_type: NO_OP noop_request { }
I20260811 18:57:06.264109 4167 raft_consensus.cc:2866] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.264624 4167 raft_consensus.cc:2258] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.265599 4167 ts_tablet_manager.cc:1905] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.268265 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.271178 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:06.272953 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
I20260811 18:57:06.273943 3893 raft_consensus.cc:2454] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:06.275565 3676 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:06.281838 3678 leader_election.cc:341] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.284276 4331 raft_consensus.cc:504] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.285562 4331 leader_election.cc:288] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.287712 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.288713 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:06.289007 3678 leader_election.cc:341] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.289453 4330 raft_consensus.cc:2781] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.289837 4330 raft_consensus.cc:686] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:06.291563 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23f41a2638e74ba19c1c527b9fc796df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.291625 4330 consensus_queue.cc:237] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.292325 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d1d58835ad24749a00ce27b4bc73c59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.293363 3893 raft_consensus.cc:3037] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.294724 3743 raft_consensus.cc:2454] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:06.306098 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:06.307785 3824 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.310585 4383 raft_consensus.cc:2781] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.310982 4383 raft_consensus.cc:686] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:06.312232 4162 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.312981 4010 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
W20260811 18:57:06.315131 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
I20260811 18:57:06.315855 4383 consensus_queue.cc:237] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.320593 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.321323 4330 raft_consensus.cc:482] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.321566 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.322184 4330 raft_consensus.cc:504] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.323642 4330 leader_election.cc:288] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.323998 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b092ff08e7a340bfb2b5ad86895ca81d" } new_table_name: "default.Table_666e839163c3494a9c95e78ff76bb015" modify_external_catalogs: true
I20260811 18:57:06.325218 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df7a07983174dd1aaf48951c9ff1a1b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.326200 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3392d9de7234297a490a976dfe10060"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.326767 3678 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.327031 4169 ts_tablet_manager.cc:1918] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:06.327397 4169 log.cc:1196] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/82d1339366a24a87b74bd1d411a3f6fe
W20260811 18:57:06.327450 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3392d9de7234297a490a976dfe10060 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.328346 4169 ts_tablet_manager.cc:1939] T 82d1339366a24a87b74bd1d411a3f6fe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.329002 4169 tablet_replica.cc:330] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:06.329142 3676 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:06.329264 4169 raft_consensus.cc:2229] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.329332 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:06.329587 3676 leader_election.cc:302] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.329738 4169 raft_consensus.cc:2258] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:06.330328 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.330476 4330 raft_consensus.cc:2726] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.331873 4163 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.332146 4163 ts_tablet_manager.cc:1397] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.446s user 0.004s sys 0.009s
I20260811 18:57:06.332564 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a108c0f554824b94acc46ae9330d1115"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.333354 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a108c0f554824b94acc46ae9330d1115 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.333909 4163 raft_consensus.cc:348] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.334234 4163 raft_consensus.cc:374] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.334882 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 82d1339366a24a87b74bd1d411a3f6fe (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:06.335670 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_27837256c8254b8c9b6493a7ca044751" } new_table_name: "default.Table_a5043728e7c842ea851193559d898d80" modify_external_catalogs: true
I20260811 18:57:06.336783 4162 tablet_bootstrap.cc:654] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.339680 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_314cf78c221448d1a5fa8708b3adf8b7" } new_table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" modify_external_catalogs: true
I20260811 18:57:06.343963 4167 ts_tablet_manager.cc:1918] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:06.349668 3541 catalog_manager.cc:4914] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): alter failed for tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]),no further retry: Not found: Tablet not found: 6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:06.344259 4167 log.cc:1196] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cd3b82603ec340ed9cabe9b2ff62a8b3
I20260811 18:57:06.357563 4167 ts_tablet_manager.cc:1939] T cd3b82603ec340ed9cabe9b2ff62a8b3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.358974 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 120) took 1225 ms. Trace:
I20260811 18:57:06.359273 4167 rpcz_store.cc:276] 0811 18:57:05.133662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.133803 (+ 141us) service_pool.cc:227] Handling call
0811 18:57:06.226478 (+1092675us) ts_tablet_manager.cc:1145] Deleting tablet cd3b82603ec340ed9cabe9b2ff62a8b3
0811 18:57:06.262393 (+ 35915us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.311285 (+ 48892us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.343705 (+ 32420us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.358890 (+ 15185us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091411}
I20260811 18:57:06.360241 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cd3b82603ec340ed9cabe9b2ff62a8b3 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
I20260811 18:57:06.371124 4411 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:06.377825 4169 ts_tablet_manager.cc:1905] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.363629 4163 raft_consensus.cc:729] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.381649 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 121) took 1245 ms. Trace:
I20260811 18:57:06.382066 4167 rpcz_store.cc:276] 0811 18:57:05.134924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.135189 (+ 265us) service_pool.cc:227] Handling call
0811 18:57:06.379474 (+1244285us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:06.379488 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.380712 (+ 1224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1243814}
I20260811 18:57:06.382404 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_178c66b5e0f1479a9a97248ecab4e779" } new_table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" modify_external_catalogs: true
I20260811 18:57:06.383283 3893 raft_consensus.cc:2454] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:06.383301 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9f8c0444ec2242299482736a73551f80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f8c0444ec2242299482736a73551f80 already in progress: creating tablet
I20260811 18:57:06.384034 4167 tablet_replica.cc:330] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:06.384001 3676 leader_election.cc:302] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.384857 4331 raft_consensus.cc:2781] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.384253 4163 consensus_queue.cc:260] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.388015 4331 raft_consensus.cc:686] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:06.388525 4163 ts_tablet_manager.cc:1428] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.056s user 0.000s sys 0.004s
I20260811 18:57:06.388682 4331 consensus_queue.cc:237] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.391021 4162 tablet_bootstrap.cc:492] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.391487 4162 ts_tablet_manager.cc:1397] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.079s user 0.003s sys 0.008s
I20260811 18:57:06.393412 4162 raft_consensus.cc:348] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.395352 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 has no consensus state
I20260811 18:57:06.396324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a594cb48b586458783a36cc76df1332c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.397173 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a594cb48b586458783a36cc76df1332c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.402638 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4eaba0dd4bfc4e45878b027d222974fd" } new_table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" modify_external_catalogs: true
I20260811 18:57:06.410452 4010 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.410662 3720 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.411190 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.411438 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.410691 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.415326 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.424027 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.424691 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.424964 4162 raft_consensus.cc:374] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.425091 4162 raft_consensus.cc:729] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.425494 4162 consensus_queue.cc:260] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.394740 4167 raft_consensus.cc:2229] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.425992 4009 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.426434 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.426730 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.427687 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.428725 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.430874 4000 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.431442 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.434604 4167 raft_consensus.cc:2258] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.434924 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.434945 4013 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.435568 4167 ts_tablet_manager.cc:1905] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.435629 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.438763 3996 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.439327 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.443257 4361 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.443607 4361 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.444576 4361 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.450387 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.450911 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.450963 4006 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.451805 4006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.452594 4162 ts_tablet_manager.cc:1428] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.047s user 0.000s sys 0.003s
I20260811 18:57:06.454839 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.456256 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.457139 4271 raft_consensus.cc:482] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.457401 4271 raft_consensus.cc:504] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:06.463500 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.464177 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.467823 4162 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.471678 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ff1be026ca404eada53f60f257ea367c (DEFAULT_TABLE table=default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.472637 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff1be026ca404eada53f60f257ea367c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.480831 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 (DEFAULT_TABLE table=default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.480803 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.481895 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba6246e6d29418cbca5f751b34f4ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.482993 3998 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.475755 4271 leader_election.cc:288] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.483530 3866 tablet_service.cc:1467] Processing CreateTablet for tablet ef393c341bb54baf87176db5b5254ec7 (DEFAULT_TABLE table=default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.484249 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef393c341bb54baf87176db5b5254ec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.486671 4001 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.487438 4004 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.490988 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.491231 3856 tablet_service.cc:1467] Processing CreateTablet for tablet c2247194db704f9abd54a8a0df1ca566 (DEFAULT_TABLE table=default.Table_db39f5cc5ace43a49931b2da095c6037 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.491971 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2247194db704f9abd54a8a0df1ca566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.494642 4004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.496052 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.499490 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 21054bdd3d7c400c95b1f41e41bafc79 (DEFAULT_TABLE table=default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.499936 4003 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.500304 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21054bdd3d7c400c95b1f41e41bafc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.507737 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 283c3fe4f41f4185ab4488d774e46299 (DEFAULT_TABLE table=default.Table_153f40da23a34fd4a2cba7e46cb4f96b [id=23b2b73efef04e0b8fbd8ba765b92c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.508548 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 283c3fe4f41f4185ab4488d774e46299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.522727 4003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.531013 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.531883 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.543491 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.544292 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:06.557112 3965 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:06.559140 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.559700 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.559918 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.559243 3872 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.562032 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.566607 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 04eb81bf6f454fbfadd27393af15ab17 (DEFAULT_TABLE table=default.Table_897c0f5693354801afbbaa4de2ad5252 [id=e98d6386b7fe47a9afba14f90faab197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.572533 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04eb81bf6f454fbfadd27393af15ab17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.584656 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (DEFAULT_TABLE table=default.Table_314cf78c221448d1a5fa8708b3adf8b7 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.585405 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.588124 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.591889 3713 tablet_service.cc:1467] Processing CreateTablet for tablet f61d3822fc2d4b569f223238a6464e0a (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.598847 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d (DEFAULT_TABLE table=default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:06.607113 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.607641 3824 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:06.608599 3967 leader_election.cc:341] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 3085d74b0bf7447d9c247b7e8872b215
I20260811 18:57:06.609051 3967 leader_election.cc:302] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59811bdf82234ef7ac5865dbecfcc388; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.609617 4271 raft_consensus.cc:2726] T 3085d74b0bf7447d9c247b7e8872b215 P 59811bdf82234ef7ac5865dbecfcc388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.609884 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a443fe209556489abb47f4faa5d696e5 (DEFAULT_TABLE table=default.Table_66613580530e4ae8989a795dd5d5ef69 [id=1a0803466858425cadc33f51f0933a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.623584 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d3822fc2d4b569f223238a6464e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.626215 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460e03cc14444c83b6c08ca194ca5a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.631455 4383 raft_consensus.cc:2726] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.611127 4163 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.632378 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a443fe209556489abb47f4faa5d696e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.641072 4413 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:06.669242 4163 tablet_bootstrap.cc:654] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.707798 4162 tablet_bootstrap.cc:654] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.707314 3547 catalog_manager.cc:5495] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:06.711027 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c70d9db064e45af91fd9901d85e211f consensus state has no leader
I20260811 18:57:06.711776 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a671c5d0d75c49779b94baa2b89c9cef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.712745 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a671c5d0d75c49779b94baa2b89c9cef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.712953 4169 ts_tablet_manager.cc:1918] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.713210 4169 log.cc:1196] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/494c09652d9e408880ca606ae0b30773
I20260811 18:57:06.713950 4169 ts_tablet_manager.cc:1939] T 494c09652d9e408880ca606ae0b30773 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.724344 4163 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.724937 4163 ts_tablet_manager.cc:1397] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.336s user 0.004s sys 0.004s
I20260811 18:57:06.730507 4163 raft_consensus.cc:348] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.731127 4163 raft_consensus.cc:374] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.731370 4163 raft_consensus.cc:729] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.735342 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 143) took 1178 ms. Trace:
I20260811 18:57:06.735432 4162 tablet_bootstrap.cc:492] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.735798 4162 ts_tablet_manager.cc:1397] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.283s user 0.005s sys 0.002s
I20260811 18:57:06.735648 4169 rpcz_store.cc:276] 0811 18:57:05.557259 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.557727 (+ 468us) service_pool.cc:227] Handling call
0811 18:57:06.328909 (+771182us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:06.328922 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.672392 (+343470us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.711802 (+ 39410us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.735160 (+ 23358us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":768021}
I20260811 18:57:06.737345 4162 raft_consensus.cc:348] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.737723 4162 raft_consensus.cc:374] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.737665 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 494c09652d9e408880ca606ae0b30773 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:06.737936 4162 raft_consensus.cc:729] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.739239 4163 consensus_queue.cc:260] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.739696 3676 consensus_peers.cc:487] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:06.740806 4167 ts_tablet_manager.cc:1918] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.743165 4167 log.cc:1196] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/82d1339366a24a87b74bd1d411a3f6fe
I20260811 18:57:06.747335 4167 ts_tablet_manager.cc:1939] T 82d1339366a24a87b74bd1d411a3f6fe P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.750172 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 125) took 1533 ms. Trace:
I20260811 18:57:06.750511 4169 tablet_replica.cc:330] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:06.751171 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 consensus state has no leader
I20260811 18:57:06.752132 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4db75b493caf48c0be36b2f872c70dd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.753633 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4db75b493caf48c0be36b2f872c70dd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.754913 4163 ts_tablet_manager.cc:1428] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.030s user 0.004s sys 0.000s
I20260811 18:57:06.755055 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet e215759d48c24d8cb99bbc3fc9009a66 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.755604 4163 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.755450 4167 rpcz_store.cc:276] 0811 18:57:05.216437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.218439 (+ 2002us) service_pool.cc:227] Handling call
0811 18:57:06.383803 (+1165364us) ts_tablet_manager.cc:1145] Deleting tablet 82d1339366a24a87b74bd1d411a3f6fe
0811 18:57:06.383825 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.679746 (+295921us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.740655 (+ 60909us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.750106 (+ 9451us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8445,"tablet-delete.queue_time_us":1150872}
I20260811 18:57:06.756358 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 82d1339366a24a87b74bd1d411a3f6fe (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:06.748232 4162 consensus_queue.cc:260] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.757318 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.758159 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 2852f9e5f67c4025836018af553388a5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.759104 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1a8b47adc7454c64a44013e919b3a485"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.759930 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a8b47adc7454c64a44013e919b3a485 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.761261 4162 ts_tablet_manager.cc:1428] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.025s user 0.002s sys 0.002s
I20260811 18:57:06.761572 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:06.762521 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9bb5acd56644427f9aec05c1ab8e46ee" } modify_external_catalogs: true
I20260811 18:57:06.762768 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9bb5acd56644427f9aec05c1ab8e46ee" } modify_external_catalogs: true
W20260811 18:57:06.764088 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e5fd9fce095e4a769dbb38c7903359cc consensus state has no leader
I20260811 18:57:06.764782 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fa5c52d66624877a73a99af0be2381a" } modify_external_catalogs: true
I20260811 18:57:06.765188 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fa5c52d66624877a73a99af0be2381a" } modify_external_catalogs: true
I20260811 18:57:06.765875 4162 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.768266 4167 tablet_replica.cc:330] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:06.769145 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_666e839163c3494a9c95e78ff76bb015 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:06.769198 4167 raft_consensus.cc:2229] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.769591 4167 raft_consensus.cc:2258] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.773092 4169 raft_consensus.cc:2229] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:06.773603 4169 pending_rounds.cc:62] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:06.773838 4169 pending_rounds.cc:68] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400069391585280 op_type: NO_OP noop_request { }
I20260811 18:57:06.774263 4169 raft_consensus.cc:2866] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.774693 4169 raft_consensus.cc:2258] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.775878 4169 ts_tablet_manager.cc:1905] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:06.780644 3678 consensus_peers.cc:487] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:06.782755 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.785867 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.786418 4167 ts_tablet_manager.cc:1905] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:06.789294 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
I20260811 18:57:06.791962 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.793340 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:06.794744 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.795701 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac4a8df2cd7346ffa89549c0750760ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.798846 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.799865 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:06.800100 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
I20260811 18:57:06.800827 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:06.801237 4163 tablet_bootstrap.cc:654] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.808614 4162 tablet_bootstrap.cc:654] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.819885 4169 ts_tablet_manager.cc:1918] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:06.820827 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 439ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:06.822432 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b4ff060dff9a4bf19670e304147315ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.823446 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4ff060dff9a4bf19670e304147315ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.831694 4169 log.cc:1196] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:06.832481 4169 ts_tablet_manager.cc:1939] T 08b8dcf7a56043efbbd40bb888616492 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.833504 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 144) took 1272 ms. Trace:
I20260811 18:57:06.833752 4169 rpcz_store.cc:276] 0811 18:57:05.561054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.573245 (+ 12191us) service_pool.cc:227] Handling call
0811 18:57:06.741869 (+1168624us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:06.741887 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.803687 (+ 61800us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.819620 (+ 15933us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.833457 (+ 13837us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21904,"tablet-delete.queue_time_us":1167523}
I20260811 18:57:06.834544 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 08b8dcf7a56043efbbd40bb888616492 (table default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]) successfully deleted
I20260811 18:57:06.835953 4163 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.836228 4163 ts_tablet_manager.cc:1397] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.081s user 0.000s sys 0.007s
I20260811 18:57:06.838407 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:06.837956 4163 raft_consensus.cc:348] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.838660 4163 raft_consensus.cc:374] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.838918 4163 raft_consensus.cc:729] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.839217 3550 catalog_manager.cc:6565] ReplaceTablet: tablet e215759d48c24d8cb99bbc3fc9009a66 deleted and replaced by tablet c08436f33db043b1bc6395cd92102e59
I20260811 18:57:06.839533 4162 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:06.839874 4162 ts_tablet_manager.cc:1397] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.002s
I20260811 18:57:06.839833 4383 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.839846 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f53cc0009ca74325a65d63adf0744d37" } modify_external_catalogs: true
I20260811 18:57:06.840082 4383 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.840320 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f53cc0009ca74325a65d63adf0744d37" } modify_external_catalogs: true
I20260811 18:57:06.841240 4383 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.841311 4162 raft_consensus.cc:348] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.841806 4162 raft_consensus.cc:374] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.842018 4162 raft_consensus.cc:729] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.839331 4163 consensus_queue.cc:260] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.842526 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.843413 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.843984 3744 raft_consensus.cc:2454] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:06.844880 3824 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:06.845602 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.845713 4009 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:55448
I20260811 18:57:06.846925 4383 raft_consensus.cc:2781] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.847114 4383 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.847378 4383 raft_consensus.cc:3037] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.852396 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:06.852981 3824 leader_election.cc:341] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.853869 4383 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.855552 4383 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:06.856303 3541 catalog_manager.cc:4821] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
I20260811 18:57:06.856421 4163 ts_tablet_manager.cc:1428] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.002s sys 0.003s
I20260811 18:57:06.856999 4163 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:06.860281 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
I20260811 18:57:06.861760 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
W20260811 18:57:06.861850 3824 leader_election.cc:341] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.862187 3744 raft_consensus.cc:3037] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.862999 4162 consensus_queue.cc:260] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.868181 4169 tablet_replica.cc:330] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:06.868636 4169 raft_consensus.cc:2229] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.869032 4169 raft_consensus.cc:2258] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.858072 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7eb5b8b42dae46398c5bf3afe86e1b09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.870915 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eb5b8b42dae46398c5bf3afe86e1b09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.875859 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db39f5cc5ace43a49931b2da095c6037" } new_table_name: "default.Table_e4b52f563c5f4629a03019c85bfbb653" modify_external_catalogs: true
I20260811 18:57:06.877458 4169 ts_tablet_manager.cc:1905] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.879767 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet dde27d8cc3eb4d09879e717fc90305fd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.885160 4163 tablet_bootstrap.cc:654] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.886307 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 5ae200f3814e431a8288c30fe94bffcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:06.893229 4162 ts_tablet_manager.cc:1428] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.053s user 0.002s sys 0.002s
I20260811 18:57:06.907261 4167 ts_tablet_manager.cc:1918] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.907609 4167 log.cc:1196] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8964f92255884b3aae0cac60f019fdb7
I20260811 18:57:06.908576 4167 ts_tablet_manager.cc:1939] T 8964f92255884b3aae0cac60f019fdb7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:06.909086 3744 raft_consensus.cc:2454] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:06.909945 3824 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.910749 4383 raft_consensus.cc:2781] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:06.910980 4383 raft_consensus.cc:686] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:06.911339 4162 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:06.911460 4383 consensus_queue.cc:237] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.915844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 133) took 1447 ms. Trace:
I20260811 18:57:06.916106 4167 rpcz_store.cc:276] 0811 18:57:05.494519 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.500270 (+ 5751us) service_pool.cc:227] Handling call
0811 18:57:06.767949 (+1267679us) ts_tablet_manager.cc:1145] Deleting tablet 8964f92255884b3aae0cac60f019fdb7
0811 18:57:06.767968 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.830599 (+ 62631us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.859093 (+ 28494us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.915738 (+ 56645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1265992}
I20260811 18:57:06.917068 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f501d17d05204712b318f4d95afce6dc
I20260811 18:57:06.916793 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8964f92255884b3aae0cac60f019fdb7 (table default.Table_34ca03d0c0144111b5d797fed15d35a6 [id=8f81ae4ba0c344fe85cf0f3b6f19bcce]) successfully deleted
I20260811 18:57:06.919914 4162 tablet_bootstrap.cc:654] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.920054 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_44def5389adf4d9ca825712dfb862206"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.921095 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44def5389adf4d9ca825712dfb862206 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.921715 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:06.922586 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1329ea40dbd4f748a0e35686a5e867f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:06.925225 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 134) took 1355 ms. Trace:
I20260811 18:57:06.925573 4167 rpcz_store.cc:276] 0811 18:57:05.570165 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.594406 (+ 24241us) service_pool.cc:227] Handling call
0811 18:57:06.924850 (+1330444us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:06.924868 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.925095 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1330118}
W20260811 18:57:06.926128 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
I20260811 18:57:06.926538 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 140) took 1269 ms. Trace:
I20260811 18:57:06.926769 4167 rpcz_store.cc:276] 0811 18:57:05.656359 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656890 (+ 531us) service_pool.cc:227] Handling call
0811 18:57:06.926081 (+1269191us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:06.926097 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.926255 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268893}
I20260811 18:57:06.927412 4167 tablet_replica.cc:330] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:06.927665 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
I20260811 18:57:06.927739 4169 ts_tablet_manager.cc:1918] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:06.927801 4167 raft_consensus.cc:2229] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:06.928145 4167 pending_rounds.cc:62] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 2 pending ops.
I20260811 18:57:06.928373 4167 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400066380767232 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" new_extra_config { } }
I20260811 18:57:06.929236 4167 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062060187648 op_type: NO_OP noop_request { }
I20260811 18:57:06.929486 4167 raft_consensus.cc:2866] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:06.929735 4167 raft_consensus.cc:2258] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:06.942600 4167 ts_tablet_manager.cc:1905] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:06.944295 4169 log.cc:1196] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/927fb981490b4cecabe40314d6f71048
I20260811 18:57:06.945163 4169 ts_tablet_manager.cc:1939] T 927fb981490b4cecabe40314d6f71048 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:06.945257 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_abaf87a40e04427fba978e8a67b89f0c" } new_table_name: "default.Table_9e2e268d15894407ab276766ff611c86" modify_external_catalogs: true
I20260811 18:57:06.946604 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 146) took 1370 ms. Trace:
I20260811 18:57:06.946820 4169 rpcz_store.cc:276] 0811 18:57:05.575579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.575653 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:06.867993 (+1292340us) ts_tablet_manager.cc:1145] Deleting tablet 927fb981490b4cecabe40314d6f71048
0811 18:57:06.868012 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.918518 (+ 50506us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.927566 (+ 9048us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:06.946522 (+ 18956us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1289286}
I20260811 18:57:06.947696 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 927fb981490b4cecabe40314d6f71048 (table default.Table_75948600dcf347f89478127c9f9b96fd [id=a1282d6fa96147be8565e1aea2f2b116]) successfully deleted
W20260811 18:57:06.948688 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2247194db704f9abd54a8a0df1ca566 consensus state has no leader
I20260811 18:57:06.949721 4383 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.949992 4383 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.951071 4383 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.952890 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:06.953635 3824 leader_election.cc:341] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.957022 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9c18120da2d245f495c52a670047760d (table default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137]). Expected version 1 got 0
I20260811 18:57:06.957866 4163 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:06.957824 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_d62ce6ce67e045c3a28f6c71419187b5 [id=221d836949c341a09235288dc5802137] alter complete (version 1)
I20260811 18:57:06.958124 4163 ts_tablet_manager.cc:1397] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.008s
I20260811 18:57:06.961760 4163 raft_consensus.cc:348] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.962324 4163 raft_consensus.cc:374] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:06.963348 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 145) took 1387 ms. Trace:
I20260811 18:57:06.963513 4169 rpcz_store.cc:276] 0811 18:57:05.575258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.575697 (+ 439us) service_pool.cc:227] Handling call
0811 18:57:06.953976 (+1378279us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:06.954018 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.962509 (+ 8491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1374978}
I20260811 18:57:06.963690 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
W20260811 18:57:06.964270 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 331396c9962f48ed9ea6dc66f1660902 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 331396c9962f48ed9ea6dc66f1660902 already in progress: creating tablet
I20260811 18:57:06.964713 4163 raft_consensus.cc:729] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:06.965525 4163 consensus_queue.cc:260] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.967276 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.967680 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:06.969300 4163 ts_tablet_manager.cc:1428] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.002s
I20260811 18:57:06.969878 4383 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.969887 4009 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.970130 4383 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.971338 4009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.971280 4383 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:06.973009 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.973453 4163 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:06.974073 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.975153 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
W20260811 18:57:06.975934 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.976274 3824 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:06.976821 4383 raft_consensus.cc:2726] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:06.979139 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7eb5b8b42dae46398c5bf3afe86e1b09" } new_table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" modify_external_catalogs: true
I20260811 18:57:06.980630 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 147) took 1236 ms. Trace:
I20260811 18:57:06.980839 4169 rpcz_store.cc:276] 0811 18:57:05.742830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.742926 (+ 96us) service_pool.cc:227] Handling call
0811 18:57:06.965215 (+1222289us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:06.965235 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.979524 (+ 14289us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1210267}
I20260811 18:57:06.981176 4331 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:06.981499 4331 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:06.981715 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:06.982761 4331 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.984125 4163 tablet_bootstrap.cc:654] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.984102 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:06.984710 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:06.985260 3678 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.985397 3890 raft_consensus.cc:2454] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:06.983662 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bcaa2e21fe064cbea9a28f034c63eae6" } new_table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" modify_external_catalogs: true
I20260811 18:57:06.986171 3676 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:06.986553 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 150) took 1230 ms. Trace:
I20260811 18:57:06.987087 4331 raft_consensus.cc:2781] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:06.987293 4331 raft_consensus.cc:482] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:06.987614 4331 raft_consensus.cc:3037] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.991866 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:06.992666 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.992815 4331 raft_consensus.cc:504] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:06.994577 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:06.994952 3890 raft_consensus.cc:3037] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:06.995550 4331 leader_election.cc:288] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:06.996605 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ba169c32ef48b984e649b211ee2af1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:06.997588 3678 leader_election.cc:341] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:06.998912 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:06.999979 3744 raft_consensus.cc:2454] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:07.000787 3824 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:07.006521 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:07.007215 4383 raft_consensus.cc:2781] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:07.007515 4383 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:07.007843 4383 raft_consensus.cc:3037] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:07.009267 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0c4cb29b3ec48749d83adaa68477c84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.013247 4001 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.014168 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.015259 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 799ba9a044e145ec89d0025c03f391ea (DEFAULT_TABLE table=default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.015705 4383 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.016098 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799ba9a044e145ec89d0025c03f391ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.016835 4383 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:07.017166 3890 raft_consensus.cc:2454] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:07.018141 3676 leader_election.cc:302] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:07.026662 4331 raft_consensus.cc:2781] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:07.026574 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388"
W20260811 18:57:07.028757 3824 leader_election.cc:341] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:07.035142 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:07.035568 3744 raft_consensus.cc:3037] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:07.036183 3866 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.036906 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.036983 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b6f36ab6bb774257b5c2692c64eff63d (DEFAULT_TABLE table=default.Table_5345e1fe57284913bcf778715a827515 [id=22529e4acc82410a93bca07dfda7d1b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.037549 4010 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.037822 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6f36ab6bb774257b5c2692c64eff63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.038203 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.050654 4383 raft_consensus.cc:482] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:07.051119 4383 raft_consensus.cc:504] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.053963 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2852f9e5f67c4025836018af553388a5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:07.054858 3824 leader_election.cc:341] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:07.064560 3859 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.065440 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.065407 3712 tablet_service.cc:1467] Processing CreateTablet for tablet ad424c8843ff434ba0449a86849bb5a0 (DEFAULT_TABLE table=default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.066228 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad424c8843ff434ba0449a86849bb5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.070412 4383 leader_election.cc:288] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:07.074100 3855 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.074908 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.986917 4169 rpcz_store.cc:276] 0811 18:57:05.755930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.755997 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:06.986275 (+1230278us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:06.986292 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.986486 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1229902}
I20260811 18:57:07.075860 4169 tablet_replica.cc:330] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:07.076179 4169 raft_consensus.cc:2229] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:07.080042 4167 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:07.089579 4169 raft_consensus.cc:2258] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:07.090569 4169 ts_tablet_manager.cc:1905] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:07.080467 4167 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:07.099387 4167 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:07.100695 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 137) took 1445 ms. Trace:
I20260811 18:57:07.100862 4167 rpcz_store.cc:276] 0811 18:57:05.655533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656133 (+ 600us) service_pool.cc:227] Handling call
0811 18:57:06.927244 (+1271111us) ts_tablet_manager.cc:1145] Deleting tablet 6ef0a376939d4d57b0d95db644f25dd9
0811 18:57:06.927256 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:06.976403 (+ 49147us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.074948 (+ 98545us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.100626 (+ 25678us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1269804}
I20260811 18:57:07.101555 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:07.101805 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 139) took 1445 ms. Trace:
I20260811 18:57:07.102034 4167 rpcz_store.cc:276] 0811 18:57:05.655926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656194 (+ 268us) service_pool.cc:227] Handling call
0811 18:57:07.101456 (+1445262us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:07.101487 (+ 31us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.101674 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1443618}
I20260811 18:57:07.140240 4163 tablet_bootstrap.cc:492] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:07.312342 4471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:38889 every 8 connection(s)
I20260811 18:57:07.140578 4163 ts_tablet_manager.cc:1397] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.171s user 0.010s sys 0.012s
I20260811 18:57:08.303539 4163 raft_consensus.cc:348] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.303872 4163 raft_consensus.cc:374] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.303993 4163 raft_consensus.cc:729] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.304265 4163 consensus_queue.cc:260] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:07.956779 4485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:08.304965 4163 ts_tablet_manager.cc:1428] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:08.305365 4163 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:08.305069 4485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:07.313058 4230 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:38889
I20260811 18:57:08.307153 4485 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
W20260811 18:57:07.103190 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
W20260811 18:57:08.307569 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 1.20463e+06us
I20260811 18:57:08.308525 4230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:57:07.167467 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 08b8dcf7a56043efbbd40bb888616492 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.167640 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.248467 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.249888 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.250545 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:07.283938 4169 ts_tablet_manager.cc:1918] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:08.309765 4169 log.cc:1196] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:08.309899 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb (DEFAULT_TABLE table=default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.310328 4169 ts_tablet_manager.cc:1939] T 6d1d58835ad24749a00ce27b4bc73c59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:07.097087 3744 raft_consensus.cc:2454] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:08.310465 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515d5b5afbd74ec5ab90a503aef2f4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.310560 3744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:51189 (request call id 24) took 1287 ms. Trace:
I20260811 18:57:08.310705 3744 rpcz_store.cc:276] 0811 18:57:07.023209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.033128 (+ 9919us) service_pool.cc:227] Handling call
0811 18:57:08.310511 (+1277383us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:07.107930 3998 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.311267 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 153) took 2553 ms. Trace:
I20260811 18:57:08.311393 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.311395 4169 rpcz_store.cc:276] 0811 18:57:05.757745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.757801 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:07.075670 (+1317869us) ts_tablet_manager.cc:1145] Deleting tablet 6d1d58835ad24749a00ce27b4bc73c59
0811 18:57:07.075685 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.179233 (+103548us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:07.283770 (+104537us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.311210 (+1027440us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1317387}
I20260811 18:57:08.311821 4169 tablet_replica.cc:330] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:08.312068 4169 raft_consensus.cc:2229] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:08.312249 4169 pending_rounds.cc:62] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 2 pending ops.
I20260811 18:57:08.312335 4169 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400066380767232 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "6ef0a376939d4d57b0d95db644f25dd9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" new_extra_config { } }
I20260811 18:57:08.312858 4169 pending_rounds.cc:68] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400062060187648 op_type: NO_OP noop_request { }
I20260811 18:57:08.313041 4169 raft_consensus.cc:2866] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:08.313182 4169 raft_consensus.cc:2258] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:08.313868 4169 ts_tablet_manager.cc:1905] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:07.308692 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (DEFAULT_TABLE table=default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.314710 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.120746 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9409f5b4b0034412aedc183b302556a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:08.317207 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 4230
W20260811 18:57:08.317308 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9409f5b4b0034412aedc183b302556a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.097311 4331 raft_consensus.cc:686] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:07.310384 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 (DEFAULT_TABLE table=default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.311396 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4687b95728fc485fa0e842bc447eb83f (DEFAULT_TABLE table=default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.318305 3721 tablet_service.cc:1467] Processing CreateTablet for tablet b4c7f500a5444bae9a015a5e903c0622 (DEFAULT_TABLE table=default.Table_a594cb48b586458783a36cc76df1332c [id=7c0d12bf15b945c68177bfb057015870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.312368 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b4c7f500a5444bae9a015a5e903c0622 (DEFAULT_TABLE table=default.Table_a594cb48b586458783a36cc76df1332c [id=7c0d12bf15b945c68177bfb057015870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.318768 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6ec419ed7747c3b24dc11992518e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.318827 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c7f500a5444bae9a015a5e903c0622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.318662 4331 consensus_queue.cc:237] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.319042 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c7f500a5444bae9a015a5e903c0622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.319389 3717 tablet_service.cc:1467] Processing CreateTablet for tablet c08436f33db043b1bc6395cd92102e59 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.108460 4034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2852f9e5f67c4025836018af553388a5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59811bdf82234ef7ac5865dbecfcc388" is_pre_election: true
I20260811 18:57:08.319792 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08436f33db043b1bc6395cd92102e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.320065 4034 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:47987 (request call id 25) took 1213 ms. Trace:
I20260811 18:57:08.320333 4034 rpcz_store.cc:276] 0811 18:57:07.106986 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.108224 (+ 1238us) service_pool.cc:227] Handling call
0811 18:57:08.319978 (+1211754us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.321075 3712 tablet_service.cc:1467] Processing CreateTablet for tablet d2d7d986900244ddbc05dbe73a708dcf (DEFAULT_TABLE table=default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.321524 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d7d986900244ddbc05dbe73a708dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.313647 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9efd2f19d74a68a8536fd057377738 (DEFAULT_TABLE table=default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 [id=64009634513f46eeae6442cfadb7b0f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.322710 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9efd2f19d74a68a8536fd057377738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.322728 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (DEFAULT_TABLE table=default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.322808 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9efd2f19d74a68a8536fd057377738 (DEFAULT_TABLE table=default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8 [id=64009634513f46eeae6442cfadb7b0f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.323134 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.323493 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 867259e37fe44c33a35823ec2d896754 (DEFAULT_TABLE table=default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.323930 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867259e37fe44c33a35823ec2d896754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.314942 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 867259e37fe44c33a35823ec2d896754 (DEFAULT_TABLE table=default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.326895 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4687b95728fc485fa0e842bc447eb83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.326915 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 (DEFAULT_TABLE table=default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.327203 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 4687b95728fc485fa0e842bc447eb83f (DEFAULT_TABLE table=default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.327369 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02808605ee6c4634af47fc3fd57e9cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.327615 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4687b95728fc485fa0e842bc447eb83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.327608 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 867259e37fe44c33a35823ec2d896754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.328104 3723 tablet_service.cc:1467] Processing CreateTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.328660 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b199f4c9e14b468cb26cbd61fc8586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:07.161681 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:08.330719 3714 tablet_service.cc:1467] Processing CreateTablet for tablet a902ad70a25546be926ff8f7bf2ed64f (DEFAULT_TABLE table=default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.330926 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0190cbd859274fcfa09c6fab899c4843 deleted and replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:08.331113 3546 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 261) took 1865 ms. Trace:
I20260811 18:57:08.331205 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902ad70a25546be926ff8f7bf2ed64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.331261 3546 rpcz_store.cc:276] 0811 18:57:06.465290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.757148 (+291858us) service_pool.cc:227] Handling call
0811 18:57:08.331047 (+1573899us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.057800 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.057899 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.057905 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:07.057907 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:07.058094 (+ 187us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.058105 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.058110 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.058181 (+ 71us) tablet.cc:699] Row locks acquired
0811 18:57:07.058183 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.058214 (+ 31us) write_op.cc:270] Start()
0811 18:57:07.058242 (+ 28us) write_op.cc:276] Timestamp: P: 1786474627058172 usec, L: 0
0811 18:57:07.058245 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.058465 (+ 220us) log.cc:842] Serialized 622 byte log entry
0811 18:57:07.158160 (+ 99695us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.159887 (+ 1727us) write_op.cc:301] APPLY: starting
0811 18:57:07.159897 (+ 10us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.159931 (+ 34us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.159954 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.160059 (+ 105us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.160132 (+ 73us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.160139 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:07.160191 (+ 52us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.160228 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160242 (+ 14us) write_op.cc:454] Released schema lock
0811 18:57:07.160247 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.160291 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160295 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1696,"apply.run_cpu_time_us":448,"apply.run_wall_time_us":448,"num_ops":2,"prepare.queue_time_us":101718,"prepare.run_cpu_time_us":866,"prepare.run_wall_time_us":65551,"raft.queue_time_us":185,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":9,"replication_time_us":99899}]]}
I20260811 18:57:08.334065 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb0316f2e1db4353b9919edf3d8e1c90" } modify_external_catalogs: true
I20260811 18:57:08.334172 3856 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 191) took 1026 ms. Trace:
I20260811 18:57:07.107638 4162 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:08.334311 3856 rpcz_store.cc:276] 0811 18:57:07.307913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.308031 (+ 118us) service_pool.cc:227] Handling call
0811 18:57:08.314483 (+1006452us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.314512 (+ 29us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.317893 (+ 3381us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.333905 (+ 16012us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.334128 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.334537 4162 ts_tablet_manager.cc:1397] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 1.440s user 0.016s sys 0.005s
I20260811 18:57:08.334757 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f335e50ae57547ed80b144129faaffbb (DEFAULT_TABLE table=default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.335215 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f335e50ae57547ed80b144129faaffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.335461 4162 raft_consensus.cc:348] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.335708 4162 raft_consensus.cc:374] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.335801 4162 raft_consensus.cc:729] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.336017 4162 consensus_queue.cc:260] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:07.378022 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 734ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:07.316912 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 (DEFAULT_TABLE table=default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.337309 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02808605ee6c4634af47fc3fd57e9cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.338110 3860 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 195) took 1024 ms. Trace:
I20260811 18:57:07.162387 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:07.162844 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:08.338718 3711 tablet_service.cc:1467] Processing CreateTablet for tablet cbf6ce167fb348be8d61372791f0cbca (DEFAULT_TABLE table=default.Table_a671c5d0d75c49779b94baa2b89c9cef [id=dda7fb95dc1549ddbc88d8525ff023e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.338840 3551 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 265) took 1858 ms. Trace:
I20260811 18:57:08.339219 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbf6ce167fb348be8d61372791f0cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.338975 3551 rpcz_store.cc:276] 0811 18:57:06.479865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.764486 (+284621us) service_pool.cc:227] Handling call
0811 18:57:06.765358 (+ 872us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.765528 (+ 170us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:06.765598 (+ 70us) catalog_manager.cc:2761] Locking tablets
0811 18:57:06.960011 (+194413us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:07.162575 (+202564us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:07.162649 (+ 74us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:07.162708 (+ 59us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:08.338769 (+1176061us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.136544 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.136752 (+ 208us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.136761 (+ 9us) write_op.cc:435] Acquired schema lock
0811 18:57:07.136766 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.136988 (+ 222us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.137005 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.137014 (+ 9us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.137158 (+ 144us) tablet.cc:699] Row locks acquired
0811 18:57:07.137161 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:07.137214 (+ 53us) write_op.cc:270] Start()
0811 18:57:07.137262 (+ 48us) write_op.cc:276] Timestamp: P: 1786474627137159 usec, L: 0
0811 18:57:07.137268 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.137591 (+ 323us) log.cc:842] Serialized 756 byte log entry
0811 18:57:07.158710 (+ 21119us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.160917 (+ 2207us) write_op.cc:301] APPLY: starting
0811 18:57:07.160932 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.160965 (+ 33us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.160989 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.161054 (+ 65us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.161092 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:07.161098 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.161153 (+ 55us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.161190 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.161208 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:07.161213 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.161260 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.161264 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2148,"apply.run_cpu_time_us":400,"apply.run_wall_time_us":401,"num_ops":2,"prepare.queue_time_us":175744,"prepare.run_cpu_time_us":1321,"prepare.run_wall_time_us":33947,"raft.queue_time_us":211,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":210,"replication_time_us":21427}]]}
I20260811 18:57:08.339929 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.340549 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e85b84e5b0e45bf80321dfb55c1704d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.165973 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dcfccd3951dc4875a1d70530eb514fb2" } new_table_name: "default.Table_53cd8000542843c984903cd25e451bdf" modify_external_catalogs: true
I20260811 18:57:07.138746 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 138) took 1478 ms. Trace:
W20260811 18:57:07.234654 3678 consensus_peers.cc:487] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:07.320669 3865 tablet_service.cc:1467] Processing CreateTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.342517 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 1.10801e+06us
I20260811 18:57:07.332844 3858 tablet_service.cc:1467] Processing CreateTablet for tablet a902ad70a25546be926ff8f7bf2ed64f (DEFAULT_TABLE table=default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.342775 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 65234acc3d5a462683edbe8f223eeefc (DEFAULT_TABLE table=default.Table_4db75b493caf48c0be36b2f872c70dd8 [id=c7f1c01155c044af8ffdf358de715e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.345032 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f335e50ae57547ed80b144129faaffbb (DEFAULT_TABLE table=default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.436466 4446 consensus_queue.cc:1035] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:07.168037 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ace00158b744bd8a97dc1c47fc6049c consensus state has no leader
I20260811 18:57:08.343235 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65234acc3d5a462683edbe8f223eeefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.343348 3548 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 275) took 1425 ms. Trace:
I20260811 18:57:07.347676 3861 tablet_service.cc:1467] Processing CreateTablet for tablet cbf6ce167fb348be8d61372791f0cbca (DEFAULT_TABLE table=default.Table_a671c5d0d75c49779b94baa2b89c9cef [id=dda7fb95dc1549ddbc88d8525ff023e8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:07.550316 3664 kernel_stack_watchdog.cc:198] Thread 3678 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 153ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fcdd5c86980 (unknown) at ??:0
@ 0x7fcdd5c809c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fcdcfd7769c google::LogMessage::Flush() at ??:0
@ 0x7fcdcfd776d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fcddddbbb37 kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7fcddddbac46 kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7fcddddc0ff1 kudu::consensus::Peer::SendNextRequest()::$_3::operator()() at ??:0
@ 0x7fcddddc0dfe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9426f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fcdd9426b6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9412dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fcdd9413321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fcdd93b0aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fcdd93af5a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:08.343506 3548 rpcz_store.cc:276] 0811 18:57:06.917826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.944831 (+ 27005us) service_pool.cc:227] Handling call
0811 18:57:06.945521 (+ 690us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.945869 (+ 348us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.946137 (+ 268us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.946245 (+ 108us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.162974 (+216729us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.343303 (+1180329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.056563 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.056703 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.056710 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:07.056713 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:07.056815 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.056827 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.056831 (+ 4us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.056892 (+ 61us) tablet.cc:699] Row locks acquired
0811 18:57:07.056894 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.056997 (+ 103us) write_op.cc:270] Start()
0811 18:57:07.057046 (+ 49us) write_op.cc:276] Timestamp: P: 1786474627056952 usec, L: 0
0811 18:57:07.057050 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.057251 (+ 201us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.158069 (+100818us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.159054 (+ 985us) write_op.cc:301] APPLY: starting
0811 18:57:07.159077 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.159159 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.159213 (+ 54us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.159280 (+ 67us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.159335 (+ 55us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.159351 (+ 16us) write_op.cc:312] APPLY: finished
0811 18:57:07.159533 (+ 182us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.159648 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.159684 (+ 36us) write_op.cc:454] Released schema lock
0811 18:57:07.159701 (+ 17us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.159798 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.159802 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":911,"apply.run_cpu_time_us":811,"apply.run_wall_time_us":811,"num_ops":1,"prepare.queue_time_us":109584,"prepare.run_cpu_time_us":795,"prepare.run_wall_time_us":1222,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":100982}]]}
I20260811 18:57:07.360954 3854 tablet_service.cc:1467] Processing CreateTablet for tablet c08436f33db043b1bc6395cd92102e59 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.344475 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet d106e0e1f4b845429b10f4e100f69a8b from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:07.647948 4476 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:08.344962 3548 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 284) took 1310 ms. Trace:
I20260811 18:57:08.344805 4476 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.345063 3548 rpcz_store.cc:276] 0811 18:57:07.034425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.344406 (+1309981us) service_pool.cc:227] Handling call
0811 18:57:08.344903 (+ 497us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.345502 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_60c6f6ccfa2747e1b4fff22a5779d63e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:08.345674 4476 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:07.362970 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 65234acc3d5a462683edbe8f223eeefc (DEFAULT_TABLE table=default.Table_4db75b493caf48c0be36b2f872c70dd8 [id=c7f1c01155c044af8ffdf358de715e3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.439107 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 490f34d7a403416fbab64e46973746de (DEFAULT_TABLE table=default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.346163 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60c6f6ccfa2747e1b4fff22a5779d63e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:07.441349 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:07.186271 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dde27d8cc3eb4d09879e717fc90305fd
W20260811 18:57:07.730131 3824 consensus_peers.cc:487] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:08.326920 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 (DEFAULT_TABLE table=default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.346803 3553 catalog_manager.cc:6565] ReplaceTablet: tablet dde27d8cc3eb4d09879e717fc90305fd deleted and replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5
I20260811 18:57:08.346940 3553 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 271) took 1697 ms. Trace:
I20260811 18:57:08.347044 3553 rpcz_store.cc:276] 0811 18:57:06.649596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.879512 (+229916us) service_pool.cc:227] Handling call
0811 18:57:08.346886 (+1467374us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.170540 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.170979 (+ 439us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.170991 (+ 12us) write_op.cc:435] Acquired schema lock
0811 18:57:07.170996 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.171211 (+ 215us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.171236 (+ 25us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.171244 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.171379 (+ 135us) tablet.cc:699] Row locks acquired
0811 18:57:07.171383 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:07.171441 (+ 58us) write_op.cc:270] Start()
0811 18:57:07.171498 (+ 57us) write_op.cc:276] Timestamp: P: 1786474627171382 usec, L: 0
0811 18:57:07.171505 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.171834 (+ 329us) log.cc:842] Serialized 622 byte log entry
0811 18:57:07.184823 (+ 12989us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.185006 (+ 183us) write_op.cc:301] APPLY: starting
0811 18:57:07.185023 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.185070 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.185113 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.185230 (+ 117us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.185304 (+ 74us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.185310 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.185459 (+ 149us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.185565 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.185599 (+ 34us) write_op.cc:454] Released schema lock
0811 18:57:07.185611 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.185702 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.185708 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":133,"apply.run_cpu_time_us":777,"apply.run_wall_time_us":775,"num_ops":2,"prepare.queue_time_us":203155,"prepare.run_cpu_time_us":1576,"prepare.run_wall_time_us":20180,"raft.queue_time_us":265,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":435,"replication_time_us":13288}]]}
W20260811 18:57:07.236001 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61c8443df21949ff9bdc63fa6aa34108 consensus state has no leader
I20260811 18:57:07.251451 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_542b2f198c2a42abb286bbdcf948584b" } modify_external_catalogs: true
I20260811 18:57:08.350589 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet ed3346bd12d04c45a511308c427856e5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:08.350626 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_542b2f198c2a42abb286bbdcf948584b" } modify_external_catalogs: true
W20260811 18:57:07.571264 3528 kernel_stack_watchdog.cc:198] Thread 3541 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 158ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f61e8213980 (unknown) at ??:0
@ 0x7f61e118d135 __write at ??:0
@ 0x7f61e110815d _IO_file_write at ??:0
@ 0x7f61e1108acf _IO_file_xsputn at ??:0
@ 0x7f61e10fc927 fwrite at ??:0
@ 0x80f130 __interceptor_fwrite.part.57 at ??:?
@ 0x7f61e2307661 google::LogMessage::SendToLog() at ??:0
@ 0x7f61e23044de google::LogMessage::Flush() at ??:0
@ 0x7f61e23046d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f61f7e30bab kudu::master::AsyncDeleteReplica::HandleResponse() at ??:0
@ 0x7f61f7d899a7 kudu::master::RetryingTSRpcTask::RpcCallback() at ??:0
@ 0x7f61f7e33540 _ZZN4kudu6master18AsyncDeleteReplica11SendRequestEiENKUlvE_clEv at ??:0
@ 0x7f61f7e3335e _ZNSt17_Function_handlerIFvvEZN4kudu6master18AsyncDeleteReplica11SendRequestEiEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61eb9b3f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f61eb9b3b6e std::_Function_handler<>::_M_invoke() at ??:0
W20260811 18:57:07.827332 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7f640c60e81b4a369458b1497a269c9c consensus state has no leader
W20260811 18:57:07.209591 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f335e50ae57547ed80b144129faaffbb has no consensus state
I20260811 18:57:08.352555 3547 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 276) took 1373 ms. Trace:
I20260811 18:57:08.352669 3547 rpcz_store.cc:276] 0811 18:57:06.978633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.978749 (+ 116us) service_pool.cc:227] Handling call
0811 18:57:06.979464 (+ 715us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.980773 (+ 1309us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.980995 (+ 222us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.981114 (+ 119us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.209119 (+228005us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.352520 (+1143401us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.190743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.190866 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.190872 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:07.190874 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:07.190982 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.190996 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.191001 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.191092 (+ 91us) tablet.cc:699] Row locks acquired
0811 18:57:07.191094 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:07.191126 (+ 32us) write_op.cc:270] Start()
0811 18:57:07.191159 (+ 33us) write_op.cc:276] Timestamp: P: 1786474627191075 usec, L: 0
0811 18:57:07.191162 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.191361 (+ 199us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.208088 (+ 16727us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.208315 (+ 227us) write_op.cc:301] APPLY: starting
0811 18:57:07.208340 (+ 25us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.208408 (+ 68us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.208492 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.208550 (+ 58us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.208592 (+ 42us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.208599 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:07.208696 (+ 97us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.208763 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.208786 (+ 23us) write_op.cc:454] Released schema lock
0811 18:57:07.208794 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.208852 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.208856 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":145,"apply.run_cpu_time_us":611,"apply.run_wall_time_us":610,"num_ops":1,"prepare.queue_time_us":208810,"prepare.run_cpu_time_us":877,"prepare.run_wall_time_us":15273,"raft.queue_time_us":254,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":454,"replication_time_us":16885}]]}
I20260811 18:57:08.353579 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0cb87feb6db445d792ba8021cbe75510"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.354187 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cb87feb6db445d792ba8021cbe75510 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.354415 3555 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 277) took 1367 ms. Trace:
I20260811 18:57:08.354499 3555 rpcz_store.cc:276] 0811 18:57:06.982626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.983161 (+ 535us) service_pool.cc:227] Handling call
0811 18:57:06.986104 (+ 2943us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.986560 (+ 456us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:06.986817 (+ 257us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:06.986903 (+ 86us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:07.235670 (+248767us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.350522 (+1114852us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.206108 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.206280 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.206290 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:07.206295 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:07.216559 (+ 10264us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.216586 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.216597 (+ 11us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:07.216713 (+ 116us) tablet.cc:699] Row locks acquired
0811 18:57:07.216718 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:07.216791 (+ 73us) write_op.cc:270] Start()
0811 18:57:07.216854 (+ 63us) write_op.cc:276] Timestamp: P: 1786474627216767 usec, L: 0
0811 18:57:07.216862 (+ 8us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.217223 (+ 361us) log.cc:842] Serialized 405 byte log entry
0811 18:57:07.234625 (+ 17402us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.234827 (+ 202us) write_op.cc:301] APPLY: starting
0811 18:57:07.234852 (+ 25us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.234919 (+ 67us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.234966 (+ 47us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.235055 (+ 89us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.235127 (+ 72us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:07.235139 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:07.235267 (+ 128us) log.cc:842] Serialized 23 byte log entry
0811 18:57:07.235362 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.235389 (+ 27us) write_op.cc:454] Released schema lock
0811 18:57:07.235401 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.235481 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.235487 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":736,"apply.run_wall_time_us":736,"num_ops":1,"prepare.queue_time_us":197670,"prepare.run_cpu_time_us":1380,"prepare.run_wall_time_us":14302,"raft.queue_time_us":204,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":4964,"replication_time_us":17720}]]}
I20260811 18:57:08.355379 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_335e9fdd802644cc93a0657a33e74605"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.355978 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_335e9fdd802644cc93a0657a33e74605 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:08.346657 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 616658us
I20260811 18:57:08.349364 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:08.338835 3554 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 264) took 1862 ms. Trace:
I20260811 18:57:08.357090 3554 rpcz_store.cc:276] 0811 18:57:06.476032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.762199 (+286167us) service_pool.cc:227] Handling call
0811 18:57:06.762957 (+ 758us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:06.763367 (+ 410us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:06.763429 (+ 62us) catalog_manager.cc:2761] Locking tablets
0811 18:57:06.958946 (+195517us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:07.161881 (+202935us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:07.161962 (+ 81us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:07.162020 (+ 58us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:08.338769 (+1176749us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:07.123394 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:07.123550 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:07.123557 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:07.123560 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:07.123723 (+ 163us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:07.123737 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:07.123743 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:07.123998 (+ 255us) tablet.cc:699] Row locks acquired
0811 18:57:07.124013 (+ 15us) write_op.cc:260] PREPARE: finished
0811 18:57:07.124106 (+ 93us) write_op.cc:270] Start()
0811 18:57:07.124159 (+ 53us) write_op.cc:276] Timestamp: P: 1786474627124043 usec, L: 0
0811 18:57:07.124164 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:07.124401 (+ 237us) log.cc:842] Serialized 725 byte log entry
0811 18:57:07.158637 (+ 34236us) op_driver.cc:444] REPLICATION: finished
0811 18:57:07.160361 (+ 1724us) write_op.cc:301] APPLY: starting
0811 18:57:07.160370 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:07.160396 (+ 26us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:07.160422 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:07.160462 (+ 40us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:07.160485 (+ 23us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:07.160491 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:07.160531 (+ 40us) log.cc:842] Serialized 29 byte log entry
0811 18:57:07.160560 (+ 29us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160570 (+ 10us) write_op.cc:454] Released schema lock
0811 18:57:07.160574 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:07.160610 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:07.160614 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":896,"child_traces":[["op",{"apply.queue_time_us":1680,"apply.run_cpu_time_us":287,"apply.run_wall_time_us":287,"num_ops":2,"prepare.queue_time_us":163713,"prepare.run_cpu_time_us":1249,"prepare.run_wall_time_us":13097,"raft.queue_time_us":2180,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":34447}]]}
I20260811 18:57:08.357918 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_220d78aeb25b4b64b594e0839928c35e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.358516 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_220d78aeb25b4b64b594e0839928c35e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.358907 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6d1d58835ad24749a00ce27b4bc73c59 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:08.346876 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b6ec419ed7747c3b24dc11992518e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.360396 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9efd2f19d74a68a8536fd057377738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.346797 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 490f34d7a403416fbab64e46973746de (DEFAULT_TABLE table=default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.342445 4167 rpcz_store.cc:276] 0811 18:57:05.655787 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656164 (+ 377us) service_pool.cc:227] Handling call
0811 18:57:07.134055 (+1477891us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:07.134090 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:07.134288 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1475939}
I20260811 18:57:08.361593 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490f34d7a403416fbab64e46973746de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.347358 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (DEFAULT_TABLE table=default.Table_57bac5d4e666472fbab036524cbbed7d [id=41ead11057b94707a815fa4e3e2e5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.347918 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb (DEFAULT_TABLE table=default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.361821 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 136) took 2706 ms. Trace:
W20260811 18:57:08.271448 3811 kernel_stack_watchdog.cc:198] Thread 3824 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 212ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f4d0c40b980 (unknown) at ??:0
@ 0x7f4d0c4059c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f4d064fc69c google::LogMessage::Flush() at ??:0
@ 0x7f4d064fc6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f4d14540b37 kudu::consensus::Peer::ProcessResponseErrorUnlocked() at ??:0
@ 0x7f4d1453fc46 kudu::consensus::Peer::ProcessResponse() at ??:0
@ 0x7f4d14545ff1 kudu::consensus::Peer::SendNextRequest()::$_3::operator()() at ??:0
@ 0x7f4d14545dfe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fbabf86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f4d0fbabb6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fb97dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f4d0fb98321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f4d0fb35aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f4d0fb345a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:08.377089 3857 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 192) took 1067 ms. Trace:
I20260811 18:57:08.378234 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3ec38f9b55489ba00ed76f9e4456f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.382746 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515d5b5afbd74ec5ab90a503aef2f4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.380399 4167 rpcz_store.cc:276] 0811 18:57:05.655254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.656094 (+ 840us) service_pool.cc:227] Handling call
0811 18:57:08.361494 (+2705400us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:08.361509 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.361651 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2703201}
I20260811 18:57:08.334311 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb0316f2e1db4353b9919edf3d8e1c90" } modify_external_catalogs: true
I20260811 18:57:08.386706 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 135) took 2729 ms. Trace:
I20260811 18:57:08.386874 4167 rpcz_store.cc:276] 0811 18:57:05.654691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.663221 (+ 8530us) service_pool.cc:227] Handling call
0811 18:57:08.383916 (+2720695us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:08.383930 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.384097 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2720437}
I20260811 18:57:08.393983 3998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55448 (request call id 188) took 1286 ms. Trace:
W20260811 18:57:08.395701 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
W20260811 18:57:08.396023 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e7259cf84f5c4c89a1b4510f743c2bc5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7259cf84f5c4c89a1b4510f743c2bc5 already in progress: creating tablet
I20260811 18:57:08.417135 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.417614 4167 tablet_replica.cc:330] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:08.417922 4167 raft_consensus.cc:2229] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:08.418659 4167 pending_rounds.cc:62] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:08.394320 3998 rpcz_store.cc:276] 0811 18:57:07.107292 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.107375 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.311103 (+1203728us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.311133 (+ 30us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.379441 (+ 68308us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.393643 (+ 14202us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.393892 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.446990 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.460008 3867 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 194) took 1147 ms. Trace:
I20260811 18:57:08.446633 4167 pending_rounds.cc:68] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400069692174336 op_type: NO_OP noop_request { }
I20260811 18:57:08.467969 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:08.476346 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.476501 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:08.479525 4167 raft_consensus.cc:2866] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:08.462198 3871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 193) took 1151 ms. Trace:
I20260811 18:57:08.482314 4167 raft_consensus.cc:2258] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:08.482754 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e51a08e6dbf04e3fa58689ee0cd45db3 consensus state has no leader
I20260811 18:57:08.483183 3549 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 280) took 1480 ms. Trace:
I20260811 18:57:08.484136 4167 ts_tablet_manager.cc:1905] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:08.483435 3549 rpcz_store.cc:276] 0811 18:57:07.002091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.165716 (+163625us) service_pool.cc:227] Handling call
0811 18:57:08.342045 (+1176329us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.342307 (+ 262us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:08.342506 (+ 199us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:08.342591 (+ 85us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:08.482307 (+139716us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:08.482947 (+ 640us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.390790 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.390944 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.390952 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:08.390954 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:08.391104 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.391119 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.391126 (+ 7us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:08.391200 (+ 74us) tablet.cc:699] Row locks acquired
0811 18:57:08.391202 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:08.391255 (+ 53us) write_op.cc:270] Start()
0811 18:57:08.391297 (+ 42us) write_op.cc:276] Timestamp: P: 1786474628391129 usec, L: 0
0811 18:57:08.391301 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.391545 (+ 244us) log.cc:842] Serialized 405 byte log entry
0811 18:57:08.451373 (+ 59828us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.478407 (+ 27034us) write_op.cc:301] APPLY: starting
0811 18:57:08.478437 (+ 30us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.478518 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.478689 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.478847 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.478900 (+ 53us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:08.478920 (+ 20us) write_op.cc:312] APPLY: finished
0811 18:57:08.479095 (+ 175us) log.cc:842] Serialized 23 byte log entry
0811 18:57:08.481601 (+ 2506us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.481632 (+ 31us) write_op.cc:454] Released schema lock
0811 18:57:08.481643 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":26897,"num_ops":1,"prepare.queue_time_us":47689,"prepare.run_cpu_time_us":920,"prepare.run_wall_time_us":18187,"replication_time_us":60034,"thread_start_us":85027,"threads_started":2}]]}
I20260811 18:57:08.499396 3550 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 279) took 1516 ms. Trace:
I20260811 18:57:08.504637 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b199f4c9e14b468cb26cbd61fc8586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.505115 3872 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 196) took 1190 ms. Trace:
I20260811 18:57:08.567466 4162 ts_tablet_manager.cc:1428] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.224s user 0.003s sys 0.016s
I20260811 18:57:08.577222 4162 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:08.799654 4503 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:08.949668 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap starting.
W20260811 18:57:08.903290 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 290ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:08.668385 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e902efbb774244d18e3ba165bfb7c695 (DEFAULT_TABLE table=default.Table_60c6f6ccfa2747e1b4fff22a5779d63e [id=11c6327a2ff84680bcd60893f05b3493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.668962 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c6852aaef1d5486b87e5462eae532719 (DEFAULT_TABLE table=default.Table_9409f5b4b0034412aedc183b302556a3 [id=75ea77c989e746f88a94e72de6cd11ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.673111 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 71df3738a5cf4b12820dda131ced62e1 (DEFAULT_TABLE table=default.Table_220d78aeb25b4b64b594e0839928c35e [id=b9face92d47f453ab5de5af21691edb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:08.674369 4506 log.cc:924] Time spent T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260811 18:57:09.015095 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e902efbb774244d18e3ba165bfb7c695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.676241 3714 tablet_service.cc:1467] Processing CreateTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 (DEFAULT_TABLE table=default.Table_9d18dc31cde54577b8bcc329191b74ef [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.016002 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2718cd4a98742dfa8014a8a38c69c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.679345 3709 tablet_service.cc:1467] Processing CreateTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f (DEFAULT_TABLE table=default.Table_335e9fdd802644cc93a0657a33e74605 [id=14b4d412d33c43fdb3d20c85b82c1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.018792 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe203387bc9c45ba82afe6b058aa5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.685612 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 (DEFAULT_TABLE table=default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.023077 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43ad6993b63e4bbd98c8cc52f1a53451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.698483 4162 tablet_bootstrap.cc:654] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.726984 4167 ts_tablet_manager.cc:1918] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:08.851405 4476 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:09.027940 4167 log.cc:1196] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6d1d58835ad24749a00ce27b4bc73c59
I20260811 18:57:09.028386 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6852aaef1d5486b87e5462eae532719. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:08.905362 3388 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.125:38889 timed out after 0.587s (SENT) [suppressed 8 similar messages]
I20260811 18:57:09.268386 4503 log.cc:824] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:09.692754 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=14 ignored=0} inserts{seen=23 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 3 replicates
W20260811 18:57:09.496275 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 2429ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:08.499670 3550 rpcz_store.cc:276] 0811 18:57:06.983097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.120052 (+136955us) service_pool.cc:227] Handling call
0811 18:57:08.317424 (+1197372us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.317911 (+ 487us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.498796 (+180885us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.498964 (+ 168us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.499116 (+ 152us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.409061 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.409239 (+ 178us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.409250 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:08.409254 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.409393 (+ 139us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.409411 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.409419 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.409585 (+ 166us) tablet.cc:699] Row locks acquired
0811 18:57:08.409587 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:08.409639 (+ 52us) write_op.cc:270] Start()
0811 18:57:08.409689 (+ 50us) write_op.cc:276] Timestamp: P: 1786474628409512 usec, L: 0
0811 18:57:08.409695 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.409971 (+ 276us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.479801 (+ 69830us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.497808 (+ 18007us) write_op.cc:301] APPLY: starting
0811 18:57:08.497829 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.497880 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.497911 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.498055 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.498095 (+ 40us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.498105 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:08.498242 (+ 137us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.498642 (+ 400us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.498668 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:08.498679 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":17949,"num_ops":2,"prepare.queue_time_us":57171,"prepare.run_cpu_time_us":1043,"prepare.run_wall_time_us":9455,"replication_time_us":70076,"threads_started":1}]]}
I20260811 18:57:09.711858 4503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: Bootstrap complete.
W20260811 18:57:09.712406 3550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 33) took 1124 ms (client timeout 1000 ms). Trace:
W20260811 18:57:09.712558 3550 rpcz_store.cc:269] 0811 18:57:08.588416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.712099 (+1123683us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:09.712333 (+ 234us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:09.713227 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_919f7bdbd76f4707813e30d37dfc3767"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:09.221671 3388 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 316178us
I20260811 18:57:08.507503 3860 rpcz_store.cc:276] 0811 18:57:07.313181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.313263 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:08.322476 (+1009213us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.322502 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.334444 (+ 11942us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.337888 (+ 3444us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.338057 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.510281 4163 tablet_bootstrap.cc:654] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.511628 3857 rpcz_store.cc:276] 0811 18:57:07.309736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.309857 (+ 121us) service_pool.cc:227] Handling call
0811 18:57:08.318522 (+1008665us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318551 (+ 29us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.326064 (+ 7513us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.376822 (+ 50758us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.377039 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.513374 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65234acc3d5a462683edbe8f223eeefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.521647 3893 raft_consensus.cc:2454] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:09.779363 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490f34d7a403416fbab64e46973746de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.779507 3893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.65:36347 (request call id 44) took 1431 ms. Trace:
I20260811 18:57:09.779712 3893 rpcz_store.cc:276] 0811 18:57:08.348367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.349231 (+ 864us) service_pool.cc:227] Handling call
0811 18:57:09.779450 (+1430219us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.521890 3871 rpcz_store.cc:276] 0811 18:57:07.310816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.310920 (+ 104us) service_pool.cc:227] Handling call
0811 18:57:08.318585 (+1007665us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318611 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.333715 (+ 15104us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.461850 (+128135us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.462042 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.513533 3872 rpcz_store.cc:276] 0811 18:57:07.314387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.314466 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:08.326530 (+1012064us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.326587 (+ 57us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.439374 (+112787us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.504845 (+ 65471us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.505031 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.873997 4503 raft_consensus.cc:348] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:57:09.809768 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3255ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:08.523305 3867 rpcz_store.cc:276] 0811 18:57:07.311920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.312003 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.318827 (+1006824us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.318851 (+ 24us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.338642 (+ 19791us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.458897 (+120255us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.459477 (+ 580us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.529765 3865 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 198) took 1209 ms. Trace:
I20260811 18:57:10.856699 3865 rpcz_store.cc:276] 0811 18:57:07.320102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.320200 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.342649 (+1022449us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.342674 (+ 25us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.518755 (+176081us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.529494 (+ 10739us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.529675 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.788858 3370 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: OK, instance_id { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" instance_seqno: 1786474627171025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:10.857882 3370 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:09.028261 4476 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:10.858765 4476 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:09.714048 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_919f7bdbd76f4707813e30d37dfc3767 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.635121 3940 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:08.810937 3870 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 197) took 1494 ms. Trace:
I20260811 18:57:10.861056 3870 rpcz_store.cc:276] 0811 18:57:07.316415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.316490 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:08.336907 (+1020417us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.336939 (+ 32us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:08.433783 (+ 96844us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.810532 (+376749us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:08.810837 (+ 305us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.510810 3824 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:09.773559 4565 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:10.861858 4565 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:09.783396 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f335e50ae57547ed80b144129faaffbb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:10.364069 3388 leader_election.cc:334] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889): Timed out: RequestConsensusVote RPC to 127.3.72.125:38889 timed out after 0.587s (SENT)
I20260811 18:57:10.862694 3388 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:09.005527 3707 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1923ce0049448fe4e28f2dd258dc (DEFAULT_TABLE table=default.Table_0cb87feb6db445d792ba8021cbe75510 [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.557698 3548 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 285) took 1491 ms. Trace:
I20260811 18:57:10.863961 3548 rpcz_store.cc:276] 0811 18:57:07.066107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.345251 (+1279144us) service_pool.cc:227] Handling call
0811 18:57:08.346290 (+ 1039us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.367395 (+ 21105us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.557224 (+189829us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.557377 (+ 153us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.557513 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.424337 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.424527 (+ 190us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.424537 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:08.424542 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:08.424683 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.424698 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.424706 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.424827 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:08.424831 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.424882 (+ 51us) write_op.cc:270] Start()
0811 18:57:08.424927 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628424753 usec, L: 0
0811 18:57:08.424933 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.429693 (+ 4760us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.555947 (+126254us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.556034 (+ 87us) write_op.cc:301] APPLY: starting
0811 18:57:08.556048 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.556110 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.556189 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.556521 (+ 332us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.556573 (+ 52us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.556581 (+ 8us) write_op.cc:312] APPLY: finished
0811 18:57:08.556743 (+ 162us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.557079 (+ 336us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.557107 (+ 28us) write_op.cc:454] Released schema lock
0811 18:57:08.557116 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.557904 (+ 788us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.557909 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":38,"apply.run_cpu_time_us":879,"apply.run_wall_time_us":1950,"num_ops":2,"prepare.queue_time_us":56294,"prepare.run_cpu_time_us":1125,"prepare.run_wall_time_us":6517,"replication_time_us":130994}]]}
W20260811 18:57:10.864952 3548 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.66:35625 (request call id 30) took 1080 ms (client timeout 1000 ms). Trace:
W20260811 18:57:10.865061 3548 rpcz_store.cc:269] 0811 18:57:09.784892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:10.864839 (+1079947us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:10.864893 (+ 54us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:10.865523 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_79396737f1a343c98184686b7eda44ab" } new_table_name: "default.Table_45c439603b1140e998b5d9199026da49" modify_external_catalogs: true
I20260811 18:57:09.827950 4163 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:10.887220 4503 raft_consensus.cc:729] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85eb97df7824bc9bad9dca349b0c180, State: Initialized, Role: FOLLOWER
W20260811 18:57:11.039822 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1284ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:08.559392 3551 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 283) took 1525 ms. Trace:
I20260811 18:57:11.321699 4503 consensus_queue.cc:260] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 1.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:11.321344 3551 rpcz_store.cc:276] 0811 18:57:07.034050 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.339680 (+1305630us) service_pool.cc:227] Handling call
0811 18:57:08.340674 (+ 994us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.341277 (+ 603us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.559015 (+217738us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.559137 (+ 122us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.559236 (+ 99us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.430915 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.431089 (+ 174us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.431098 (+ 9us) write_op.cc:435] Acquired schema lock
0811 18:57:08.431103 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:08.431224 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.431238 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.431246 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.431355 (+ 109us) tablet.cc:699] Row locks acquired
0811 18:57:08.431359 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.431407 (+ 48us) write_op.cc:270] Start()
0811 18:57:08.431451 (+ 44us) write_op.cc:276] Timestamp: P: 1786474628431277 usec, L: 0
0811 18:57:08.431457 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.431744 (+ 287us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.558098 (+126354us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.558196 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:08.558210 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.558258 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.558280 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.558494 (+ 214us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.558529 (+ 35us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.558536 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.558622 (+ 86us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.558885 (+ 263us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.558903 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:08.558911 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.559830 (+ 919us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.559837 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42,"apply.run_cpu_time_us":660,"apply.run_wall_time_us":1692,"num_ops":2,"prepare.queue_time_us":61989,"prepare.run_cpu_time_us":989,"prepare.run_wall_time_us":14055,"replication_time_us":126611}]]}
I20260811 18:57:11.328462 4503 sys_catalog.cc:564] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:10.866535 4163 ts_tablet_manager.cc:1397] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 2.561s user 0.005s sys 0.012s
I20260811 18:57:11.322451 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet f61d3822fc2d4b569f223238a6464e0a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:10.877125 4593 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:11.370034 4163 raft_consensus.cc:348] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:10.005074 3811 kernel_stack_watchdog.cc:198] Thread 3824 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 1009ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f4d0c40b980 (unknown) at ??:0
@ 0x7f4d0c4059c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7f4d064fc69c google::LogMessage::Flush() at ??:0
@ 0x7f4d064fc6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f4d145a79f7 kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7f4d145a8ce1 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7f4d145ab73a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7f4d145ab4fe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fbabf86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7f4d0fbabb6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4d0fb97dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f4d0fb98321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f4d0fb35aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f4d0fb345a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:09.029347 4167 ts_tablet_manager.cc:1939] T 6d1d58835ad24749a00ce27b4bc73c59 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:10.877368 3388 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 513282us
I20260811 18:57:08.574699 3554 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 289) took 1476 ms. Trace:
I20260811 18:57:11.370949 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 142) took 5599 ms. Trace:
W20260811 18:57:10.889950 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 2.5329e+06us
I20260811 18:57:11.371064 4167 rpcz_store.cc:276] 0811 18:57:05.771325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.771685 (+ 360us) service_pool.cc:227] Handling call
0811 18:57:08.387238 (+2615553us) ts_tablet_manager.cc:1145] Deleting tablet 6d1d58835ad24749a00ce27b4bc73c59
0811 18:57:08.387252 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.627570 (+240318us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:08.726739 (+ 99169us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.370883 (+2644144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2615102}
I20260811 18:57:10.037185 4572 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.371551 4167 tablet_replica.cc:330] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:11.328495 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:11.371536 4572 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:11.372249 4438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43854 (request call id 1) took 1596 ms (client timeout 587 ms). Trace:
I20260811 18:57:11.372567 4572 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:10.052956 4573 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:11.372822 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 71df3738a5cf4b12820dda131ced62e1 (DEFAULT_TABLE table=default.Table_220d78aeb25b4b64b594e0839928c35e [id=b9face92d47f453ab5de5af21691edb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:11.548520 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3550ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:11.370733 3554 rpcz_store.cc:276] 0811 18:57:07.098547 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.357708 (+1259161us) service_pool.cc:227] Handling call
0811 18:57:08.358620 (+ 912us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.373671 (+ 15051us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.374122 (+ 451us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.574240 (+200118us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.574514 (+ 274us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.574618 (+ 104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.448721 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.448837 (+ 116us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.448844 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:08.448848 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.448949 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.448961 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.448968 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.449058 (+ 90us) tablet.cc:699] Row locks acquired
0811 18:57:08.449062 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.449106 (+ 44us) write_op.cc:270] Start()
0811 18:57:08.449137 (+ 31us) write_op.cc:276] Timestamp: P: 1786474628448980 usec, L: 0
0811 18:57:08.449143 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.449313 (+ 170us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.573603 (+124290us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.573663 (+ 60us) write_op.cc:301] APPLY: starting
0811 18:57:08.573672 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.573710 (+ 38us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.573740 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.573835 (+ 95us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.573869 (+ 34us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.573878 (+ 9us) write_op.cc:312] APPLY: finished
0811 18:57:08.573948 (+ 70us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.574151 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.574164 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:08.574170 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.575573 (+ 1403us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.575579 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27846912,"child_traces":[["op",{"apply.queue_time_us":28,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":1954,"num_ops":2,"prepare.queue_time_us":74175,"prepare.run_cpu_time_us":729,"prepare.run_wall_time_us":38374,"replication_time_us":124445}]]}
I20260811 18:57:11.651948 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a82335ab20e412c93a5ac0938a96012"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:09.030548 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71df3738a5cf4b12820dda131ced62e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:11.652922 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a82335ab20e412c93a5ac0938a96012 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.372884 4573 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.653365 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1923ce0049448fe4e28f2dd258dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:11.372421 4438 rpcz_store.cc:269] 0811 18:57:09.776212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.776292 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:11.372178 (+1595886us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:09.836782 3852 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 204) took 2398 ms. Trace:
I20260811 18:57:11.334926 4441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 280 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
W20260811 18:57:11.654244 4441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43854 (request call id 0) took 2984 ms (client timeout 587 ms). Trace:
W20260811 18:57:11.654470 4441 rpcz_store.cc:269] 0811 18:57:08.669507 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.669568 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:11.654198 (+2984630us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:08.574985 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ed3346bd12d04c45a511308c427856e5
I20260811 18:57:10.873214 3864 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 200) took 3528 ms. Trace:
I20260811 18:57:10.879350 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a902ad70a25546be926ff8f7bf2ed64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.655200 3553 catalog_manager.cc:6565] ReplaceTablet: tablet ed3346bd12d04c45a511308c427856e5 deleted and replaced by tablet b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:11.655349 3553 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 286) took 4557 ms. Trace:
I20260811 18:57:10.879983 3853 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 203) took 3517 ms. Trace:
I20260811 18:57:10.889523 4592 raft_consensus.cc:2781] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:10.891278 4169 ts_tablet_manager.cc:1918] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:11.373798 3871 tablet_service.cc:1467] Processing CreateTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f (DEFAULT_TABLE table=default.Table_335e9fdd802644cc93a0657a33e74605 [id=14b4d412d33c43fdb3d20c85b82c1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.655467 3553 rpcz_store.cc:276] 0811 18:57:07.097824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.350461 (+1252637us) service_pool.cc:227] Handling call
0811 18:57:08.371038 (+ 20577us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7dada82 00007f61f7f4c6c7 00007f61ef23eda7 00007f61ef23ead5 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:11.655306 (+3284268us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.445044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.445223 (+ 179us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.445233 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:08.445237 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.445436 (+ 199us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.445453 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.445461 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.445566 (+ 105us) tablet.cc:699] Row locks acquired
0811 18:57:08.445571 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.445618 (+ 47us) write_op.cc:270] Start()
0811 18:57:08.445660 (+ 42us) write_op.cc:276] Timestamp: P: 1786474628445490 usec, L: 0
0811 18:57:08.445666 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.446055 (+ 389us) log.cc:842] Serialized 622 byte log entry
0811 18:57:08.571255 (+125200us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.571353 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:08.571371 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.571421 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.571505 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.571633 (+ 128us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.571752 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:08.571763 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:08.571870 (+ 107us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.572224 (+ 354us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.572250 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:08.572258 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.572588 (+ 330us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.572595 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2383,"spinlock_wait_cycles":45226240,"child_traces":[["op",{"apply.queue_time_us":41,"apply.run_cpu_time_us":784,"apply.run_wall_time_us":1306,"num_ops":2,"prepare.queue_time_us":72796,"prepare.run_cpu_time_us":1006,"prepare.run_wall_time_us":1854,"replication_time_us":125554}]]}
I20260811 18:57:11.656610 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a85e1485efcd445a972fe25c37f2bb9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:11.657272 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a85e1485efcd445a972fe25c37f2bb9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.328147 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:09.044541 4162 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:11.658514 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:11.658643 4162 ts_tablet_manager.cc:1397] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 3.087s user 0.002s sys 0.005s
I20260811 18:57:10.759351 4586 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.412786 4443 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.14->[1.15-1.278] Dedup: 1.17->[1.18-1.278]
I20260811 18:57:09.106843 3796 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:11.374663 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 (DEFAULT_TABLE table=default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.375042 3856 tablet_service.cc:1467] Processing CreateTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 (DEFAULT_TABLE table=default.Table_9d18dc31cde54577b8bcc329191b74ef [id=cdb9b1499b924d41b8e4a872a6dc20da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.375172 4163 raft_consensus.cc:374] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:11.659547 4162 raft_consensus.cc:348] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.375473 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.659860 4162 raft_consensus.cc:374] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:11.659947 4162 raft_consensus.cc:729] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:11.375681 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.375846 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.375998 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:10.866863 4565 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.376595 3857 tablet_service.cc:1467] Processing CreateTablet for tablet e902efbb774244d18e3ba165bfb7c695 (DEFAULT_TABLE table=default.Table_60c6f6ccfa2747e1b4fff22a5779d63e [id=11c6327a2ff84680bcd60893f05b3493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.660218 4162 consensus_queue.cc:260] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.377022 3860 tablet_service.cc:1467] Processing CreateTablet for tablet c6852aaef1d5486b87e5462eae532719 (DEFAULT_TABLE table=default.Table_9409f5b4b0034412aedc183b302556a3 [id=75ea77c989e746f88a94e72de6cd11ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.379426 4572 consensus_queue.cc:1035] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:08.597003 3547 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 287) took 1498 ms. Trace:
I20260811 18:57:08.599251 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:11.661315 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 282) took 4658 ms. Trace:
I20260811 18:57:11.660813 3547 rpcz_store.cc:276] 0811 18:57:07.097992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.353343 (+1255351us) service_pool.cc:227] Handling call
0811 18:57:08.354280 (+ 937us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.373692 (+ 19412us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.374121 (+ 429us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.596594 (+222473us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.596745 (+ 151us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.596859 (+ 114us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.487200 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.487382 (+ 182us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.487393 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:08.487397 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.487545 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.487561 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.487569 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.487815 (+ 246us) tablet.cc:699] Row locks acquired
0811 18:57:08.487820 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.487878 (+ 58us) write_op.cc:270] Start()
0811 18:57:08.487929 (+ 51us) write_op.cc:276] Timestamp: P: 1786474628487743 usec, L: 0
0811 18:57:08.487935 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.561192 (+ 73257us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.561488 (+ 296us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0418d02 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c 00007f61e4f50344 00007f61e4f5391b 00007f61e4f53798 00000000009edb7c
0811 18:57:08.594556 (+ 33068us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.595105 (+ 549us) write_op.cc:301] APPLY: starting
0811 18:57:08.595124 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.595177 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.595179 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.595384 (+ 205us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.595436 (+ 52us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.595445 (+ 9us) write_op.cc:312] APPLY: finished
0811 18:57:08.595559 (+ 114us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.596442 (+ 883us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.596466 (+ 24us) write_op.cc:454] Released schema lock
0811 18:57:08.596476 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.597175 (+ 699us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.597181 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":37933824,"child_traces":[["op",{"apply.queue_time_us":460,"apply.run_cpu_time_us":804,"apply.run_wall_time_us":2147,"mutex_wait_us":23,"num_ops":2,"prepare.queue_time_us":112617,"prepare.run_cpu_time_us":2436,"prepare.run_wall_time_us":74373,"raft.queue_time_us":78,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":382,"replication_time_us":106582,"spinlock_wait_cycles":167904000}]]}
I20260811 18:57:11.661455 3546 rpcz_store.cc:276] 0811 18:57:07.002632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.333874 (+1331242us) service_pool.cc:227] Handling call
0811 18:57:08.386724 (+ 52850us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.386973 (+ 249us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:08.387061 (+ 88us) catalog_manager.cc:2761] Locking tablets
0811 18:57:08.573999 (+186938us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:08.598897 (+ 24898us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:08.599023 (+ 126us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:08.599079 (+ 56us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:11.661266 (+3062187us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.575682 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.575832 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.575838 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.575842 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:08.576002 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.576016 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.576023 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.576274 (+ 251us) tablet.cc:699] Row locks acquired
0811 18:57:08.576277 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:08.576328 (+ 51us) write_op.cc:270] Start()
0811 18:57:08.576410 (+ 82us) write_op.cc:276] Timestamp: P: 1786474628576190 usec, L: 0
0811 18:57:08.576414 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.582094 (+ 5680us) log.cc:842] Serialized 725 byte log entry
0811 18:57:08.582817 (+ 723us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0418d02 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c 00007f61e4f50344 00007f61e4f5391b 00007f61e4f53798 00000000009edb7c
0811 18:57:08.597811 (+ 14994us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.597941 (+ 130us) write_op.cc:301] APPLY: starting
0811 18:57:08.597955 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.597999 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.598002 (+ 3us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.598068 (+ 66us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.598101 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:08.598107 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:08.598192 (+ 85us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.598778 (+ 586us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.598803 (+ 25us) write_op.cc:454] Released schema lock
0811 18:57:08.598811 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.604806 (+ 5995us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.604816 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":13696,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":595,"apply.run_wall_time_us":6953,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":1670,"prepare.run_wall_time_us":8181,"raft.queue_time_us":40,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":325,"replication_time_us":21356,"spinlock_wait_cycles":12143488}]]}
I20260811 18:57:11.661950 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_056e34af1fcd4642abd908808acf8a67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:11.662451 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" } modify_external_catalogs: true
W20260811 18:57:11.662650 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_056e34af1fcd4642abd908808acf8a67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:11.662731 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a2c107af5134adbb9cdfb4b0981da6b" } modify_external_catalogs: true
W20260811 18:57:08.843233 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet b1ab2eee21264b1aa5ba00528cae3843 consensus state has no leader
W20260811 18:57:11.664884 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet e5fd9fce095e4a769dbb38c7903359cc consensus state has no leader
I20260811 18:57:09.518744 4554 consensus_queue.cc:1035] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260811 18:57:09.504180 3522 master_runner.cc:186] Error getting master registration for 127.3.72.125:38889: OK, instance_id { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" instance_seqno: 1786474627171025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:11.667068 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:09.602849 4555 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:09.965920 4556 raft_consensus.cc:482] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.667270 4555 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.667609 4167 raft_consensus.cc:2229] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:10.276178 4574 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:11.667830 4574 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.668063 4167 pending_rounds.cc:62] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:11.668183 4167 pending_rounds.cc:68] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400066460418048 op_type: NO_OP noop_request { }
I20260811 18:57:10.336215 4576 raft_consensus.cc:482] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.668463 4576 raft_consensus.cc:504] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.555601 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:11.669837 3552 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 281) took 4667 ms. Trace:
I20260811 18:57:11.669982 3552 rpcz_store.cc:276] 0811 18:57:07.002427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.250686 (+248259us) service_pool.cc:227] Handling call
0811 18:57:08.350789 (+1100103us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:08.350965 (+ 176us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:08.351018 (+ 53us) catalog_manager.cc:2761] Locking tablets
0811 18:57:08.351170 (+ 152us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:08.555139 (+203969us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:08.555262 (+ 123us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:08.555323 (+ 61us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:11.669791 (+3114468us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.418556 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.418682 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.418688 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.418691 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:08.418868 (+ 177us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.418885 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.418893 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.419098 (+ 205us) tablet.cc:699] Row locks acquired
0811 18:57:08.419102 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:08.419149 (+ 47us) write_op.cc:270] Start()
0811 18:57:08.419194 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628419022 usec, L: 0
0811 18:57:08.419198 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.419430 (+ 232us) log.cc:842] Serialized 725 byte log entry
0811 18:57:08.553934 (+134504us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.554065 (+ 131us) write_op.cc:301] APPLY: starting
0811 18:57:08.554094 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.554161 (+ 67us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.554236 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.554358 (+ 122us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.554400 (+ 42us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:08.554407 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.554584 (+ 177us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.554974 (+ 390us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.555014 (+ 40us) write_op.cc:454] Released schema lock
0811 18:57:08.555023 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.555791 (+ 768us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.555797 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":57,"apply.run_cpu_time_us":804,"apply.run_wall_time_us":1803,"num_ops":2,"prepare.queue_time_us":66615,"prepare.run_cpu_time_us":987,"prepare.run_wall_time_us":5718,"replication_time_us":134700}]]}
I20260811 18:57:11.670979 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6d55543667e9439eb0420ca7ec6e8e74" } new_table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" modify_external_catalogs: true
I20260811 18:57:09.782995 3676 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:10.387156 3664 kernel_stack_watchdog.cc:198] Thread 3676 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 265ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fcdd5c86980 (unknown) at ??:0
@ 0x7fcdd5c809c6 __GI___pthread_rwlock_wrlock at ??:0
@ 0x7fcdcfd7769c google::LogMessage::Flush() at ??:0
@ 0x7fcdcfd776d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fcddde229f7 kudu::consensus::LeaderElection::CheckForDecision() at ??:0
@ 0x7fcddde23ce1 kudu::consensus::LeaderElection::VoteResponseRpcCallback() at ??:0
@ 0x7fcddde2673a kudu::consensus::LeaderElection::Run()::$_0::operator()() at ??:0
@ 0x7fcddde264fe std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9426f86 kudu::rpc::Proxy::AsyncRequest()::$_1::operator()() at ??:0
@ 0x7fcdd9426b6e std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fcdd9412dea kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7fcdd9413321 kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7fcdd93b0aa9 kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7fcdd93af5a5 kudu::rpc::Connection::ReadHandler() at ??:0
I20260811 18:57:10.380551 4578 raft_consensus.cc:482] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.672713 4578 raft_consensus.cc:504] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:10.222826 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 89b5c0f3673445bbba9136a040ca9bbb (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 89b5c0f3673445bbba9136a040ca9bbb consensus state has no leader
I20260811 18:57:10.782948 4580 raft_consensus.cc:482] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.674500 4580 raft_consensus.cc:504] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:11.087174 4476 raft_consensus.cc:482] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:11.675932 4476 raft_consensus.cc:504] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:08.560760 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc0e1659d31645b28fe322618e22cb6c" } modify_external_catalogs: true
I20260811 18:57:11.677273 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc0e1659d31645b28fe322618e22cb6c" } modify_external_catalogs: true
I20260811 18:57:11.377933 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:11.667346 4556 raft_consensus.cc:504] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:11.378454 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:08.574854 3555 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 288) took 1476 ms. Trace:
I20260811 18:57:11.377341 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6d1d58835ad24749a00ce27b4bc73c59 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
W20260811 18:57:11.681180 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 303848us
I20260811 18:57:11.682976 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:11.688743 4555 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.699486 4167 raft_consensus.cc:2866] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:11.699774 4167 raft_consensus.cc:2258] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:12.471638 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1460ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:11.723701 3864 rpcz_store.cc:276] 0811 18:57:07.344536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.344630 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:08.342976 (+998346us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.343075 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:10.868245 (+2525170us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.872936 (+ 4691us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:10.873136 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.723459 3853 rpcz_store.cc:276] 0811 18:57:07.362128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.362226 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.345928 (+983702us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.345951 (+ 23us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:10.867425 (+2521474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.879446 (+ 12021us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:10.879849 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.741096 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71df3738a5cf4b12820dda131ced62e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.700870 4167 ts_tablet_manager.cc:1905] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:11.741484 3852 rpcz_store.cc:276] 0811 18:57:07.438203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.438286 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:08.346150 (+907864us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.346168 (+ 18us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:09.783687 (+1437519us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:09.836415 (+ 52728us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:09.836685 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.741743 4163 raft_consensus.cc:729] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:11.723608 4586 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.770839 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08436f33db043b1bc6395cd92102e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.771343 4586 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.743510 4169 log.cc:1196] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6ef0a376939d4d57b0d95db644f25dd9
I20260811 18:57:12.771960 4169 ts_tablet_manager.cc:1939] T 6ef0a376939d4d57b0d95db644f25dd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:12.772406 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 149) took 7017 ms. Trace:
I20260811 18:57:12.772526 4169 rpcz_store.cc:276] 0811 18:57:05.755415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.755624 (+ 209us) service_pool.cc:227] Handling call
0811 18:57:08.311745 (+2556121us) ts_tablet_manager.cc:1145] Deleting tablet 6ef0a376939d4d57b0d95db644f25dd9
0811 18:57:08.311759 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:08.429688 (+117929us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:10.891127 (+2461439us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.772340 (+1881213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2508479}
I20260811 18:57:11.681022 3555 rpcz_store.cc:276] 0811 18:57:07.098217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.355150 (+1256933us) service_pool.cc:227] Handling call
0811 18:57:08.356103 (+ 953us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:08.372451 (+ 16348us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x6160000160b8. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61e4ebc351 00007f61f7dd4376 00007f61f7d45420 00007f61f7d3ed35 00007f61f7f4046a 00007f61ef232466 00007f61ef232195 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25
0811 18:57:08.372931 (+ 480us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:08.574552 (+201621us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:08.574625 (+ 73us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:08.574684 (+ 59us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:08.446965 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:08.447246 (+ 281us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:08.447252 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:08.447268 (+ 16us) tablet.cc:660] Decoding operations
0811 18:57:08.447395 (+ 127us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:08.447408 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:08.447424 (+ 16us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:08.447539 (+ 115us) tablet.cc:699] Row locks acquired
0811 18:57:08.447544 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:08.447596 (+ 52us) write_op.cc:270] Start()
0811 18:57:08.447641 (+ 45us) write_op.cc:276] Timestamp: P: 1786474628447458 usec, L: 0
0811 18:57:08.447647 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:08.447925 (+ 278us) log.cc:842] Serialized 433 byte log entry
0811 18:57:08.572767 (+124842us) op_driver.cc:444] REPLICATION: finished
0811 18:57:08.572851 (+ 84us) write_op.cc:301] APPLY: starting
0811 18:57:08.572866 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:08.572910 (+ 44us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:08.572940 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:08.573060 (+ 120us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:08.573093 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:08.573100 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:08.573170 (+ 70us) log.cc:842] Serialized 29 byte log entry
0811 18:57:08.573418 (+ 248us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.573438 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:08.573446 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:08.573502 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:08.573506 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":33415040,"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":519,"apply.run_wall_time_us":700,"num_ops":2,"prepare.queue_time_us":73449,"prepare.run_cpu_time_us":1086,"prepare.run_wall_time_us":1733,"replication_time_us":125095}]]}
I20260811 18:57:12.773059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 151) took 7016 ms. Trace:
I20260811 18:57:12.773175 4169 rpcz_store.cc:276] 0811 18:57:05.756857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.756916 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:12.772830 (+7015914us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:12.772868 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.772995 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6968997}
I20260811 18:57:12.773535 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 152) took 7016 ms. Trace:
I20260811 18:57:12.773634 4169 rpcz_store.cc:276] 0811 18:57:05.757290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.757356 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:12.773389 (+7016033us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:12.773398 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.773476 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6968546}
I20260811 18:57:12.773977 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 148) took 7024 ms. Trace:
I20260811 18:57:12.774068 4169 rpcz_store.cc:276] 0811 18:57:05.749406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.749503 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:12.773827 (+7024324us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:12.773837 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.773918 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6959143}
I20260811 18:57:11.744118 3872 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1923ce0049448fe4e28f2dd258dc (DEFAULT_TABLE table=default.Table_0cb87feb6db445d792ba8021cbe75510 [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.774667 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1923ce0049448fe4e28f2dd258dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.775630 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3ec38f9b55489ba00ed76f9e4456f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:11.745339 4573 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:11.744593 4592 raft_consensus.cc:686] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:12.778896 4592 consensus_queue.cc:237] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:11.701783 3707 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38286 (request call id 208) took 2697 ms. Trace:
I20260811 18:57:12.780272 3707 rpcz_store.cc:276] 0811 18:57:09.003914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.004891 (+ 977us) service_pool.cc:227] Handling call
0811 18:57:10.863369 (+1858478us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:10.863402 (+ 33us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.657490 (+794088us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.701376 (+ 43886us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.701646 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:11.860515 3858 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 199) took 4528 ms. Trace:
I20260811 18:57:11.843371 3550 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 291) took 4686 ms. Trace:
I20260811 18:57:12.780773 3858 rpcz_store.cc:276] 0811 18:57:07.332193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.332291 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:08.426502 (+1094211us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.426544 (+ 42us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.750107 (+3323563us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.860220 (+110113us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.860461 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:11.742302 3824 leader_election.cc:341] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:12.581609 4658 raft_consensus.cc:482] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
W20260811 18:57:12.781216 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 1.05441e+06us
I20260811 18:57:12.781235 4658 raft_consensus.cc:504] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.782198 4658 leader_election.cc:288] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.699005 4162 ts_tablet_manager.cc:1428] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.040s user 0.001s sys 0.001s
W20260811 18:57:12.783990 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.017s (SENT)
W20260811 18:57:12.784042 4162 ts_tablet_manager.cc:1460] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Tablet startup took 1125ms
W20260811 18:57:12.784173 4162 ts_tablet_manager.cc:1462] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Trace:
0811 18:57:03.144267 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.144338 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:03.145184 (+ 846us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:03.145224 (+ 40us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:03.150679 (+ 5455us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.158855 (+ 8176us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:03.159011 (+ 156us) inbound_call.cc:173] Queueing success response
0811 18:57:08.571485 (+5412474us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0811 18:57:11.658747 (+3087262us) ts_tablet_manager.cc:1429] Starting tablet replica
0811 18:57:11.658996 (+ 249us) tablet_replica.cc:259] Starting instrumentation
0811 18:57:11.659147 (+ 151us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":6368,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":53,"tablet-open.queue_time_us":5412394,"thread_start_us":30122,"threads_started":1}
I20260811 18:57:12.784718 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3060822021431584a494e6941be756 (DEFAULT_TABLE table=default.Table_a85e1485efcd445a972fe25c37f2bb9e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.784870 4162 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:12.784924 3867 tablet_service.cc:1467] Processing CreateTablet for tablet c26851245cab4a2d80e94d4fe278fd50 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.786003 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.770850 4163 consensus_queue.cc:260] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.786197 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3060822021431584a494e6941be756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.707816 4660 raft_consensus.cc:482] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
I20260811 18:57:12.787289 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.787292 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.788442 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 (DEFAULT_TABLE table=default.Table_056e34af1fcd4642abd908808acf8a67 [id=1bcc678c540844b38d761cec1618c852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:11.905488 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:12.789505 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:11.702407 4574 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702447 4576 leader_election.cc:288] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702677 4578 leader_election.cc:288] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.702881 4580 leader_election.cc:288] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.930943 3554 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 294) took 4705 ms. Trace:
I20260811 18:57:11.702955 4476 leader_election.cc:288] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:11.932816 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8eb041f3d704762868a048faec0e0fc consensus state has no leader
I20260811 18:57:11.934561 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f335e50ae57547ed80b144129faaffbb
I20260811 18:57:11.703188 3744 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:11.703454 4556 leader_election.cc:288] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:11.703598 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:11.703456 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:12.790515 3744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.3.72.66:51189 (request call id 27) took 1414 ms. Trace:
W20260811 18:57:11.703984 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 1.8895e+06us
I20260811 18:57:12.790666 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc5448a555944dfba8ccae09e25cbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.790663 3744 rpcz_store.cc:276] 0811 18:57:11.376369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.377764 (+ 1395us) service_pool.cc:227] Handling call
0811 18:57:12.790455 (+1412691us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:11.951768 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e1efffade0f486d8d627e56e0be6ff6 consensus state has no leader
I20260811 18:57:11.704895 4637 raft_consensus.cc:2781] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:12.791616 4637 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:12.791929 4637 raft_consensus.cc:3037] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:11.952519 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:11.958628 3547 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 296) took 4505 ms. Trace:
I20260811 18:57:12.004621 3553 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 295) took 4677 ms. Trace:
W20260811 18:57:12.673421 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0ace00158b744bd8a97dc1c47fc6049c consensus state has no leader
I20260811 18:57:11.704561 3719 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38286 (request call id 205) took 3036 ms. Trace:
I20260811 18:57:12.001572 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c26851245cab4a2d80e94d4fe278fd50 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.794628 3719 rpcz_store.cc:276] 0811 18:57:08.667626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.669424 (+ 1798us) service_pool.cc:227] Handling call
0811 18:57:09.014788 (+345364us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:09.014825 (+ 37us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:11.662562 (+2647737us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:11.704206 (+ 41644us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:11.704443 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.794845 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26851245cab4a2d80e94d4fe278fd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.001849 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 (DEFAULT_TABLE table=default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.795262 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 221d0d8d69f142a29ec7065a82b3f769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.796968 4637 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.002604 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 (DEFAULT_TABLE table=default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.056010 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 (DEFAULT_TABLE table=default.Table_056e34af1fcd4642abd908808acf8a67 [id=1bcc678c540844b38d761cec1618c852]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:11.828282 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.269s (SENT)
I20260811 18:57:12.798051 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122b7898fea44ef78f269b1505d8ddc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.095178 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3060822021431584a494e6941be756 (DEFAULT_TABLE table=default.Table_a85e1485efcd445a972fe25c37f2bb9e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.798628 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f3060822021431584a494e6941be756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.798656 4637 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.799652 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122b7898fea44ef78f269b1505d8ddc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.799806 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc5448a555944dfba8ccae09e25cbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.799808 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 (DEFAULT_TABLE table=default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:12.801436 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26851245cab4a2d80e94d4fe278fd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.801967 4163 ts_tablet_manager.cc:1428] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 1.419s user 0.000s sys 0.002s
I20260811 18:57:12.804090 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:12.810135 4163 ts_tablet_manager.cc:1460] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Tablet startup took 1433ms
W20260811 18:57:12.810418 4163 ts_tablet_manager.cc:1462] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trace:
0811 18:57:03.114182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:03.114276 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:03.115152 (+ 876us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:03.115179 (+ 27us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:03.156195 (+ 41016us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:03.176303 (+ 20108us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:03.176459 (+ 156us) inbound_call.cc:173] Queueing success response
0811 18:57:08.305182 (+5128723us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0811 18:57:11.369027 (+3063845us) ts_tablet_manager.cc:1429] Starting tablet replica
0811 18:57:11.369305 (+ 278us) tablet_replica.cc:259] Starting instrumentation
0811 18:57:11.369513 (+ 208us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":270,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"tablet-open.queue_time_us":5128756}
I20260811 18:57:12.811581 4163 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:12.811898 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 166) took 6503 ms. Trace:
I20260811 18:57:12.812115 4169 rpcz_store.cc:276] 0811 18:57:06.285266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.286322 (+ 1056us) service_pool.cc:227] Handling call
0811 18:57:12.774285 (+6487963us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:12.774295 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.788159 (+ 13864us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6485272}
I20260811 18:57:12.814651 4162 tablet_bootstrap.cc:654] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:12.822791 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 167) took 6491 ms. Trace:
I20260811 18:57:12.824594 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 221d0d8d69f142a29ec7065a82b3f769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.823091 4169 rpcz_store.cc:276] 0811 18:57:06.321194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.321258 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:12.812444 (+6491186us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:12.812459 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.812714 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6490620}
I20260811 18:57:12.832489 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43ad6993b63e4bbd98c8cc52f1a53451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.809335 4660 raft_consensus.cc:504] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.833508 4658 raft_consensus.cc:482] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:12.843942 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbf6ce167fb348be8d61372791f0cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.846686 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.849061 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.849776 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 179) took 6064 ms. Trace:
I20260811 18:57:12.847469 4658 raft_consensus.cc:504] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.853428 3892 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:12.857013 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106e0e1f4b845429b10f4e100f69a8b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.857899 4169 rpcz_store.cc:276] 0811 18:57:06.769668 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.791769 (+ 22101us) service_pool.cc:227] Handling call
0811 18:57:12.826488 (+6034719us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:12.826532 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.834014 (+ 7482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6034216}
I20260811 18:57:12.858383 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe203387bc9c45ba82afe6b058aa5e5f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:12.864652 3676 leader_election.cc:341] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:12.865103 3676 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.865450 4556 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:57:12.865607 4556 raft_consensus.cc:482] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
W20260811 18:57:12.865545 3676 leader_election.cc:341] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:12.865734 4556 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Advancing to term 2
W20260811 18:57:12.866108 3676 leader_election.cc:341] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8efb663060ef457cbece962fce701eb1
I20260811 18:57:12.870564 4556 raft_consensus.cc:504] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.872483 4556 leader_election.cc:288] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.863276 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae03bc2e2b4b4d0d9e26cc4c2fbe5d97" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.875278 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e902efbb774244d18e3ba165bfb7c695. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:12.862075 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 70817.5us
I20260811 18:57:12.878609 4167 ts_tablet_manager.cc:1918] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:12.878840 4660 leader_election.cc:288] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:12.879171 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 180) took 6066 ms. Trace:
I20260811 18:57:12.879318 4169 rpcz_store.cc:276] 0811 18:57:06.797118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.800501 (+ 3383us) service_pool.cc:227] Handling call
0811 18:57:12.858224 (+6057723us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:12.858237 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.863866 (+ 5629us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6056725}
I20260811 18:57:12.880057 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 181) took 6027 ms. Trace:
I20260811 18:57:12.880174 4169 rpcz_store.cc:276] 0811 18:57:06.852109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.852232 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:12.879706 (+6027474us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:12.879721 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.879948 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6027108}
I20260811 18:57:12.880594 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 187) took 5713 ms. Trace:
I20260811 18:57:12.880769 4169 rpcz_store.cc:276] 0811 18:57:07.167102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.167217 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:12.880421 (+5713204us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:12.880432 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.880519 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4571098}
I20260811 18:57:12.881422 4169 tablet_replica.cc:330] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:12.882900 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 1.05457e+06us
W20260811 18:57:12.884316 3676 leader_election.cc:341] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:12.886624 4167 log.cc:1196] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/494c09652d9e408880ca606ae0b30773
I20260811 18:57:12.892618 4167 ts_tablet_manager.cc:1939] T 494c09652d9e408880ca606ae0b30773 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:12.893275 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2718cd4a98742dfa8014a8a38c69c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.893891 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 141) took 7122 ms. Trace:
I20260811 18:57:12.894747 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6852aaef1d5486b87e5462eae532719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.896159 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:12.894084 4167 rpcz_store.cc:276] 0811 18:57:05.771018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.771538 (+ 520us) service_pool.cc:227] Handling call
0811 18:57:11.371452 (+5599914us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:11.371467 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:12.802176 (+1430709us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.878439 (+ 76263us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.893813 (+ 15374us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":295795,"tablet-delete.queue_time_us":5581187}
I20260811 18:57:12.899655 4162 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:12.902207 4162 ts_tablet_manager.cc:1397] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.117s user 0.008s sys 0.003s
I20260811 18:57:12.902238 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b227bc7305f4b2dae28f04e7001553b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:12.902508 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:12.902722 3890 raft_consensus.cc:3037] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:12.903749 4162 raft_consensus.cc:348] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.903996 4162 raft_consensus.cc:374] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:12.904718 4556 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:12.904968 4556 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.908274 3824 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.874951 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:12.910493 3889 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:12.914321 4162 raft_consensus.cc:729] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:13.119899 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 3106ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:12.915189 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:12.915545 4162 consensus_queue.cc:260] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:12.914685 4167 tablet_replica.cc:330] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.137910 4162 ts_tablet_manager.cc:1428] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.236s user 0.004s sys 0.000s
I20260811 18:57:12.914844 3676 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.914624 4556 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:12.920097 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:12.920986 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.139981 4167 raft_consensus.cc:2229] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:12.924568 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af3e74889b64c408c9ed9c54916f890" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:12.924937 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c18120da2d245f495c52a670047760d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.140413 4167 raft_consensus.cc:2258] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:12.925601 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81334e7da3741a9b24f8978312181b9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:12.998378 4651 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 15995523294368684499) tablet 00000000000000000000000000000000 cur holder 0x617000072a80
I20260811 18:57:12.925302 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6ef0a376939d4d57b0d95db644f25dd9 (table default.Table_bdb9cbaf32744ad7b14399222fb784f6 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
W20260811 18:57:13.141040 3539 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 351204us
I20260811 18:57:13.141392 4167 ts_tablet_manager.cc:1905] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:13.141444 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33d995d72f8142afaeb57a2c29910fa3 already in progress: creating tablet
I20260811 18:57:12.956321 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:13.141772 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a30bb384136f4a169a5751655437d5a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a30bb384136f4a169a5751655437d5a3 already in progress: creating tablet
I20260811 18:57:13.141904 3740 raft_consensus.cc:2379] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
W20260811 18:57:13.141992 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33979de5b48c4674a46c8da3b7151753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33979de5b48c4674a46c8da3b7151753 already in progress: creating tablet
W20260811 18:57:13.138245 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 223688us
W20260811 18:57:13.142411 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
W20260811 18:57:13.142656 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
W20260811 18:57:13.142899 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
W20260811 18:57:13.143124 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
W20260811 18:57:13.143317 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
W20260811 18:57:13.143565 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 08b8dcf7a56043efbbd40bb888616492 because the tablet was not found. No further retry: Not found: Tablet not found: 08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:12.912178 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:13.137239 3744 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:12.925493 3552 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 298) took 4233 ms. Trace:
I20260811 18:57:13.145462 4556 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:57:13.145980 4556 raft_consensus.cc:482] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 59811bdf82234ef7ac5865dbecfcc388)
I20260811 18:57:13.146155 4556 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.145514 3552 rpcz_store.cc:276] 0811 18:57:08.557320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.670771 (+3113451us) service_pool.cc:227] Handling call
0811 18:57:11.671210 (+ 439us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.671496 (+ 286us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:11.671740 (+ 244us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:11.671812 (+ 72us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:11.695381 (+ 23569us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d70cc2 00007f61f7d54693 00007f61f7f426e9 00007f61ef235de6
0811 18:57:11.951283 (+255902us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:12.791017 (+839734us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.697919 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.698017 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.698023 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:11.698027 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.698120 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.698130 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.698136 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:11.698182 (+ 46us) tablet.cc:699] Row locks acquired
0811 18:57:11.698184 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:11.698208 (+ 24us) write_op.cc:270] Start()
0811 18:57:11.698229 (+ 21us) write_op.cc:276] Timestamp: P: 1786474631698170 usec, L: 0
0811 18:57:11.698232 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.699121 (+ 889us) log.cc:842] Serialized 405 byte log entry
0811 18:57:11.701390 (+ 2269us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f03677cf 00007f61f0366c7e 00007f61f0417e45 00007f61f0418cd9 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c
0811 18:57:11.948867 (+247477us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.950327 (+ 1460us) write_op.cc:301] APPLY: starting
0811 18:57:11.950364 (+ 37us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.950479 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.950485 (+ 6us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.950564 (+ 79us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.950615 (+ 51us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.950627 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:11.950757 (+ 130us) log.cc:842] Serialized 23 byte log entry
0811 18:57:11.951118 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951144 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:11.951156 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.951237 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951242 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":52106624,"child_traces":[["op",{"apply.queue_time_us":1355,"apply.run_cpu_time_us":723,"apply.run_wall_time_us":995,"num_ops":1,"prepare.queue_time_us":2478,"prepare.run_cpu_time_us":1028,"prepare.run_wall_time_us":3512,"replication_time_us":250589,"spinlock_wait_cycles":4404992}]]}
I20260811 18:57:13.148312 3552 catalog_manager.cc:5495] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.148912 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.149518 4556 raft_consensus.cc:504] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:12.917147 3888 raft_consensus.cc:3037] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.150130 4556 leader_election.cc:288] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:13.152194 3676 leader_election.cc:341] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.153234 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de9acdb33a2438eb4378a5eac844c35" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:13.153486 3891 raft_consensus.cc:3037] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:13.153848 3888 raft_consensus.cc:2454] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 2.
I20260811 18:57:13.154006 4592 raft_consensus.cc:2741] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260811 18:57:12.930791 3554 rpcz_store.cc:276] 0811 18:57:07.225481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.651484 (+4426003us) service_pool.cc:227] Handling call
0811 18:57:11.653080 (+ 1596us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.653741 (+ 661us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.930446 (+276705us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.930616 (+ 170us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.930752 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.694910 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.695166 (+ 256us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.695177 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:11.695182 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:11.695324 (+ 142us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.695342 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.695350 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.695476 (+ 126us) tablet.cc:699] Row locks acquired
0811 18:57:11.695480 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:11.695534 (+ 54us) write_op.cc:270] Start()
0811 18:57:11.695583 (+ 49us) write_op.cc:276] Timestamp: P: 1786474631695488 usec, L: 0
0811 18:57:11.695590 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.695964 (+ 374us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.915620 (+219656us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.927947 (+ 12327us) write_op.cc:301] APPLY: starting
0811 18:57:11.927976 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.928148 (+ 172us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.928306 (+ 158us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.928672 (+ 366us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.928756 (+ 84us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.928768 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:11.929047 (+ 279us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.929928 (+ 881us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.930083 (+ 155us) write_op.cc:454] Released schema lock
0811 18:57:11.930097 (+ 14us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.930563 (+ 466us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.930635 (+ 72us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12198,"apply.run_cpu_time_us":2083,"apply.run_wall_time_us":38127,"num_ops":2,"prepare.queue_time_us":96,"prepare.run_cpu_time_us":1176,"prepare.run_wall_time_us":1177,"replication_time_us":219982}]]}
I20260811 18:57:12.932365 3547 rpcz_store.cc:276] 0811 18:57:07.453306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.661671 (+4208365us) service_pool.cc:227] Handling call
0811 18:57:11.662760 (+ 1089us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.689596 (+ 26836us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.697715 (+ 8119us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d4024d 00007f61f7f4046a 00007f61ef232466 00007f61ef232195
0811 18:57:11.958242 (+260527us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.958408 (+ 166us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.958538 (+ 130us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.730323 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.730941 (+ 618us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.730952 (+ 11us) write_op.cc:435] Acquired schema lock
0811 18:57:11.731009 (+ 57us) tablet.cc:660] Decoding operations
0811 18:57:11.731595 (+ 586us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.731668 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.731739 (+ 71us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.732259 (+ 520us) tablet.cc:699] Row locks acquired
0811 18:57:11.732264 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.732485 (+ 221us) write_op.cc:270] Start()
0811 18:57:11.732647 (+ 162us) write_op.cc:276] Timestamp: P: 1786474631732377 usec, L: 0
0811 18:57:11.732707 (+ 60us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.734554 (+ 1847us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.949490 (+214936us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.952026 (+ 2536us) write_op.cc:301] APPLY: starting
0811 18:57:11.952039 (+ 13us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.952077 (+ 38us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.952130 (+ 53us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.952286 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.952324 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.952334 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:11.952399 (+ 65us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.952920 (+ 521us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.953004 (+ 84us) write_op.cc:454] Released schema lock
0811 18:57:11.953075 (+ 71us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.954267 (+ 1192us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.954277 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":10548992,"child_traces":[["op",{"apply.queue_time_us":2444,"apply.run_cpu_time_us":970,"apply.run_wall_time_us":2543,"num_ops":2,"prepare.queue_time_us":32357,"prepare.run_cpu_time_us":2588,"prepare.run_wall_time_us":5337,"replication_time_us":216704}]]}
I20260811 18:57:13.155179 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c70a74ca4b24d0f9e9720c31ebd8e4d" } new_table_name: "default.Table_34c1bb6042d54870bb236a2204ce71ed" modify_external_catalogs: true
I20260811 18:57:13.155412 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_666e839163c3494a9c95e78ff76bb015" } new_table_name: "default.Table_fce143fe6d0849a0a3af8939df804986" modify_external_catalogs: true
I20260811 18:57:12.934023 3553 rpcz_store.cc:276] 0811 18:57:07.326971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.656289 (+4329318us) service_pool.cc:227] Handling call
0811 18:57:11.657385 (+ 1096us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:11.657947 (+ 562us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:12.002962 (+345015us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:12.004356 (+ 1394us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:12.004499 (+ 143us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.697102 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.697270 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.697278 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:11.697282 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.697432 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.697443 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.697450 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.697542 (+ 92us) tablet.cc:699] Row locks acquired
0811 18:57:11.697547 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.697581 (+ 34us) write_op.cc:270] Start()
0811 18:57:11.697619 (+ 38us) write_op.cc:276] Timestamp: P: 1786474631697539 usec, L: 0
0811 18:57:11.697624 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.697816 (+ 192us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.915797 (+217981us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.935105 (+ 19308us) write_op.cc:301] APPLY: starting
0811 18:57:11.935185 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.935416 (+ 231us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.935577 (+ 161us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.940932 (+ 5355us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.942990 (+ 2058us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.972971 (+ 29981us) write_op.cc:312] APPLY: finished
0811 18:57:11.982130 (+ 9159us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.998703 (+ 16573us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.999025 (+ 322us) write_op.cc:454] Released schema lock
0811 18:57:11.999193 (+ 168us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:12.003570 (+ 4377us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:12.004387 (+ 817us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19064,"apply.run_cpu_time_us":3835,"apply.run_wall_time_us":74508,"num_ops":2,"prepare.queue_time_us":1955,"prepare.run_cpu_time_us":773,"prepare.run_wall_time_us":772,"replication_time_us":218154,"thread_start_us":32908,"threads_started":1}]]}
I20260811 18:57:13.156795 3891 raft_consensus.cc:2454] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 2.
I20260811 18:57:12.923978 4658 leader_election.cc:288] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.157152 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f" } modify_external_catalogs: true
I20260811 18:57:13.157549 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1329ea40dbd4f748a0e35686a5e867f" } modify_external_catalogs: true
I20260811 18:57:13.157661 4162 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:12.931778 3872 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 217) took 1195 ms. Trace:
I20260811 18:57:12.935269 3854 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 202) took 5554 ms. Trace:
I20260811 18:57:12.937847 3863 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 214) took 1560 ms. Trace:
I20260811 18:57:13.157913 3863 rpcz_store.cc:276] 0811 18:57:11.372370 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.372427 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:11.373085 (+ 658us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.373112 (+ 27us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.788968 (+1415856us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.932308 (+143340us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.932473 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.967528 4163 tablet_bootstrap.cc:654] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.159142 3676 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:12.980262 3890 raft_consensus.cc:2454] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:12.935139 3555 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 34) took 1110 ms (client timeout 1000 ms). Trace:
I20260811 18:57:12.989931 3851 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 205) took 5548 ms. Trace:
I20260811 18:57:13.157846 3854 rpcz_store.cc:276] 0811 18:57:07.359923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.360052 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:08.344516 (+984464us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.344539 (+ 23us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.787033 (+4442494us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.914429 (+127396us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.914679 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.159490 3851 rpcz_store.cc:276] 0811 18:57:07.440720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.440822 (+ 102us) service_pool.cc:227] Handling call
0811 18:57:08.346383 (+905561us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.346401 (+ 18us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.856364 (+4509963us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.983964 (+127600us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.988820 (+ 4856us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:13.159444 3555 rpcz_store.cc:269] 0811 18:57:11.682920 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.793122 (+1110202us) service_pool.cc:214] Skipping call since client already timed out
0811 18:57:12.793198 (+ 76us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260811 18:57:13.157826 3872 rpcz_store.cc:276] 0811 18:57:11.726261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.726354 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:12.774457 (+1048103us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:12.774476 (+ 19us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.777896 (+ 3420us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:12.921324 (+143428us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:12.921560 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.044679 3848 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 68) took 1159 ms. Trace:
I20260811 18:57:13.159992 3848 rpcz_store.cc:276] 0811 18:57:11.884726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.884802 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:12.777201 (+892399us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x615000193aa4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.783564 (+ 6363us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x6150001610a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.799887 (+ 16323us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.846333 (+ 46446us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.021782 (+175449us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.044495 (+ 22713us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":268899328}
I20260811 18:57:13.160223 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6460e311a6da4deaba4efce4c6dbd883" } new_table_name: "default.Table_1996443a5de84ded8919fb5c51dc6554" modify_external_catalogs: true
I20260811 18:57:13.047983 3871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 211) took 1675 ms. Trace:
I20260811 18:57:13.160516 3871 rpcz_store.cc:276] 0811 18:57:11.371970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.373455 (+ 1485us) service_pool.cc:227] Handling call
0811 18:57:11.723958 (+350503us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.723998 (+ 40us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.021685 (+1297687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.047548 (+ 25863us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.047815 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.051541 3857 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 212) took 1679 ms. Trace:
I20260811 18:57:13.053131 3856 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 215) took 1680 ms. Trace:
I20260811 18:57:13.093641 3847 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 67) took 4533 ms. Trace:
I20260811 18:57:13.160800 3857 rpcz_store.cc:276] 0811 18:57:11.372106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.376316 (+ 4210us) service_pool.cc:227] Handling call
0811 18:57:11.660363 (+284047us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.660394 (+ 31us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.027361 (+1366967us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.051104 (+ 23743us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.051380 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.160828 3847 rpcz_store.cc:276] 0811 18:57:08.556166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.556538 (+ 372us) service_pool.cc:227] Handling call
0811 18:57:11.374499 (+2817961us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x6150001d2524. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.788413 (+1413914us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.852339 (+ 63926us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.920534 (+ 68195us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x6150000a3824. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.992166 (+ 71632us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.089609 (+ 97443us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1375992704}
I20260811 18:57:13.106549 4169 raft_consensus.cc:2229] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:13.161906 4169 raft_consensus.cc:2258] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:12.935541 3548 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 292) took 5632 ms. Trace:
I20260811 18:57:12.938139 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 297) took 4367 ms. Trace:
I20260811 18:57:12.938419 3549 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 290) took 5793 ms. Trace:
I20260811 18:57:13.107273 3850 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 66) took 4538 ms. Trace:
I20260811 18:57:13.162842 3850 rpcz_store.cc:276] 0811 18:57:08.556006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.556231 (+ 225us) service_pool.cc:227] Handling call
0811 18:57:12.774534 (+4218303us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x615000193aa4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.782527 (+ 7993us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x6150001610a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.790853 (+ 8326us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x6150000a76a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:12.847058 (+ 56205us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x6150000c7c24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.022077 (+175019us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x6150001926a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.094798 (+ 72721us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4644815488}
I20260811 18:57:13.162587 3549 rpcz_store.cc:276] 0811 18:57:07.131470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.485073 (+1353603us) service_pool.cc:227] Handling call
0811 18:57:08.560532 (+ 75459us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f412e6 00007f61ef234126 00007f61ef233e55 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:11.677507 (+3116975us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.677718 (+ 211us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:11.677803 (+ 85us) catalog_manager.cc:2761] Locking tablets
0811 18:57:11.677984 (+ 181us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:11.695827 (+ 17843us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d58d4c 00007f61f7d4dc70 00007f61f7d4c1bf 00007f61f7f41446
0811 18:57:11.951994 (+256167us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:11.952115 (+ 121us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:11.952350 (+ 235us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:12.918958 (+966608us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.701486 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.701595 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.701602 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:11.701606 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.701769 (+ 163us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.701780 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.701787 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.701916 (+ 129us) tablet.cc:699] Row locks acquired
0811 18:57:11.701921 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.701955 (+ 34us) write_op.cc:270] Start()
0811 18:57:11.702001 (+ 46us) write_op.cc:276] Timestamp: P: 1786474631701914 usec, L: 0
0811 18:57:11.702005 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.702166 (+ 161us) log.cc:842] Serialized 725 byte log entry
0811 18:57:11.703865 (+ 1699us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f03677cf 00007f61f0366c7e 00007f61f0417e45 00007f61f0418cd9 00007f61f174b7f1 00007f61f174a359 00007f61f17521bb 00007f61f1752038 00000000009edb7c
0811 18:57:11.948970 (+245105us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.951359 (+ 2389us) write_op.cc:301] APPLY: starting
0811 18:57:11.951373 (+ 14us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.951419 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.951423 (+ 4us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.951490 (+ 67us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.951534 (+ 44us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:11.951544 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:11.951618 (+ 74us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.951877 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951890 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:11.951896 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.951935 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.951938 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":208608512,"child_traces":[["op",{"apply.queue_time_us":2348,"apply.run_cpu_time_us":421,"apply.run_wall_time_us":627,"num_ops":2,"prepare.queue_time_us":5585,"prepare.run_cpu_time_us":5480,"prepare.run_wall_time_us":28502,"raft.queue_time_us":9584,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":246945,"spinlock_wait_cycles":2499456,"thread_start_us":25961,"threads_started":1}]]}
I20260811 18:57:13.163357 3856 rpcz_store.cc:276] 0811 18:57:11.372485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.374329 (+ 1844us) service_pool.cc:227] Handling call
0811 18:57:11.659695 (+285366us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.659716 (+ 21us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.986291 (+1326575us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.052865 (+ 66574us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.053078 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.119066 3861 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 201) took 5771 ms. Trace:
I20260811 18:57:13.121114 3860 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 213) took 1748 ms. Trace:
I20260811 18:57:13.167724 3860 rpcz_store.cc:276] 0811 18:57:11.372246 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.376774 (+ 4528us) service_pool.cc:227] Handling call
0811 18:57:11.724237 (+347463us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.724261 (+ 24us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.101735 (+1377474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.120879 (+ 19144us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.121041 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260811 18:57:13.141623 3824 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.110463 3862 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37348 (request call id 216) took 1737 ms. Trace:
I20260811 18:57:13.168118 3862 rpcz_store.cc:276] 0811 18:57:11.372614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.374396 (+ 1782us) service_pool.cc:227] Handling call
0811 18:57:11.659540 (+285144us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:11.659577 (+ 37us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:12.965609 (+1306032us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.109894 (+144285us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.110131 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:13.168365 3824 leader_election.cc:302] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.163771 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.164542 4169 ts_tablet_manager.cc:1905] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.168823 4658 raft_consensus.cc:2741] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260811 18:57:13.169132 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_511d4c7dbd2b41ae95fc5f910147870d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.169559 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.169863 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.170006 3861 rpcz_store.cc:276] 0811 18:57:07.347057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.347162 (+ 105us) service_pool.cc:227] Handling call
0811 18:57:08.344002 (+996840us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0811 18:57:08.344028 (+ 26us) ts_tablet_manager.cc:676] Creating new metadata...
0811 18:57:13.091897 (+4747869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.118720 (+ 26823us) tablet_replica.cc:205] Creating consensus instance
0811 18:57:13.118874 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:12.780762 3550 rpcz_store.cc:276] 0811 18:57:07.157109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:09.712862 (+2555753us) service_pool.cc:227] Handling call
0811 18:57:10.859077 (+1146215us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:10.859839 (+ 762us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:11.842737 (+982898us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:11.842905 (+ 168us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:11.843049 (+ 144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:10.891801 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:10.891966 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:10.891976 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:10.891981 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:10.892110 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:10.892130 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:10.892138 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:10.892254 (+ 116us) tablet.cc:699] Row locks acquired
0811 18:57:10.892258 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:10.892315 (+ 57us) write_op.cc:270] Start()
0811 18:57:10.892361 (+ 46us) write_op.cc:276] Timestamp: P: 1786474630892286 usec, L: 0
0811 18:57:10.892368 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:10.892686 (+ 318us) log.cc:842] Serialized 433 byte log entry
0811 18:57:11.700434 (+807748us) spinlock_profiling.cc:243] Waited 938 us on lock 0x6160000226a0. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f0363390 00007f61f037aa59 00007f61f037a7c1 00007f61f7ea947d 00007f61f03c68a1 00007f61f03cb5ae 00007f61f03cb331 00007f61f7ea947d 00007f61efe03308 00007f61efdeb9c3
0811 18:57:11.796921 (+ 96487us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.839233 (+ 42312us) write_op.cc:301] APPLY: starting
0811 18:57:11.839266 (+ 33us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.839350 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.839355 (+ 5us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.839590 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.839658 (+ 68us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:11.839669 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.839852 (+ 183us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.841916 (+ 2064us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.842083 (+ 167us) write_op.cc:454] Released schema lock
0811 18:57:11.842271 (+ 188us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.903045 (+ 60774us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.903056 (+ 11us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42190,"apply.run_cpu_time_us":2034,"apply.run_wall_time_us":63946,"num_ops":2,"prepare.queue_time_us":31162,"prepare.run_cpu_time_us":2541,"prepare.run_wall_time_us":28111,"replication_time_us":904513,"spinlock_wait_cycles":3560960,"thread_start_us":101672,"threads_started":3,"wal-append.queue_time_us":24399}]]}
I20260811 18:57:12.946574 3551 catalog_manager.cc:6565] ReplaceTablet: tablet f61d3822fc2d4b569f223238a6464e0a deleted and replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:13.172159 3551 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 293) took 6014 ms. Trace:
I20260811 18:57:13.172366 3551 rpcz_store.cc:276] 0811 18:57:07.157840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.322305 (+4164465us) service_pool.cc:227] Handling call
0811 18:57:13.172103 (+1849798us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.374038 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.374210 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.374222 (+ 12us) write_op.cc:435] Acquired schema lock
0811 18:57:11.374226 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.374431 (+ 205us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.374447 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.374455 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.374576 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:11.374581 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.374634 (+ 53us) write_op.cc:270] Start()
0811 18:57:11.374681 (+ 47us) write_op.cc:276] Timestamp: P: 1786474631374615 usec, L: 0
0811 18:57:11.374686 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.374991 (+ 305us) log.cc:842] Serialized 622 byte log entry
0811 18:57:11.883917 (+508926us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.903236 (+ 19319us) write_op.cc:301] APPLY: starting
0811 18:57:11.903259 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.903309 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.903472 (+ 163us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.903625 (+ 153us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.903702 (+ 77us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.903713 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.903820 (+ 107us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.904716 (+ 896us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.904806 (+ 90us) write_op.cc:454] Released schema lock
0811 18:57:11.904878 (+ 72us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.905352 (+ 474us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.905359 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19250,"apply.run_cpu_time_us":1310,"apply.run_wall_time_us":2442,"num_ops":2,"prepare.queue_time_us":3352,"prepare.run_cpu_time_us":1114,"prepare.run_wall_time_us":1114,"replication_time_us":509206,"thread_start_us":31504,"threads_started":1}]]}
I20260811 18:57:13.173753 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c87170e0a947438da8d0091a78e401c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.162463 3546 rpcz_store.cc:276] 0811 18:57:08.553033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.662250 (+3109217us) service_pool.cc:227] Handling call
0811 18:57:11.662858 (+ 608us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:11.663101 (+ 243us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:11.663170 (+ 69us) catalog_manager.cc:2761] Locking tablets
0811 18:57:11.663361 (+ 191us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:11.695041 (+ 31680us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x616000021a78. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f61f17574fd 00007f61f174641a 00007f61f16eeef8 00007f61f16ee8b9 00007f61f7f9cf93 00007f61f7f9e47e 00007f61f7d58d4c 00007f61f7d4dc70 00007f61f7d4c1bf 00007f61f7f41446
0811 18:57:11.934199 (+239158us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:11.934322 (+ 123us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:11.934400 (+ 78us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:12.920176 (+985776us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:11.696137 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:11.696269 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:11.696279 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:11.696283 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:11.696420 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:11.696434 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:11.696442 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:11.696688 (+ 246us) tablet.cc:699] Row locks acquired
0811 18:57:11.696693 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:11.696748 (+ 55us) write_op.cc:270] Start()
0811 18:57:11.696787 (+ 39us) write_op.cc:276] Timestamp: P: 1786474631696702 usec, L: 0
0811 18:57:11.696793 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:11.696998 (+ 205us) log.cc:842] Serialized 756 byte log entry
0811 18:57:11.915749 (+218751us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.932566 (+ 16817us) write_op.cc:301] APPLY: starting
0811 18:57:11.932585 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.932644 (+ 59us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.932723 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.932824 (+ 101us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.932870 (+ 46us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:11.932881 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:11.932984 (+ 103us) log.cc:842] Serialized 29 byte log entry
0811 18:57:11.933785 (+ 801us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.933869 (+ 84us) write_op.cc:454] Released schema lock
0811 18:57:11.933940 (+ 71us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.934257 (+ 317us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.934323 (+ 66us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":70439168,"child_traces":[["op",{"apply.queue_time_us":16772,"apply.run_cpu_time_us":1142,"apply.run_wall_time_us":2031,"num_ops":2,"prepare.queue_time_us":1023,"prepare.run_cpu_time_us":926,"prepare.run_wall_time_us":925,"replication_time_us":218936}]]}
I20260811 18:57:13.162505 3548 rpcz_store.cc:276] 0811 18:57:07.157709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:10.865282 (+3707573us) service_pool.cc:227] Handling call
0811 18:57:10.865753 (+ 471us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:10.866086 (+ 333us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:10.868398 (+ 2312us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:10.868536 (+ 138us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:11.932469 (+1063933us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:12.790318 (+857849us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:10.919971 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:10.920148 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:10.920156 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:10.920161 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:10.920314 (+ 153us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:10.920333 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:10.920341 (+ 8us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:10.920433 (+ 92us) tablet.cc:699] Row locks acquired
0811 18:57:10.920437 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:10.920489 (+ 52us) write_op.cc:270] Start()
0811 18:57:10.920536 (+ 47us) write_op.cc:276] Timestamp: P: 1786474630920461 usec, L: 0
0811 18:57:10.920543 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:10.920827 (+ 284us) log.cc:842] Serialized 405 byte log entry
0811 18:57:11.840848 (+920021us) op_driver.cc:444] REPLICATION: finished
0811 18:57:11.879192 (+ 38344us) write_op.cc:301] APPLY: starting
0811 18:57:11.879215 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:11.879449 (+ 234us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:11.880107 (+ 658us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:11.881283 (+ 1176us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:11.881591 (+ 308us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:11.881668 (+ 77us) write_op.cc:312] APPLY: finished
0811 18:57:11.882942 (+ 1274us) log.cc:842] Serialized 23 byte log entry
0811 18:57:11.884740 (+ 1798us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.932325 (+ 47585us) write_op.cc:454] Released schema lock
0811 18:57:11.932338 (+ 13us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:11.932436 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:11.932442 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":38225,"apply.run_cpu_time_us":2248,"apply.run_wall_time_us":53335,"num_ops":1,"prepare.queue_time_us":50871,"prepare.run_cpu_time_us":945,"prepare.run_wall_time_us":944,"replication_time_us":920270,"thread_start_us":42902,"threads_started":1}]]}
I20260811 18:57:13.174809 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c33809efd2a943a3bfd700f8cf0750c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.175370 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dae19af473234626ac121fc618e92a4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.175590 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c33809efd2a943a3bfd700f8cf0750c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:13.175956 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dae19af473234626ac121fc618e92a4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:13.178987 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c87170e0a947438da8d0091a78e401c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.181710 3940 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:13.185855 3849 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 65) took 4630 ms. Trace:
I20260811 18:57:13.189114 4637 raft_consensus.cc:2781] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:13.190274 4556 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.191359 4556 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.198628 3676 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.186162 3849 rpcz_store.cc:276] 0811 18:57:08.555773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.555866 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:13.159914 (+4604048us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x6150001629a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:13.185700 (+ 25786us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":586254080}
I20260811 18:57:13.203586 3676 leader_election.cc:302] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.203666 4556 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.204286 4556 raft_consensus.cc:2781] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:13.205647 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:13.206919 4576 raft_consensus.cc:2781] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.207011 4556 raft_consensus.cc:686] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.207978 4169 ts_tablet_manager.cc:1918] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.208544 4637 raft_consensus.cc:686] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.208475 4556 consensus_queue.cc:237] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.209064 4637 consensus_queue.cc:237] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.218993 4576 raft_consensus.cc:686] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:13.219574 4576 consensus_queue.cc:237] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:13.228225 3676 leader_election.cc:341] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.235982 4169 log.cc:1196] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d106e0e1f4b845429b10f4e100f69a8b
I20260811 18:57:13.222106 4163 tablet_bootstrap.cc:492] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.237460 4163 ts_tablet_manager.cc:1397] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.426s user 0.011s sys 0.011s
I20260811 18:57:13.239217 4162 tablet_bootstrap.cc:654] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.239246 4163 raft_consensus.cc:348] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.239605 4163 raft_consensus.cc:374] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.239866 4163 raft_consensus.cc:729] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.251436 4163 consensus_queue.cc:260] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.256218 4163 ts_tablet_manager.cc:1428] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260811 18:57:13.260279 4163 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.264389 4163 tablet_bootstrap.cc:654] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.254071 4169 ts_tablet_manager.cc:1939] T d106e0e1f4b845429b10f4e100f69a8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.284416 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 186) took 6119 ms. Trace:
I20260811 18:57:13.295426 4169 rpcz_store.cc:276] 0811 18:57:07.164788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.167248 (+ 2460us) service_pool.cc:227] Handling call
0811 18:57:12.881170 (+5713922us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:12.881181 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.161591 (+280410us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x6150000c9024. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:13.187833 (+ 26242us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.207729 (+ 19896us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.284338 (+ 76609us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":501723520,"tablet-delete.queue_time_us":4571817}
I20260811 18:57:13.295974 4167 ts_tablet_manager.cc:1918] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.296280 4167 log.cc:1196] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5df7a07983174dd1aaf48951c9ff1a1b
I20260811 18:57:13.296358 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d106e0e1f4b845429b10f4e100f69a8b (table default.Table_8e4b85a9b7d5460b83544a3677cde81a [id=17b08a7402a34713a7329c297b221da9]) successfully deleted
I20260811 18:57:13.296916 4169 tablet_replica.cc:330] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.304411 4169 raft_consensus.cc:2229] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.304880 4169 raft_consensus.cc:2258] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.305879 4169 ts_tablet_manager.cc:1905] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.306319 4162 tablet_bootstrap.cc:492] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.306720 4162 ts_tablet_manager.cc:1397] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.149s user 0.007s sys 0.005s
I20260811 18:57:13.308143 4162 raft_consensus.cc:348] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.308810 4162 raft_consensus.cc:374] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.308997 4162 raft_consensus.cc:729] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.309716 4162 consensus_queue.cc:260] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.314314 4167 ts_tablet_manager.cc:1939] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.315169 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 143) took 7453 ms. Trace:
I20260811 18:57:13.315562 4167 rpcz_store.cc:276] 0811 18:57:05.861871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.863597 (+ 1726us) service_pool.cc:227] Handling call
0811 18:57:12.906106 (+7042509us) ts_tablet_manager.cc:1145] Deleting tablet 5df7a07983174dd1aaf48951c9ff1a1b
0811 18:57:12.906122 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.154080 (+247958us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.295658 (+141578us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.315117 (+ 19459us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2345,"tablet-delete.queue_time_us":7042212}
I20260811 18:57:13.316628 4167 tablet_replica.cc:330] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.316905 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5df7a07983174dd1aaf48951c9ff1a1b (table default.Table_e584229a7f164fc8be8d7963583c65c0 [id=370de44b5af74559b2f14860aff79907]) successfully deleted
I20260811 18:57:13.317132 4167 raft_consensus.cc:2229] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.317684 4167 raft_consensus.cc:2258] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.318830 4167 ts_tablet_manager.cc:1905] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.333734 4163 tablet_bootstrap.cc:492] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.334079 4163 ts_tablet_manager.cc:1397] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.003s
I20260811 18:57:13.335685 4163 raft_consensus.cc:348] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.336220 4163 raft_consensus.cc:374] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.336484 4163 raft_consensus.cc:729] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.336901 4163 consensus_queue.cc:260] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.334156 4162 ts_tablet_manager.cc:1428] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:13.346987 4163 ts_tablet_manager.cc:1428] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.013s user 0.000s sys 0.003s
I20260811 18:57:13.349975 4163 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.351442 4162 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.366493 4162 tablet_bootstrap.cc:654] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.362299 4169 ts_tablet_manager.cc:1918] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.393146 4169 log.cc:1196] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2852f9e5f67c4025836018af553388a5
I20260811 18:57:13.395973 4169 ts_tablet_manager.cc:1939] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.401141 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9292c8eb06fb4a0d8e33b6796398b740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.402561 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 188) took 6148 ms. Trace:
W20260811 18:57:13.404186 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9292c8eb06fb4a0d8e33b6796398b740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.402694 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
I20260811 18:57:13.404874 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:13.405205 4169 rpcz_store.cc:276] 0811 18:57:07.247876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.247995 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:13.296745 (+6048750us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:13.296763 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.328880 (+ 32117us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.362133 (+ 33253us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.396659 (+ 34526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4987213}
I20260811 18:57:13.402882 4163 tablet_bootstrap.cc:654] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.406821 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2852f9e5f67c4025836018af553388a5 (table default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]) successfully deleted
I20260811 18:57:13.407227 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet d106e0e1f4b845429b10f4e100f69a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.411258 4169 tablet_replica.cc:330] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.411299 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c09652d9e408880ca606ae0b30773 with delete_type TABLET_DATA_DELETED (replaced by tablet a300a7aca2814da6b999eb4d865fea66) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.411551 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84" } modify_external_catalogs: true
I20260811 18:57:13.411708 4169 raft_consensus.cc:2229] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.411860 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a0c4cb29b3ec48749d83adaa68477c84" } modify_external_catalogs: true
I20260811 18:57:13.413919 4169 raft_consensus.cc:2258] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.414942 4169 ts_tablet_manager.cc:1905] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.423571 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.427713 4658 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.428050 4162 tablet_bootstrap.cc:492] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.428022 4658 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.428258 4162 ts_tablet_manager.cc:1397] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.001s
I20260811 18:57:13.428889 4658 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.429888 4162 raft_consensus.cc:348] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.430065 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.430460 4162 raft_consensus.cc:374] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.430572 3744 raft_consensus.cc:2454] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:13.433139 4162 raft_consensus.cc:729] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.433792 3824 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.434479 4658 raft_consensus.cc:2781] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:13.434748 4658 raft_consensus.cc:482] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:13.435081 4658 raft_consensus.cc:3037] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.435075 4162 consensus_queue.cc:260] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.440433 4658 raft_consensus.cc:504] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.440974 4443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43838 (request call id 0) took 4835 ms. Trace:
I20260811 18:57:13.442373 4658 leader_election.cc:288] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.442437 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:13.443089 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:13.443938 4655 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:13.444655 4655 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
W20260811 18:57:13.447786 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 799ba9a044e145ec89d0025c03f391ea consensus state has no leader
I20260811 18:57:13.448135 3554 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 299) took 4887 ms. Trace:
I20260811 18:57:13.448308 3554 rpcz_store.cc:276] 0811 18:57:08.560504 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.154949 (+4594445us) service_pool.cc:227] Handling call
0811 18:57:13.155417 (+ 468us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.155665 (+ 248us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.155887 (+ 222us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.155979 (+ 92us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.447369 (+291390us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.447998 (+ 629us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.229301 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.229429 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.229436 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.229439 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.229559 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.229575 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.229580 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.229661 (+ 81us) tablet.cc:699] Row locks acquired
0811 18:57:13.229664 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.229706 (+ 42us) write_op.cc:270] Start()
0811 18:57:13.229743 (+ 37us) write_op.cc:276] Timestamp: P: 1786474633229673 usec, L: 0
0811 18:57:13.229746 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.230007 (+ 261us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.444257 (+214250us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.445328 (+ 1071us) write_op.cc:301] APPLY: starting
0811 18:57:13.445350 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.445395 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.445423 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.445485 (+ 62us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.445522 (+ 37us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.445528 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.445625 (+ 97us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.447145 (+ 1520us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.447167 (+ 22us) write_op.cc:454] Released schema lock
0811 18:57:13.447175 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":996,"num_ops":1,"prepare.queue_time_us":72678,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":784,"replication_time_us":214476}]]}
W20260811 18:57:13.452656 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:13.452972 3547 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 300) took 4892 ms. Trace:
I20260811 18:57:13.453269 3547 rpcz_store.cc:276] 0811 18:57:08.560750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.154998 (+4594248us) service_pool.cc:227] Handling call
0811 18:57:13.155724 (+ 726us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.155936 (+ 212us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.156113 (+ 177us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.156226 (+ 113us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.452111 (+295885us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.452841 (+ 730us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.230132 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.230265 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.230273 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.230278 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.240800 (+ 10522us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.240823 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.240834 (+ 11us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.241001 (+ 167us) tablet.cc:699] Row locks acquired
0811 18:57:13.241006 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:13.241071 (+ 65us) write_op.cc:270] Start()
0811 18:57:13.241126 (+ 55us) write_op.cc:276] Timestamp: P: 1786474633241029 usec, L: 0
0811 18:57:13.241133 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.241497 (+ 364us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.444353 (+202856us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.450723 (+ 6370us) write_op.cc:301] APPLY: starting
0811 18:57:13.450742 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.450787 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.450830 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.450895 (+ 65us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.450939 (+ 44us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.450949 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:13.451035 (+ 86us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.451950 (+ 915us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.451971 (+ 21us) write_op.cc:454] Released schema lock
0811 18:57:13.451981 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.452067 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.452073 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6321,"apply.run_cpu_time_us":590,"apply.run_wall_time_us":1421,"num_ops":1,"prepare.queue_time_us":73414,"prepare.run_cpu_time_us":1168,"prepare.run_wall_time_us":11469,"replication_time_us":203191,"threads_started":1}]]}
I20260811 18:57:13.454771 4655 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:13.442518 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c8443df21949ff9bdc63fa6aa34108" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:13.457726 3744 raft_consensus.cc:3037] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.460044 4162 ts_tablet_manager.cc:1428] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.030s user 0.002s sys 0.002s
I20260811 18:57:13.432768 4167 ts_tablet_manager.cc:1918] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.461547 4167 log.cc:1196] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8efb663060ef457cbece962fce701eb1
I20260811 18:57:13.462180 4162 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.466747 3744 raft_consensus.cc:2454] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:13.469419 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
W20260811 18:57:13.470714 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 125757b8c1a6448b841c63339f392339 consensus state has no leader
I20260811 18:57:13.470942 3555 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:38308 (request call id 302) took 4784 ms. Trace:
I20260811 18:57:13.471446 3549 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 303) took 3554 ms. Trace:
I20260811 18:57:13.471719 3549 rpcz_store.cc:276] 0811 18:57:09.916809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.163536 (+3246727us) service_pool.cc:227] Handling call
0811 18:57:13.169303 (+ 5767us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.169963 (+ 660us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.470876 (+300913us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.471003 (+ 127us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.471158 (+ 155us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.243636 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.243717 (+ 81us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.243722 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:13.243725 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.243809 (+ 84us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.243817 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.243821 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.243878 (+ 57us) tablet.cc:699] Row locks acquired
0811 18:57:13.243882 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.243909 (+ 27us) write_op.cc:270] Start()
0811 18:57:13.243932 (+ 23us) write_op.cc:276] Timestamp: P: 1786474633243879 usec, L: 0
0811 18:57:13.243935 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.244068 (+ 133us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.467514 (+223446us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.469818 (+ 2304us) write_op.cc:301] APPLY: starting
0811 18:57:13.469833 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.469870 (+ 37us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.469932 (+ 62us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.470356 (+ 424us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.470399 (+ 43us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.470409 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:13.470484 (+ 75us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.470742 (+ 258us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.470763 (+ 21us) write_op.cc:454] Released schema lock
0811 18:57:13.470772 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.470838 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.470844 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2267,"apply.run_cpu_time_us":669,"apply.run_wall_time_us":1076,"num_ops":2,"prepare.queue_time_us":73034,"prepare.run_cpu_time_us":491,"prepare.run_wall_time_us":489,"replication_time_us":223560,"threads_started":1}]]}
I20260811 18:57:13.472764 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.472906 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.470248 3553 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 301) took 4909 ms. Trace:
I20260811 18:57:13.473605 3553 rpcz_store.cc:276] 0811 18:57:08.560905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.156843 (+4595938us) service_pool.cc:227] Handling call
0811 18:57:13.157735 (+ 892us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.157898 (+ 163us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.157957 (+ 59us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.158232 (+ 275us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:13.469065 (+310833us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:13.469170 (+ 105us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:13.469241 (+ 71us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:13.470040 (+ 799us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.241668 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.241889 (+ 221us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.241899 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:13.241904 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.242061 (+ 157us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.242077 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.242084 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.242489 (+ 405us) tablet.cc:699] Row locks acquired
0811 18:57:13.242495 (+ 6us) write_op.cc:260] PREPARE: finished
0811 18:57:13.242545 (+ 50us) write_op.cc:270] Start()
0811 18:57:13.242587 (+ 42us) write_op.cc:276] Timestamp: P: 1786474633242507 usec, L: 0
0811 18:57:13.242593 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.242886 (+ 293us) log.cc:842] Serialized 725 byte log entry
0811 18:57:13.467364 (+224478us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.467868 (+ 504us) write_op.cc:301] APPLY: starting
0811 18:57:13.467891 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.467944 (+ 53us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.468002 (+ 58us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.468108 (+ 106us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.468157 (+ 49us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:13.468168 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.468264 (+ 96us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.468595 (+ 331us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.468621 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:13.468632 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.468706 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.468712 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":423,"apply.run_cpu_time_us":674,"apply.run_wall_time_us":919,"num_ops":2,"prepare.queue_time_us":82465,"prepare.run_cpu_time_us":1101,"prepare.run_wall_time_us":1288,"replication_time_us":224735}]]}
I20260811 18:57:13.471139 3555 rpcz_store.cc:276] 0811 18:57:08.685984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.159850 (+4473866us) service_pool.cc:227] Handling call
0811 18:57:13.160428 (+ 578us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.160638 (+ 210us) catalog_manager.cc:3616] Apply alter schema
0811 18:57:13.160841 (+ 203us) catalog_manager.cc:3639] Acquired catalog manager lock
0811 18:57:13.160924 (+ 83us) catalog_manager.cc:3786] Updating metadata on disk
0811 18:57:13.470449 (+309525us) catalog_manager.cc:3817] Committing alterations to in-memory state
0811 18:57:13.470872 (+ 423us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.242992 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.243128 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.243133 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:13.243136 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:13.243225 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.243233 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.243238 (+ 5us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:13.243292 (+ 54us) tablet.cc:699] Row locks acquired
0811 18:57:13.243295 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.243324 (+ 29us) write_op.cc:270] Start()
0811 18:57:13.243350 (+ 26us) write_op.cc:276] Timestamp: P: 1786474633243292 usec, L: 0
0811 18:57:13.243356 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.243491 (+ 135us) log.cc:842] Serialized 405 byte log entry
0811 18:57:13.467465 (+223974us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.468839 (+ 1374us) write_op.cc:301] APPLY: starting
0811 18:57:13.468855 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.468894 (+ 39us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.468925 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.469065 (+ 140us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.469120 (+ 55us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:13.469131 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.469210 (+ 79us) log.cc:842] Serialized 23 byte log entry
0811 18:57:13.469628 (+ 418us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.469647 (+ 19us) write_op.cc:454] Released schema lock
0811 18:57:13.469656 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.469717 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.469723 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1323,"apply.run_cpu_time_us":535,"apply.run_wall_time_us":939,"num_ops":1,"prepare.queue_time_us":81490,"prepare.run_cpu_time_us":550,"prepare.run_wall_time_us":549,"replication_time_us":224090}]]}
I20260811 18:57:13.475342 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet e5fd9fce095e4a769dbb38c7903359cc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.476998 3824 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.477720 4658 raft_consensus.cc:2781] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.478263 4658 raft_consensus.cc:686] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:13.478652 4658 consensus_queue.cc:237] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.482007 4162 tablet_bootstrap.cc:654] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.484774 4167 ts_tablet_manager.cc:1939] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.500510 4163 tablet_bootstrap.cc:492] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.500862 4163 ts_tablet_manager.cc:1397] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.151s user 0.008s sys 0.001s
I20260811 18:57:13.502545 4163 raft_consensus.cc:348] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.503026 4163 raft_consensus.cc:374] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.503279 4163 raft_consensus.cc:729] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.504020 4163 consensus_queue.cc:260] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.504935 4163 ts_tablet_manager.cc:1428] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:13.505388 4163 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.509908 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 144) took 7627 ms. Trace:
I20260811 18:57:13.519702 3546 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 305) took 1637 ms. Trace:
I20260811 18:57:13.520025 3546 rpcz_store.cc:276] 0811 18:57:11.881979 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.174483 (+1292504us) service_pool.cc:227] Handling call
0811 18:57:13.175679 (+ 1196us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.176284 (+ 605us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.519130 (+342846us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.519331 (+ 201us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.519458 (+ 127us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.244170 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.244343 (+ 173us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.244351 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.244356 (+ 5us) tablet.cc:660] Decoding operations
0811 18:57:13.244450 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.244461 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.244468 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.244554 (+ 86us) tablet.cc:699] Row locks acquired
0811 18:57:13.244558 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.244589 (+ 31us) write_op.cc:270] Start()
0811 18:57:13.244618 (+ 29us) write_op.cc:276] Timestamp: P: 1786474633244557 usec, L: 0
0811 18:57:13.244624 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.244799 (+ 175us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.513302 (+268503us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.513607 (+ 305us) write_op.cc:301] APPLY: starting
0811 18:57:13.513640 (+ 33us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.513718 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.513775 (+ 57us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.514041 (+ 266us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.514086 (+ 45us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.514092 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.514272 (+ 180us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.518001 (+ 3729us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.518053 (+ 52us) write_op.cc:454] Released schema lock
0811 18:57:13.518067 (+ 14us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.518188 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.518194 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":227,"apply.run_cpu_time_us":1106,"apply.run_wall_time_us":4674,"num_ops":2,"prepare.queue_time_us":67181,"prepare.run_cpu_time_us":685,"prepare.run_wall_time_us":683,"replication_time_us":268633}]]}
I20260811 18:57:13.521090 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 405773849fb94800be61ad8a7a6055a4 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.521818 3548 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 306) took 1639 ms. Trace:
I20260811 18:57:13.522035 3548 rpcz_store.cc:276] 0811 18:57:11.882163 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.175053 (+1292890us) service_pool.cc:227] Handling call
0811 18:57:13.176049 (+ 996us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.176616 (+ 567us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.521346 (+344730us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.521493 (+ 147us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.521616 (+ 123us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.244902 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.245009 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.245017 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.245021 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.245113 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.245120 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.245124 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.245197 (+ 73us) tablet.cc:699] Row locks acquired
0811 18:57:13.245202 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:13.245230 (+ 28us) write_op.cc:270] Start()
0811 18:57:13.245259 (+ 29us) write_op.cc:276] Timestamp: P: 1786474633245199 usec, L: 0
0811 18:57:13.245262 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.245418 (+ 156us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.513472 (+268054us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.518372 (+ 4900us) write_op.cc:301] APPLY: starting
0811 18:57:13.518392 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.518454 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.518555 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.518730 (+ 175us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.518787 (+ 57us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.518798 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.518918 (+ 120us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.519459 (+ 541us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.519486 (+ 27us) write_op.cc:454] Released schema lock
0811 18:57:13.519497 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.519738 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.519744 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":9472,"child_traces":[["op",{"apply.queue_time_us":4808,"apply.run_cpu_time_us":884,"apply.run_wall_time_us":1478,"num_ops":2,"prepare.queue_time_us":67882,"prepare.run_cpu_time_us":579,"prepare.run_wall_time_us":577,"replication_time_us":268139}]]}
I20260811 18:57:13.523504 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 125757b8c1a6448b841c63339f392339 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.531648 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]). Expected version 2 got 0
I20260811 18:57:13.532048 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.533983 3551 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 304) took 1652 ms. Trace:
I20260811 18:57:13.534425 3551 rpcz_store.cc:276] 0811 18:57:11.881785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.173409 (+1291624us) service_pool.cc:227] Handling call
0811 18:57:13.179112 (+ 5703us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.179564 (+ 452us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.533493 (+353929us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.533679 (+ 186us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.533834 (+ 155us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.245535 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.245652 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.245660 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:13.245664 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.245778 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.245790 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.245797 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.245888 (+ 91us) tablet.cc:699] Row locks acquired
0811 18:57:13.245892 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.245926 (+ 34us) write_op.cc:270] Start()
0811 18:57:13.245957 (+ 31us) write_op.cc:276] Timestamp: P: 1786474633245892 usec, L: 0
0811 18:57:13.245963 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.246217 (+ 254us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.519917 (+273700us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.520019 (+ 102us) write_op.cc:301] APPLY: starting
0811 18:57:13.520036 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.520099 (+ 63us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.520146 (+ 47us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.520359 (+ 213us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.520408 (+ 49us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.520419 (+ 11us) write_op.cc:312] APPLY: finished
0811 18:57:13.520522 (+ 103us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.521558 (+ 1036us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.521583 (+ 25us) write_op.cc:454] Released schema lock
0811 18:57:13.521593 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:13.521681 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.521687 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48,"apply.run_cpu_time_us":806,"apply.run_wall_time_us":1727,"num_ops":2,"prepare.queue_time_us":65321,"prepare.run_cpu_time_us":749,"prepare.run_wall_time_us":748,"replication_time_us":273929}]]}
I20260811 18:57:13.535766 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet a0ba169c32ef48b984e649b211ee2af1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.537520 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5332d093ac81423fa33cb860f7d9a460" } new_table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" modify_external_catalogs: true
I20260811 18:57:13.541165 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3392d9de7234297a490a976dfe10060" } modify_external_catalogs: true
I20260811 18:57:13.541610 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3392d9de7234297a490a976dfe10060" } modify_external_catalogs: true
I20260811 18:57:13.542629 4658 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.542693 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_46eb50be61c143f2930a20efd2cd56b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.542945 4658 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:13.543783 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46eb50be61c143f2930a20efd2cd56b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.544108 3552 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:38308 (request call id 307) took 1661 ms. Trace:
I20260811 18:57:13.544691 4658 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.544409 3552 rpcz_store.cc:276] 0811 18:57:11.882761 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.400448 (+1517687us) service_pool.cc:227] Handling call
0811 18:57:13.407237 (+ 6789us) catalog_manager.cc:2075] Acquired catalog manager lock
0811 18:57:13.408069 (+ 832us) catalog_manager.cc:2121] Created new table and tablet info
0811 18:57:13.543785 (+135716us) catalog_manager.cc:2181] Wrote table and tablets to system table
0811 18:57:13.543908 (+ 123us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0811 18:57:13.544024 (+ 116us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.408936 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.409092 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.409102 (+ 10us) write_op.cc:435] Acquired schema lock
0811 18:57:13.409106 (+ 4us) tablet.cc:660] Decoding operations
0811 18:57:13.409291 (+ 185us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.409308 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.409315 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.409434 (+ 119us) tablet.cc:699] Row locks acquired
0811 18:57:13.409438 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:13.409492 (+ 54us) write_op.cc:270] Start()
0811 18:57:13.409537 (+ 45us) write_op.cc:276] Timestamp: P: 1786474633409452 usec, L: 0
0811 18:57:13.409544 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.409818 (+ 274us) log.cc:842] Serialized 433 byte log entry
0811 18:57:13.543011 (+133193us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.543109 (+ 98us) write_op.cc:301] APPLY: starting
0811 18:57:13.543125 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.543197 (+ 72us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.543269 (+ 72us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.543427 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.543463 (+ 36us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0811 18:57:13.543469 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.543551 (+ 82us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.543656 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.543680 (+ 24us) write_op.cc:454] Released schema lock
0811 18:57:13.543692 (+ 12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":42,"num_ops":2,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":991,"prepare.run_wall_time_us":991,"replication_time_us":133439}]]}
I20260811 18:57:13.545043 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:13.547976 3824 leader_election.cc:341] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:13.510160 4167 rpcz_store.cc:276] 0811 18:57:05.865078 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.865240 (+ 162us) service_pool.cc:227] Handling call
0811 18:57:13.316414 (+7451174us) ts_tablet_manager.cc:1145] Deleting tablet 8efb663060ef457cbece962fce701eb1
0811 18:57:13.316430 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.374156 (+ 57726us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.432625 (+ 58469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.489672 (+ 57047us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7450872}
I20260811 18:57:13.548826 4167 tablet_replica.cc:330] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.549222 4167 raft_consensus.cc:2229] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.549636 4167 raft_consensus.cc:2258] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.550640 4167 ts_tablet_manager.cc:1905] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.550598 4592 consensus_queue.cc:1035] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:13.551260 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8efb663060ef457cbece962fce701eb1 (table default.Table_6300adb327414f04a6ef25bfe7da7e02 [id=326e36c59f7849bca00167e55e335396]) successfully deleted
I20260811 18:57:13.556663 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 490f34d7a403416fbab64e46973746de
I20260811 18:57:13.557101 3796 heartbeater.cc:347] Connected to a master server at 127.3.72.125:38889
I20260811 18:57:13.557536 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 308) took 1667 ms. Trace:
I20260811 18:57:13.557893 3550 rpcz_store.cc:276] 0811 18:57:11.889647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.411120 (+1521473us) service_pool.cc:227] Handling call
0811 18:57:13.412038 (+ 918us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.412257 (+ 219us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.412326 (+ 69us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.412555 (+ 229us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:13.556323 (+143768us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:13.556443 (+ 120us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:13.556504 (+ 61us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:13.557380 (+ 876us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.413741 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.413894 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.413901 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.413914 (+ 13us) tablet.cc:660] Decoding operations
0811 18:57:13.414073 (+ 159us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.414086 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.414091 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.414275 (+ 184us) tablet.cc:699] Row locks acquired
0811 18:57:13.414278 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.414324 (+ 46us) write_op.cc:270] Start()
0811 18:57:13.414995 (+ 671us) write_op.cc:276] Timestamp: P: 1786474633414287 usec, L: 0
0811 18:57:13.415000 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.415231 (+ 231us) log.cc:842] Serialized 725 byte log entry
0811 18:57:13.555485 (+140254us) op_driver.cc:444] REPLICATION: finished
0811 18:57:13.555637 (+ 152us) write_op.cc:301] APPLY: starting
0811 18:57:13.555654 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:13.555709 (+ 55us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:13.555794 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:13.555884 (+ 90us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:13.555930 (+ 46us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:13.555936 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:13.556063 (+ 127us) log.cc:842] Serialized 29 byte log entry
0811 18:57:13.556177 (+ 114us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:13.556197 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:13.556205 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"num_ops":2,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":986,"prepare.run_wall_time_us":1572,"replication_time_us":140448}]]}
I20260811 18:57:13.561574 4163 tablet_bootstrap.cc:654] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.564535 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.564441 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.582893 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 125757b8c1a6448b841c63339f392339
I20260811 18:57:13.583564 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 125757b8c1a6448b841c63339f392339 deleted and replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:13.584197 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_77c2e3d67a494888be8b3405d4a06cf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.585074 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77c2e3d67a494888be8b3405d4a06cf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.587833 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.589478 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cec1a44bf54f48019648c9338236fcc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.590325 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cec1a44bf54f48019648c9338236fcc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.595247 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.602370 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:13.603245 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]). Expected version 2 got 0
I20260811 18:57:13.603835 3551 catalog_manager.cc:6565] ReplaceTablet: tablet a0ba169c32ef48b984e649b211ee2af1 deleted and replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:13.605499 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba169c32ef48b984e649b211ee2af1 with delete_type TABLET_DATA_DELETED (replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.608748 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ba169c32ef48b984e649b211ee2af1 with delete_type TABLET_DATA_DELETED (replaced by tablet 6432abd4fc5c4f9ca95971ca4795e1cf) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.603698 3547 catalog_manager.cc:5495] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 1 to 2, leader changed from 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67) to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 2 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.609588 3547 catalog_manager.cc:5495] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 1 to 2, leader changed from 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67) to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 2 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.610446 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:13.610895 3547 catalog_manager.cc:5495] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:13.611500 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]). Expected version 2 got 0
I20260811 18:57:13.619586 4704 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:13.621838 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:13.622689 3555 catalog_manager.cc:6565] ReplaceTablet: tablet e5fd9fce095e4a769dbb38c7903359cc deleted and replaced by tablet 1b73a97b7abf4bd58c792426be6e938d
I20260811 18:57:13.623844 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_57bac5d4e666472fbab036524cbbed7d" } new_table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" modify_external_catalogs: true
I20260811 18:57:13.624265 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.626677 4162 tablet_bootstrap.cc:492] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.627148 4162 ts_tablet_manager.cc:1397] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.165s user 0.007s sys 0.004s
I20260811 18:57:13.628805 4162 raft_consensus.cc:348] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.629249 4162 raft_consensus.cc:374] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.629488 4163 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.629776 4162 raft_consensus.cc:729] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.629835 4163 ts_tablet_manager.cc:1397] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.000s sys 0.008s
I20260811 18:57:13.630508 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.631237 4163 raft_consensus.cc:348] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.630592 4162 consensus_queue.cc:260] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.631654 4163 raft_consensus.cc:374] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.631801 4163 raft_consensus.cc:729] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.632349 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea" } modify_external_catalogs: true
I20260811 18:57:13.632244 4163 consensus_queue.cc:260] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.632975 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_575c5aca059048d68252cca6583a1f67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.636691 4162 ts_tablet_manager.cc:1428] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260811 18:57:13.642937 4162 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.636406 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44def5389adf4d9ca825712dfb862206" } modify_external_catalogs: true
I20260811 18:57:13.645272 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44def5389adf4d9ca825712dfb862206" } modify_external_catalogs: true
I20260811 18:57:13.647336 4658 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.647696 4658 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.648703 4658 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.650171 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:13.650646 3744 raft_consensus.cc:2454] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:13.636658 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ac4a8df2cd7346ffa89549c0750760ea" } modify_external_catalogs: true
I20260811 18:57:13.651857 3824 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:13.652382 4658 raft_consensus.cc:2781] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:13.652523 4658 raft_consensus.cc:482] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:13.653267 4658 raft_consensus.cc:3037] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.656355 4162 tablet_bootstrap.cc:654] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:13.634315 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_575c5aca059048d68252cca6583a1f67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.662536 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4dfa50c8631f4b70b6b743837d5a9032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.663880 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dfa50c8631f4b70b6b743837d5a9032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.664041 4163 ts_tablet_manager.cc:1428] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:13.664716 4163 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:13.670598 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet ef393c341bb54baf87176db5b5254ec7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.672142 4167 ts_tablet_manager.cc:1918] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.672119 4169 ts_tablet_manager.cc:1918] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:13.672313 4167 log.cc:1196] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/08b8dcf7a56043efbbd40bb888616492
I20260811 18:57:13.672375 4169 log.cc:1196] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:13.672914 4167 ts_tablet_manager.cc:1939] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.673187 4162 tablet_bootstrap.cc:492] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.673220 4169 ts_tablet_manager.cc:1939] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.673483 4162 ts_tablet_manager.cc:1397] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.005s
I20260811 18:57:13.675022 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 145) took 7809 ms. Trace:
I20260811 18:57:13.677207 4167 rpcz_store.cc:276] 0811 18:57:05.865275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:05.865355 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:13.548531 (+7683176us) ts_tablet_manager.cc:1145] Deleting tablet 08b8dcf7a56043efbbd40bb888616492
0811 18:57:13.548546 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.576546 (+ 28000us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.671908 (+ 95362us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.674980 (+ 3072us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7682834}
I20260811 18:57:13.678050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 159) took 7410 ms. Trace:
I20260811 18:57:13.678269 4167 rpcz_store.cc:276] 0811 18:57:06.267916 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.268001 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:13.677784 (+7409783us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:13.677798 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.677970 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7409312}
W20260811 18:57:13.679384 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 01d564ad80064912a8ce8e31c1b6c4d2 consensus state has no leader
I20260811 18:57:13.679390 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 160) took 7360 ms. Trace:
I20260811 18:57:13.679661 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 08b8dcf7a56043efbbd40bb888616492 (table default.Table_3444887e982747ff9c397d48da9ae8c0 [id=652e9766fc574e28bf20b3bf53b591f6]) successfully deleted
I20260811 18:57:13.679666 4167 rpcz_store.cc:276] 0811 18:57:06.319248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.319896 (+ 648us) service_pool.cc:227] Handling call
0811 18:57:13.679218 (+7359322us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:13.679233 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.679354 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7358386}
W20260811 18:57:13.680050 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d6daff4b7540309ff58755758e9752 already in progress: creating tablet
W20260811 18:57:13.680400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f25ff8a9e3a944269e9799995c73ebe0 already in progress: creating tablet
I20260811 18:57:13.680464 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 172) took 6898 ms. Trace:
I20260811 18:57:13.680650 4167 rpcz_store.cc:276] 0811 18:57:06.781802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.781879 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:13.680224 (+6898345us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:13.680236 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.680389 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6897169}
W20260811 18:57:13.681234 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a300a7aca2814da6b999eb4d865fea66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a300a7aca2814da6b999eb4d865fea66 already in progress: creating tablet
I20260811 18:57:13.681700 4420 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" instance_seqno: 1786474620633424) as {username='slave'} at 127.3.72.66:50257; Asking this server to re-register.
I20260811 18:57:13.682138 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 173) took 6882 ms. Trace:
I20260811 18:57:13.682439 4167 rpcz_store.cc:276] 0811 18:57:06.799430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.799507 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:13.681565 (+6882058us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:13.681577 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.681785 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6881396}
W20260811 18:57:13.682999 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 828993fb34bb4c62bce2a9322d7c992a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 828993fb34bb4c62bce2a9322d7c992a already in progress: creating tablet
I20260811 18:57:13.683467 3940 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:13.683337 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bdb9cbaf32744ad7b14399222fb784f6" } new_table_name: "default.Table_0d93e11fb1464fcbb4999abaebc3d611" modify_external_catalogs: true
I20260811 18:57:13.686295 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 174) took 6838 ms. Trace:
I20260811 18:57:13.686599 4167 rpcz_store.cc:276] 0811 18:57:06.845125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.845191 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:13.683538 (+6838347us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:13.683550 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.683665 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6837618}
W20260811 18:57:13.687600 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e215759d48c24d8cb99bbc3fc9009a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e215759d48c24d8cb99bbc3fc9009a66 already in progress: creating tablet
I20260811 18:57:13.688077 4167 tablet_replica.cc:330] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.688932 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 189) took 6426 ms. Trace:
I20260811 18:57:13.689318 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2852f9e5f67c4025836018af553388a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 987b9ac6859444e3a86c56af786cc84e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.689314 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_335e9fdd802644cc93a0657a33e74605" } new_table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" modify_external_catalogs: true
I20260811 18:57:13.689364 4169 rpcz_store.cc:276] 0811 18:57:07.249545 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.249659 (+ 114us) service_pool.cc:227] Handling call
0811 18:57:13.411008 (+6161349us) ts_tablet_manager.cc:1145] Deleting tablet 8af3e74889b64c408c9ed9c54916f890
0811 18:57:13.411021 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.424749 (+ 13728us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.671900 (+247151us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.675778 (+ 3878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5101430}
I20260811 18:57:13.688529 4167 raft_consensus.cc:2229] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.689908 4169 tablet_replica.cc:330] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:13.690052 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8af3e74889b64c408c9ed9c54916f890 (table default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]) successfully deleted
I20260811 18:57:13.690294 4167 raft_consensus.cc:2258] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.690447 4169 raft_consensus.cc:2229] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:13.690913 4169 raft_consensus.cc:2258] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.691676 4167 ts_tablet_manager.cc:1905] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.692245 4169 ts_tablet_manager.cc:1905] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.698552 4423 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" instance_seqno: 1786474620110993) as {username='slave'} at 127.3.72.65:40455; Asking this server to re-register.
I20260811 18:57:13.701712 3796 heartbeater.cc:464] Registering TS with master...
W20260811 18:57:13.705026 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b3ec38f9b55489ba00ed76f9e4456f5 consensus state has no leader
I20260811 18:57:13.700306 4162 raft_consensus.cc:348] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.716173 4162 raft_consensus.cc:374] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.718685 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:13.716786 4162 raft_consensus.cc:729] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.719659 4162 consensus_queue.cc:260] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.720508 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.721724 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 867259e37fe44c33a35823ec2d896754
I20260811 18:57:13.725807 4162 ts_tablet_manager.cc:1428] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.035s user 0.000s sys 0.004s
I20260811 18:57:13.726478 4162 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.729589 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:13.729983 4162 tablet_bootstrap.cc:654] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.730568 4420 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:13.732438 4423 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:13.738698 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0859cafbc76b4eae9fe4453842a95178"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:13.741366 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a671c5d0d75c49779b94baa2b89c9cef" } new_table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" modify_external_catalogs: true
I20260811 18:57:13.739760 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" } modify_external_catalogs: true
I20260811 18:57:13.746007 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2664a3dc787a4a4d98a334656bc584f3" } modify_external_catalogs: true
W20260811 18:57:13.740597 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0859cafbc76b4eae9fe4453842a95178 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.757792 4719 tablet.cc:1722] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:13.769804 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b7f48893a8614cacb6393af837f050f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.775899 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7f48893a8614cacb6393af837f050f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.790066 4658 raft_consensus.cc:504] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:13.793139 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe203387bc9c45ba82afe6b058aa5e5f consensus state has no leader
W20260811 18:57:13.797595 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 1240ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:13.799332 4658 leader_election.cc:288] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:13.802728 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f640c60e81b4a369458b1497a269c9c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:13.803891 4162 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.805228 4556 raft_consensus.cc:482] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:13.805662 3744 raft_consensus.cc:3037] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:13.806118 4556 raft_consensus.cc:504] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.806757 4162 ts_tablet_manager.cc:1397] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.078s user 0.003s sys 0.006s
I20260811 18:57:13.807161 4720 tablet.cc:1722] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:13.808480 4556 leader_election.cc:288] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:13.812098 3676 leader_election.cc:341] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:13.812846 4162 raft_consensus.cc:348] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.817711 4163 tablet_bootstrap.cc:654] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.819516 4162 raft_consensus.cc:374] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:13.819762 4162 raft_consensus.cc:729] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:13.820611 4162 consensus_queue.cc:260] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:13.823699 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 278 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
W20260811 18:57:13.824391 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9bc5448a555944dfba8ccae09e25cbe9 consensus state has no leader
I20260811 18:57:13.825238 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_efa46b4a159b4805b21635ef65051022"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.826292 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efa46b4a159b4805b21635ef65051022 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.830107 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_00f2f8bec82f44f3a42420818d7dc6a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.831875 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00f2f8bec82f44f3a42420818d7dc6a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.844980 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:13.846293 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e75a55498d4c4072968ee03aaf48b831"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.847009 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e75a55498d4c4072968ee03aaf48b831 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.846819 3550 catalog_manager.cc:5495] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:13.847594 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:13.849684 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (DEFAULT_TABLE table=default.Table_dae19af473234626ac121fc618e92a4b [id=afadeb62ce0d4ceda9714b891c134737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.849937 3861 tablet_service.cc:1467] Processing CreateTablet for tablet b3f0cdcee681460084662e01152ad9dd (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.849756 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 (DEFAULT_TABLE table=default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.852821 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.852972 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.853420 4162 ts_tablet_manager.cc:1428] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.002s sys 0.002s
I20260811 18:57:13.853982 3722 tablet_service.cc:1467] Processing CreateTablet for tablet b3f0cdcee681460084662e01152ad9dd (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.855419 4162 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:13.859035 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 (DEFAULT_TABLE table=default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.860132 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (DEFAULT_TABLE table=default.Table_dae19af473234626ac121fc618e92a4b [id=afadeb62ce0d4ceda9714b891c134737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:13.860832 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af3e74889b64c408c9ed9c54916f890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:02 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.862090 4162 tablet_bootstrap.cc:654] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:13.886907 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 278 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:13.888021 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:13.888797 4655 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:13.889751 3744 raft_consensus.cc:2454] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:13.890894 4167 ts_tablet_manager.cc:1918] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.891240 4167 log.cc:1196] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d106e0e1f4b845429b10f4e100f69a8b
I20260811 18:57:13.893381 4167 ts_tablet_manager.cc:1939] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:13.895357 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f0cdcee681460084662e01152ad9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.895815 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebe3fed5aaf45918dbadfde06b2c24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.896189 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c07150e31994bdf9a00ac2e7a954691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.907356 4169 ts_tablet_manager.cc:1918] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:13.911188 3555 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1b73a97b7abf4bd58c792426be6e938d
I20260811 18:57:13.913022 3824 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:13.913395 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cbf6ce167fb348be8d61372791f0cbca consensus state has no leader
I20260811 18:57:13.913662 4658 raft_consensus.cc:2781] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:13.935737 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_267303174d744009a1803a363c1e431f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.936950 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_267303174d744009a1803a363c1e431f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.939234 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_13fbac538d16421e921aa37f54683662"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:13.940448 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13fbac538d16421e921aa37f54683662 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:13.943337 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e851b9ff216483bb8768c88e4b6e1b9" } modify_external_catalogs: true
I20260811 18:57:13.943702 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e851b9ff216483bb8768c88e4b6e1b9" } modify_external_catalogs: true
I20260811 18:57:13.951709 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:13.953792 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 9bc5448a555944dfba8ccae09e25cbe9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:13.954331 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:13.956473 4658 raft_consensus.cc:686] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:13.957080 4658 consensus_queue.cc:237] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:13.967084 4162 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:13.967983 4162 ts_tablet_manager.cc:1397] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.113s user 0.001s sys 0.009s
I20260811 18:57:13.969866 4169 log.cc:1196] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
I20260811 18:57:13.971570 4169 ts_tablet_manager.cc:1939] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:13.976089 4163 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:13.976431 4163 ts_tablet_manager.cc:1397] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.312s user 0.001s sys 0.009s
I20260811 18:57:13.982795 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f0cdcee681460084662e01152ad9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.987346 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebe3fed5aaf45918dbadfde06b2c24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.990610 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c07150e31994bdf9a00ac2e7a954691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:13.995344 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 175) took 7028 ms. Trace:
I20260811 18:57:13.995604 4167 rpcz_store.cc:276] 0811 18:57:06.966713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.967008 (+ 295us) service_pool.cc:227] Handling call
0811 18:57:13.687964 (+6720956us) ts_tablet_manager.cc:1145] Deleting tablet d106e0e1f4b845429b10f4e100f69a8b
0811 18:57:13.687978 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.802712 (+114734us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.890718 (+ 88006us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.995120 (+104402us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6720508}
I20260811 18:57:13.996711 4167 tablet_replica.cc:330] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:13.997069 4167 raft_consensus.cc:2229] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:13.997534 4167 raft_consensus.cc:2258] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:13.997790 3891 raft_consensus.cc:1261] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 18:57:13.998579 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d106e0e1f4b845429b10f4e100f69a8b (table default.Table_8e4b85a9b7d5460b83544a3677cde81a [id=17b08a7402a34713a7329c297b221da9]) successfully deleted
I20260811 18:57:13.998974 4556 consensus_queue.cc:1035] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 18:57:14.001370 4163 raft_consensus.cc:348] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.004766 4167 ts_tablet_manager.cc:1905] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:13.441311 4443 rpcz_store.cc:276] 0811 18:57:08.606364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.606645 (+ 281us) service_pool.cc:227] Handling call
0811 18:57:11.328738 (+2722093us) raft_consensus.cc:1457] Updating replica for 264 ops
0811 18:57:11.659409 (+330671us) raft_consensus.cc:1520] Early marking committed up to index 17
0811 18:57:11.827079 (+167670us) raft_consensus.cc:1525] Triggering prepare for 261 ops
0811 18:57:11.827658 (+ 579us) write_op.cc:270] Start()
0811 18:57:11.827748 (+ 90us) write_op.cc:276] Timestamp: P: 1786474621520868 usec, L: 0
0811 18:57:11.918883 (+ 91135us) write_op.cc:270] Start()
0811 18:57:11.918934 (+ 51us) write_op.cc:276] Timestamp: P: 1786474621522053 usec, L: 0
0811 18:57:11.919202 (+ 268us) write_op.cc:270] Start()
0811 18:57:11.919229 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621523450 usec, L: 0
0811 18:57:11.919381 (+ 152us) write_op.cc:270] Start()
0811 18:57:11.919403 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621524997 usec, L: 0
0811 18:57:11.919587 (+ 184us) write_op.cc:270] Start()
0811 18:57:11.919612 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621527610 usec, L: 0
0811 18:57:11.919872 (+ 260us) write_op.cc:270] Start()
0811 18:57:11.919899 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621528508 usec, L: 0
0811 18:57:11.920054 (+ 155us) write_op.cc:270] Start()
0811 18:57:11.920109 (+ 55us) write_op.cc:276] Timestamp: P: 1786474621529384 usec, L: 0
0811 18:57:11.920352 (+ 243us) write_op.cc:270] Start()
0811 18:57:11.920381 (+ 29us) write_op.cc:276] Timestamp: P: 1786474621544629 usec, L: 0
0811 18:57:11.920595 (+ 214us) write_op.cc:270] Start()
0811 18:57:11.920618 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621651974 usec, L: 0
0811 18:57:11.920865 (+ 247us) write_op.cc:270] Start()
0811 18:57:11.920892 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621652817 usec, L: 0
0811 18:57:11.921122 (+ 230us) write_op.cc:270] Start()
0811 18:57:11.921150 (+ 28us) write_op.cc:276] Timestamp: P: 1786474621676972 usec, L: 0
0811 18:57:11.921319 (+ 169us) write_op.cc:270] Start()
0811 18:57:11.921365 (+ 46us) write_op.cc:276] Timestamp: P: 1786474621677939 usec, L: 0
0811 18:57:11.921532 (+ 167us) write_op.cc:270] Start()
0811 18:57:11.921556 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621678607 usec, L: 0
0811 18:57:11.921780 (+ 224us) write_op.cc:270] Start()
0811 18:57:11.921807 (+ 27us) write_op.cc:276] Timestamp: P: 1786474621711143 usec, L: 0
0811 18:57:11.922019 (+ 212us) write_op.cc:270] Start()
0811 18:57:11.922044 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621727651 usec, L: 0
0811 18:57:11.922235 (+ 191us) write_op.cc:270] Start()
0811 18:57:11.922260 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621731810 usec, L: 0
0811 18:57:11.990413 (+ 68153us) write_op.cc:270] Start()
0811 18:57:11.990519 (+ 106us) write_op.cc:276] Timestamp: P: 1786474621748443 usec, L: 0
0811 18:57:11.990902 (+ 383us) write_op.cc:270] Start()
0811 18:57:11.990939 (+ 37us) write_op.cc:276] Timestamp: P: 1786474621751639 usec, L: 0
0811 18:57:11.991142 (+ 203us) write_op.cc:270] Start()
0811 18:57:11.991166 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621774538 usec, L: 0
0811 18:57:11.991325 (+ 159us) write_op.cc:270] Start()
0811 18:57:11.991349 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621788092 usec, L: 0
0811 18:57:11.991494 (+ 145us) write_op.cc:270] Start()
0811 18:57:11.991517 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621792386 usec, L: 0
0811 18:57:11.991664 (+ 147us) write_op.cc:270] Start()
0811 18:57:11.991686 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621814381 usec, L: 0
0811 18:57:11.991828 (+ 142us) write_op.cc:270] Start()
0811 18:57:11.991853 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621818940 usec, L: 0
0811 18:57:11.991990 (+ 137us) write_op.cc:270] Start()
0811 18:57:11.992012 (+ 22us) write_op.cc:276] Timestamp: P: 1786474621820953 usec, L: 0
0811 18:57:11.992186 (+ 174us) write_op.cc:270] Start()
0811 18:57:11.992209 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621824136 usec, L: 0
0811 18:57:11.992384 (+ 175us) write_op.cc:270] Start()
0811 18:57:11.992415 (+ 31us) write_op.cc:276] Timestamp: P: 1786474621826027 usec, L: 0
0811 18:57:11.992572 (+ 157us) write_op.cc:270] Start()
0811 18:57:11.992596 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621847361 usec, L: 0
0811 18:57:11.992735 (+ 139us) write_op.cc:270] Start()
0811 18:57:11.992752 (+ 17us) write_op.cc:276] Timestamp: P: 1786474621848770 usec, L: 0
0811 18:57:11.992908 (+ 156us) write_op.cc:270] Start()
0811 18:57:11.992944 (+ 36us) write_op.cc:276] Timestamp: P: 1786474621852710 usec, L: 0
0811 18:57:11.993098 (+ 154us) write_op.cc:270] Start()
0811 18:57:11.993122 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621853943 usec, L: 0
0811 18:57:11.993280 (+ 158us) write_op.cc:270] Start()
0811 18:57:11.993303 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621874157 usec, L: 0
0811 18:57:11.993468 (+ 165us) write_op.cc:270] Start()
0811 18:57:11.993492 (+ 24us) write_op.cc:276] Timestamp: P: 1786474621904587 usec, L: 0
0811 18:57:11.993666 (+ 174us) write_op.cc:270] Start()
0811 18:57:11.993689 (+ 23us) write_op.cc:276] Timestamp: P: 1786474621908927 usec, L: 0
0811 18:57:11.993826 (+ 137us) write_op.cc:270] Start()
0811 18:57:11.993851 (+ 25us) write_op.cc:276] Timestamp: P: 1786474621909657 usec, L: 0
0811 18:57:11.994005 (+ 154us) write_op.cc:270] Start()
0811 18:57:11.994028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622004715 usec, L: 0
0811 18:57:11.994181 (+ 153us) write_op.cc:270] Start()
0811 18:57:11.994207 (+ 26us) write_op.cc:276] Timestamp: P: 1786474622006470 usec, L: 0
0811 18:57:12.056002 (+ 61795us) write_op.cc:270] Start()
0811 18:57:12.059310 (+ 3308us) write_op.cc:276] Timestamp: P: 1786474622045673 usec, L: 0
0811 18:57:12.083380 (+ 24070us) write_op.cc:270] Start()
0811 18:57:12.086523 (+ 3143us) write_op.cc:276] Timestamp: P: 1786474622046391 usec, L: 0
0811 18:57:12.106318 (+ 19795us) write_op.cc:270] Start()
0811 18:57:12.110145 (+ 3827us) write_op.cc:276] Timestamp: P: 1786474622047787 usec, L: 0
0811 18:57:12.134923 (+ 24778us) write_op.cc:270] Start()
0811 18:57:12.140077 (+ 5154us) write_op.cc:276] Timestamp: P: 1786474622048501 usec, L: 0
0811 18:57:12.178586 (+ 38509us) write_op.cc:270] Start()
0811 18:57:12.178658 (+ 72us) write_op.cc:276] Timestamp: P: 1786474622049494 usec, L: 0
0811 18:57:12.179113 (+ 455us) write_op.cc:270] Start()
0811 18:57:12.179147 (+ 34us) write_op.cc:276] Timestamp: P: 1786474622067276 usec, L: 0
0811 18:57:12.179302 (+ 155us) write_op.cc:270] Start()
0811 18:57:12.179321 (+ 19us) write_op.cc:276] Timestamp: P: 1786474622081817 usec, L: 0
0811 18:57:12.179477 (+ 156us) write_op.cc:270] Start()
0811 18:57:12.179505 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622108942 usec, L: 0
0811 18:57:12.179654 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.179679 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622109557 usec, L: 0
0811 18:57:12.188784 (+ 9105us) write_op.cc:270] Start()
0811 18:57:12.189831 (+ 1047us) write_op.cc:276] Timestamp: P: 1786474622112463 usec, L: 0
0811 18:57:12.202322 (+ 12491us) write_op.cc:270] Start()
0811 18:57:12.205552 (+ 3230us) write_op.cc:276] Timestamp: P: 1786474622125688 usec, L: 0
0811 18:57:12.239304 (+ 33752us) write_op.cc:270] Start()
0811 18:57:12.242909 (+ 3605us) write_op.cc:276] Timestamp: P: 1786474622212794 usec, L: 0
0811 18:57:12.267796 (+ 24887us) write_op.cc:270] Start()
0811 18:57:12.271423 (+ 3627us) write_op.cc:276] Timestamp: P: 1786474622214897 usec, L: 0
0811 18:57:12.292568 (+ 21145us) write_op.cc:270] Start()
0811 18:57:12.294008 (+ 1440us) write_op.cc:276] Timestamp: P: 1786474622216185 usec, L: 0
0811 18:57:12.306470 (+ 12462us) write_op.cc:270] Start()
0811 18:57:12.309344 (+ 2874us) write_op.cc:276] Timestamp: P: 1786474622257586 usec, L: 0
0811 18:57:12.325469 (+ 16125us) write_op.cc:270] Start()
0811 18:57:12.326818 (+ 1349us) write_op.cc:276] Timestamp: P: 1786474622289720 usec, L: 0
0811 18:57:12.345826 (+ 19008us) write_op.cc:270] Start()
0811 18:57:12.349294 (+ 3468us) write_op.cc:276] Timestamp: P: 1786474622314198 usec, L: 0
0811 18:57:12.367562 (+ 18268us) write_op.cc:270] Start()
0811 18:57:12.371573 (+ 4011us) write_op.cc:276] Timestamp: P: 1786474622315614 usec, L: 0
0811 18:57:12.408088 (+ 36515us) write_op.cc:270] Start()
0811 18:57:12.412307 (+ 4219us) write_op.cc:276] Timestamp: P: 1786474622318020 usec, L: 0
0811 18:57:12.430256 (+ 17949us) write_op.cc:270] Start()
0811 18:57:12.433204 (+ 2948us) write_op.cc:276] Timestamp: P: 1786474622325784 usec, L: 0
0811 18:57:12.460123 (+ 26919us) write_op.cc:270] Start()
0811 18:57:12.463765 (+ 3642us) write_op.cc:276] Timestamp: P: 1786474622327031 usec, L: 0
0811 18:57:12.503004 (+ 39239us) write_op.cc:270] Start()
0811 18:57:12.507644 (+ 4640us) write_op.cc:276] Timestamp: P: 1786474622328122 usec, L: 0
0811 18:57:12.542953 (+ 35309us) write_op.cc:270] Start()
0811 18:57:12.543014 (+ 61us) write_op.cc:276] Timestamp: P: 1786474622353667 usec, L: 0
0811 18:57:12.543246 (+ 232us) write_op.cc:270] Start()
0811 18:57:12.543307 (+ 61us) write_op.cc:276] Timestamp: P: 1786474622355045 usec, L: 0
0811 18:57:12.543480 (+ 173us) write_op.cc:270] Start()
0811 18:57:12.543507 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622364171 usec, L: 0
0811 18:57:12.543691 (+ 184us) write_op.cc:270] Start()
0811 18:57:12.543719 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622364888 usec, L: 0
0811 18:57:12.543909 (+ 190us) write_op.cc:270] Start()
0811 18:57:12.543937 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622365415 usec, L: 0
0811 18:57:12.544136 (+ 199us) write_op.cc:270] Start()
0811 18:57:12.544163 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622366051 usec, L: 0
0811 18:57:12.544347 (+ 184us) write_op.cc:270] Start()
0811 18:57:12.544373 (+ 26us) write_op.cc:276] Timestamp: P: 1786474622366666 usec, L: 0
0811 18:57:12.544548 (+ 175us) write_op.cc:270] Start()
0811 18:57:12.544576 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622404720 usec, L: 0
0811 18:57:12.544747 (+ 171us) write_op.cc:270] Start()
0811 18:57:12.544775 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622412481 usec, L: 0
0811 18:57:12.544916 (+ 141us) write_op.cc:270] Start()
0811 18:57:12.544939 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622426342 usec, L: 0
0811 18:57:12.545101 (+ 162us) write_op.cc:270] Start()
0811 18:57:12.545126 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622437132 usec, L: 0
0811 18:57:12.545312 (+ 186us) write_op.cc:270] Start()
0811 18:57:12.545340 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622446932 usec, L: 0
0811 18:57:12.545509 (+ 169us) write_op.cc:270] Start()
0811 18:57:12.545537 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622447678 usec, L: 0
0811 18:57:12.545728 (+ 191us) write_op.cc:270] Start()
0811 18:57:12.545756 (+ 28us) write_op.cc:276] Timestamp: P: 1786474622448255 usec, L: 0
0811 18:57:12.545910 (+ 154us) write_op.cc:270] Start()
0811 18:57:12.545937 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622464675 usec, L: 0
0811 18:57:12.546102 (+ 165us) write_op.cc:270] Start()
0811 18:57:12.546131 (+ 29us) write_op.cc:276] Timestamp: P: 1786474622489315 usec, L: 0
0811 18:57:12.546288 (+ 157us) write_op.cc:270] Start()
0811 18:57:12.546312 (+ 24us) write_op.cc:276] Timestamp: P: 1786474622547221 usec, L: 0
0811 18:57:12.630867 (+ 84555us) write_op.cc:270] Start()
0811 18:57:12.630927 (+ 60us) write_op.cc:276] Timestamp: P: 1786474622552886 usec, L: 0
0811 18:57:12.631193 (+ 266us) write_op.cc:270] Start()
0811 18:57:12.631226 (+ 33us) write_op.cc:276] Timestamp: P: 1786474622686514 usec, L: 0
0811 18:57:12.631437 (+ 211us) write_op.cc:270] Start()
0811 18:57:12.631464 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622687469 usec, L: 0
0811 18:57:12.631613 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.631668 (+ 55us) write_op.cc:276] Timestamp: P: 1786474622688362 usec, L: 0
0811 18:57:12.631863 (+ 195us) write_op.cc:270] Start()
0811 18:57:12.631887 (+ 24us) write_op.cc:276] Timestamp: P: 1786474622692612 usec, L: 0
0811 18:57:12.632051 (+ 164us) write_op.cc:270] Start()
0811 18:57:12.632073 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622694298 usec, L: 0
0811 18:57:12.632228 (+ 155us) write_op.cc:270] Start()
0811 18:57:12.632255 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622695082 usec, L: 0
0811 18:57:12.632437 (+ 182us) write_op.cc:270] Start()
0811 18:57:12.632458 (+ 21us) write_op.cc:276] Timestamp: P: 1786474622695781 usec, L: 0
0811 18:57:12.632593 (+ 135us) write_op.cc:270] Start()
0811 18:57:12.632614 (+ 21us) write_op.cc:276] Timestamp: P: 1786474622748237 usec, L: 0
0811 18:57:12.632782 (+ 168us) write_op.cc:270] Start()
0811 18:57:12.632802 (+ 20us) write_op.cc:276] Timestamp: P: 1786474622786588 usec, L: 0
0811 18:57:12.632960 (+ 158us) write_op.cc:270] Start()
0811 18:57:12.632983 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622787350 usec, L: 0
0811 18:57:12.633132 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.633157 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622803836 usec, L: 0
0811 18:57:12.633306 (+ 149us) write_op.cc:270] Start()
0811 18:57:12.633328 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622804502 usec, L: 0
0811 18:57:12.633476 (+ 148us) write_op.cc:270] Start()
0811 18:57:12.633501 (+ 25us) write_op.cc:276] Timestamp: P: 1786474622805136 usec, L: 0
0811 18:57:12.633645 (+ 144us) write_op.cc:270] Start()
0811 18:57:12.633672 (+ 27us) write_op.cc:276] Timestamp: P: 1786474622823222 usec, L: 0
0811 18:57:12.633837 (+ 165us) write_op.cc:270] Start()
0811 18:57:12.633859 (+ 22us) write_op.cc:276] Timestamp: P: 1786474622823974 usec, L: 0
0811 18:57:12.634005 (+ 146us) write_op.cc:270] Start()
0811 18:57:12.634028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474622825294 usec, L: 0
0811 18:57:12.634173 (+ 145us) write_op.cc:270] Start()
0811 18:57:12.634193 (+ 20us) write_op.cc:276] Timestamp: P: 1786474622826372 usec, L: 0
0811 18:57:12.672180 (+ 37987us) write_op.cc:270] Start()
0811 18:57:12.672480 (+ 300us) write_op.cc:276] Timestamp: P: 1786474622827042 usec, L: 0
0811 18:57:12.679526 (+ 7046us) write_op.cc:270] Start()
0811 18:57:12.679784 (+ 258us) write_op.cc:276] Timestamp: P: 1786474622829576 usec, L: 0
0811 18:57:12.681637 (+ 1853us) write_op.cc:270] Start()
0811 18:57:12.682790 (+ 1153us) write_op.cc:276] Timestamp: P: 1786474622837820 usec, L: 0
0811 18:57:12.686993 (+ 4203us) write_op.cc:270] Start()
0811 18:57:12.687520 (+ 527us) write_op.cc:276] Timestamp: P: 1786474622838640 usec, L: 0
0811 18:57:12.740282 (+ 52762us) write_op.cc:270] Start()
0811 18:57:12.744388 (+ 4106us) write_op.cc:276] Timestamp: P: 1786474622871381 usec, L: 0
0811 18:57:12.761430 (+ 17042us) write_op.cc:270] Start()
0811 18:57:12.763737 (+ 2307us) write_op.cc:276] Timestamp: P: 1786474622883864 usec, L: 0
0811 18:57:12.818384 (+ 54647us) write_op.cc:270] Start()
0811 18:57:12.818556 (+ 172us) write_op.cc:276] Timestamp: P: 1786474622891993 usec, L: 0
0811 18:57:12.836150 (+ 17594us) write_op.cc:270] Start()
0811 18:57:12.836366 (+ 216us) write_op.cc:276] Timestamp: P: 1786474622911019 usec, L: 0
0811 18:57:12.839841 (+ 3475us) write_op.cc:270] Start()
0811 18:57:12.840824 (+ 983us) write_op.cc:276] Timestamp: P: 1786474622937463 usec, L: 0
0811 18:57:12.848971 (+ 8147us) write_op.cc:270] Start()
0811 18:57:12.852766 (+ 3795us) write_op.cc:276] Timestamp: P: 1786474622949955 usec, L: 0
0811 18:57:12.857047 (+ 4281us) write_op.cc:270] Start()
0811 18:57:12.857603 (+ 556us) write_op.cc:276] Timestamp: P: 1786474622976378 usec, L: 0
0811 18:57:12.877860 (+ 20257us) write_op.cc:270] Start()
0811 18:57:12.878306 (+ 446us) write_op.cc:276] Timestamp: P: 1786474622995113 usec, L: 0
0811 18:57:12.880231 (+ 1925us) write_op.cc:270] Start()
0811 18:57:12.880454 (+ 223us) write_op.cc:276] Timestamp: P: 1786474622998372 usec, L: 0
0811 18:57:12.882900 (+ 2446us) write_op.cc:270] Start()
0811 18:57:12.884419 (+ 1519us) write_op.cc:276] Timestamp: P: 1786474622999173 usec, L: 0
0811 18:57:12.886114 (+ 1695us) write_op.cc:270] Start()
0811 18:57:12.886351 (+ 237us) write_op.cc:276] Timestamp: P: 1786474623043663 usec, L: 0
0811 18:57:12.888347 (+ 1996us) write_op.cc:270] Start()
0811 18:57:12.888715 (+ 368us) write_op.cc:276] Timestamp: P: 1786474623044497 usec, L: 0
0811 18:57:12.889920 (+ 1205us) write_op.cc:270] Start()
0811 18:57:12.890121 (+ 201us) write_op.cc:276] Timestamp: P: 1786474623046123 usec, L: 0
0811 18:57:12.891415 (+ 1294us) write_op.cc:270] Start()
0811 18:57:12.891622 (+ 207us) write_op.cc:276] Timestamp: P: 1786474623048341 usec, L: 0
0811 18:57:12.893060 (+ 1438us) write_op.cc:270] Start()
0811 18:57:12.893090 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623055330 usec, L: 0
0811 18:57:12.899906 (+ 6816us) write_op.cc:270] Start()
0811 18:57:12.899956 (+ 50us) write_op.cc:276] Timestamp: P: 1786474623190642 usec, L: 0
0811 18:57:12.901612 (+ 1656us) write_op.cc:270] Start()
0811 18:57:12.901658 (+ 46us) write_op.cc:276] Timestamp: P: 1786474623235306 usec, L: 0
0811 18:57:12.909259 (+ 7601us) write_op.cc:270] Start()
0811 18:57:12.917736 (+ 8477us) write_op.cc:276] Timestamp: P: 1786474623256077 usec, L: 0
0811 18:57:12.937842 (+ 20106us) write_op.cc:270] Start()
0811 18:57:12.937900 (+ 58us) write_op.cc:276] Timestamp: P: 1786474623257645 usec, L: 0
0811 18:57:12.945400 (+ 7500us) write_op.cc:270] Start()
0811 18:57:12.945467 (+ 67us) write_op.cc:276] Timestamp: P: 1786474623259198 usec, L: 0
0811 18:57:12.952540 (+ 7073us) write_op.cc:270] Start()
0811 18:57:12.952623 (+ 83us) write_op.cc:276] Timestamp: P: 1786474623329019 usec, L: 0
0811 18:57:12.952943 (+ 320us) write_op.cc:270] Start()
0811 18:57:12.952972 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623331218 usec, L: 0
0811 18:57:12.953276 (+ 304us) write_op.cc:270] Start()
0811 18:57:12.953307 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623333719 usec, L: 0
0811 18:57:12.953524 (+ 217us) write_op.cc:270] Start()
0811 18:57:12.953543 (+ 19us) write_op.cc:276] Timestamp: P: 1786474623335405 usec, L: 0
0811 18:57:12.953679 (+ 136us) write_op.cc:270] Start()
0811 18:57:12.953706 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623336724 usec, L: 0
0811 18:57:12.957689 (+ 3983us) write_op.cc:270] Start()
0811 18:57:12.957745 (+ 56us) write_op.cc:276] Timestamp: P: 1786474623338424 usec, L: 0
0811 18:57:12.957987 (+ 242us) write_op.cc:270] Start()
0811 18:57:12.958017 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623343161 usec, L: 0
0811 18:57:12.958530 (+ 513us) write_op.cc:270] Start()
0811 18:57:12.958566 (+ 36us) write_op.cc:276] Timestamp: P: 1786474623347212 usec, L: 0
0811 18:57:12.958857 (+ 291us) write_op.cc:270] Start()
0811 18:57:12.958891 (+ 34us) write_op.cc:276] Timestamp: P: 1786474623349546 usec, L: 0
0811 18:57:12.959174 (+ 283us) write_op.cc:270] Start()
0811 18:57:12.959205 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623422449 usec, L: 0
0811 18:57:12.959522 (+ 317us) write_op.cc:270] Start()
0811 18:57:12.959554 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623430473 usec, L: 0
0811 18:57:12.960031 (+ 477us) write_op.cc:270] Start()
0811 18:57:12.960066 (+ 35us) write_op.cc:276] Timestamp: P: 1786474623432035 usec, L: 0
0811 18:57:12.960290 (+ 224us) write_op.cc:270] Start()
0811 18:57:12.960310 (+ 20us) write_op.cc:276] Timestamp: P: 1786474623435592 usec, L: 0
0811 18:57:12.995410 (+ 35100us) write_op.cc:270] Start()
0811 18:57:12.995610 (+ 200us) write_op.cc:276] Timestamp: P: 1786474623440107 usec, L: 0
0811 18:57:12.996055 (+ 445us) write_op.cc:270] Start()
0811 18:57:12.996107 (+ 52us) write_op.cc:276] Timestamp: P: 1786474623448895 usec, L: 0
0811 18:57:12.998732 (+ 2625us) write_op.cc:270] Start()
0811 18:57:12.998776 (+ 44us) write_op.cc:276] Timestamp: P: 1786474623487172 usec, L: 0
0811 18:57:12.999071 (+ 295us) write_op.cc:270] Start()
0811 18:57:12.999102 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623488870 usec, L: 0
0811 18:57:12.999275 (+ 173us) write_op.cc:270] Start()
0811 18:57:12.999302 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623497373 usec, L: 0
0811 18:57:12.999586 (+ 284us) write_op.cc:270] Start()
0811 18:57:12.999616 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623512725 usec, L: 0
0811 18:57:12.999849 (+ 233us) write_op.cc:270] Start()
0811 18:57:12.999879 (+ 30us) write_op.cc:276] Timestamp: P: 1786474623526487 usec, L: 0
0811 18:57:13.000132 (+ 253us) write_op.cc:270] Start()
0811 18:57:13.000158 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623537613 usec, L: 0
0811 18:57:13.000382 (+ 224us) write_op.cc:270] Start()
0811 18:57:13.000428 (+ 46us) write_op.cc:276] Timestamp: P: 1786474623538331 usec, L: 0
0811 18:57:13.000684 (+ 256us) write_op.cc:270] Start()
0811 18:57:13.000715 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623549647 usec, L: 0
0811 18:57:13.000937 (+ 222us) write_op.cc:270] Start()
0811 18:57:13.000969 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623550546 usec, L: 0
0811 18:57:13.001189 (+ 220us) write_op.cc:270] Start()
0811 18:57:13.001218 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623607852 usec, L: 0
0811 18:57:13.001436 (+ 218us) write_op.cc:270] Start()
0811 18:57:13.001464 (+ 28us) write_op.cc:276] Timestamp: P: 1786474623735005 usec, L: 0
0811 18:57:13.001759 (+ 295us) write_op.cc:270] Start()
0811 18:57:13.001792 (+ 33us) write_op.cc:276] Timestamp: P: 1786474623746020 usec, L: 0
0811 18:57:13.002057 (+ 265us) write_op.cc:270] Start()
0811 18:57:13.002088 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623746835 usec, L: 0
0811 18:57:13.002311 (+ 223us) write_op.cc:270] Start()
0811 18:57:13.002368 (+ 57us) write_op.cc:276] Timestamp: P: 1786474623759665 usec, L: 0
0811 18:57:13.002621 (+ 253us) write_op.cc:270] Start()
0811 18:57:13.002711 (+ 90us) write_op.cc:276] Timestamp: P: 1786474623762433 usec, L: 0
0811 18:57:13.002901 (+ 190us) write_op.cc:270] Start()
0811 18:57:13.002930 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623770621 usec, L: 0
0811 18:57:13.003262 (+ 332us) write_op.cc:270] Start()
0811 18:57:13.003296 (+ 34us) write_op.cc:276] Timestamp: P: 1786474623773272 usec, L: 0
0811 18:57:13.003562 (+ 266us) write_op.cc:270] Start()
0811 18:57:13.003593 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623778446 usec, L: 0
0811 18:57:13.003774 (+ 181us) write_op.cc:270] Start()
0811 18:57:13.003800 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623786685 usec, L: 0
0811 18:57:13.004087 (+ 287us) write_op.cc:270] Start()
0811 18:57:13.004118 (+ 31us) write_op.cc:276] Timestamp: P: 1786474623955702 usec, L: 0
0811 18:57:13.004437 (+ 319us) write_op.cc:270] Start()
0811 18:57:13.004476 (+ 39us) write_op.cc:276] Timestamp: P: 1786474623956496 usec, L: 0
0811 18:57:13.004695 (+ 219us) write_op.cc:270] Start()
0811 18:57:13.004724 (+ 29us) write_op.cc:276] Timestamp: P: 1786474623960711 usec, L: 0
0811 18:57:13.004908 (+ 184us) write_op.cc:270] Start()
0811 18:57:13.004934 (+ 26us) write_op.cc:276] Timestamp: P: 1786474623975063 usec, L: 0
0811 18:57:13.005085 (+ 151us) write_op.cc:270] Start()
0811 18:57:13.005112 (+ 27us) write_op.cc:276] Timestamp: P: 1786474623989664 usec, L: 0
0811 18:57:13.022384 (+ 17272us) write_op.cc:270] Start()
0811 18:57:13.022436 (+ 52us) write_op.cc:276] Timestamp: P: 1786474623992708 usec, L: 0
0811 18:57:13.022684 (+ 248us) write_op.cc:270] Start()
0811 18:57:13.022709 (+ 25us) write_op.cc:276] Timestamp: P: 1786474623994235 usec, L: 0
0811 18:57:13.023014 (+ 305us) write_op.cc:270] Start()
0811 18:57:13.023046 (+ 32us) write_op.cc:276] Timestamp: P: 1786474623997481 usec, L: 0
0811 18:57:13.023189 (+ 143us) write_op.cc:270] Start()
0811 18:57:13.023218 (+ 29us) write_op.cc:276] Timestamp: P: 1786474624059689 usec, L: 0
0811 18:57:13.023402 (+ 184us) write_op.cc:270] Start()
0811 18:57:13.023429 (+ 27us) write_op.cc:276] Timestamp: P: 1786474624243324 usec, L: 0
0811 18:57:13.023581 (+ 152us) write_op.cc:270] Start()
0811 18:57:13.023606 (+ 25us) write_op.cc:276] Timestamp: P: 1786474624256820 usec, L: 0
0811 18:57:13.023895 (+ 289us) write_op.cc:270] Start()
0811 18:57:13.023926 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624352114 usec, L: 0
0811 18:57:13.052012 (+ 28086us) write_op.cc:270] Start()
0811 18:57:13.052075 (+ 63us) write_op.cc:276] Timestamp: P: 1786474624402904 usec, L: 0
0811 18:57:13.052346 (+ 271us) write_op.cc:270] Start()
0811 18:57:13.052371 (+ 25us) write_op.cc:276] Timestamp: P: 1786474624474253 usec, L: 0
0811 18:57:13.052641 (+ 270us) write_op.cc:270] Start()
0811 18:57:13.052678 (+ 37us) write_op.cc:276] Timestamp: P: 1786474624498885 usec, L: 0
0811 18:57:13.052975 (+ 297us) write_op.cc:270] Start()
0811 18:57:13.053089 (+ 114us) write_op.cc:276] Timestamp: P: 1786474624499707 usec, L: 0
0811 18:57:13.053505 (+ 416us) write_op.cc:270] Start()
0811 18:57:13.053540 (+ 35us) write_op.cc:276] Timestamp: P: 1786474624500506 usec, L: 0
0811 18:57:13.053827 (+ 287us) write_op.cc:270] Start()
0811 18:57:13.053858 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624501269 usec, L: 0
0811 18:57:13.054062 (+ 204us) write_op.cc:270] Start()
0811 18:57:13.054095 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624539494 usec, L: 0
0811 18:57:13.054384 (+ 289us) write_op.cc:270] Start()
0811 18:57:13.054482 (+ 98us) write_op.cc:276] Timestamp: P: 1786474624540391 usec, L: 0
0811 18:57:13.054860 (+ 378us) write_op.cc:270] Start()
0811 18:57:13.054893 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624541239 usec, L: 0
0811 18:57:13.055116 (+ 223us) write_op.cc:270] Start()
0811 18:57:13.055149 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624600335 usec, L: 0
0811 18:57:13.055395 (+ 246us) write_op.cc:270] Start()
0811 18:57:13.055427 (+ 32us) write_op.cc:276] Timestamp: P: 1786474624630105 usec, L: 0
0811 18:57:13.055609 (+ 182us) write_op.cc:270] Start()
0811 18:57:13.055638 (+ 29us) write_op.cc:276] Timestamp: P: 1786474624657814 usec, L: 0
0811 18:57:13.055873 (+ 235us) write_op.cc:270] Start()
0811 18:57:13.055904 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624675500 usec, L: 0
0811 18:57:13.056170 (+ 266us) write_op.cc:270] Start()
0811 18:57:13.056208 (+ 38us) write_op.cc:276] Timestamp: P: 1786474624676763 usec, L: 0
0811 18:57:13.056416 (+ 208us) write_op.cc:270] Start()
0811 18:57:13.056447 (+ 31us) write_op.cc:276] Timestamp: P: 1786474624677587 usec, L: 0
0811 18:57:13.056800 (+ 353us) write_op.cc:270] Start()
0811 18:57:13.056837 (+ 37us) write_op.cc:276] Timestamp: P: 1786474624966055 usec, L: 0
0811 18:57:13.057139 (+ 302us) write_op.cc:270] Start()
0811 18:57:13.057172 (+ 33us) write_op.cc:276] Timestamp: P: 1786474624967959 usec, L: 0
0811 18:57:13.057407 (+ 235us) write_op.cc:270] Start()
0811 18:57:13.057441 (+ 34us) write_op.cc:276] Timestamp: P: 1786474624969041 usec, L: 0
0811 18:57:13.057680 (+ 239us) write_op.cc:270] Start()
0811 18:57:13.057711 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625027920 usec, L: 0
0811 18:57:13.058194 (+ 483us) write_op.cc:270] Start()
0811 18:57:13.058233 (+ 39us) write_op.cc:276] Timestamp: P: 1786474625046678 usec, L: 0
0811 18:57:13.058503 (+ 270us) write_op.cc:270] Start()
0811 18:57:13.058538 (+ 35us) write_op.cc:276] Timestamp: P: 1786474625049907 usec, L: 0
0811 18:57:13.058735 (+ 197us) write_op.cc:270] Start()
0811 18:57:13.058878 (+ 143us) write_op.cc:276] Timestamp: P: 1786474625051415 usec, L: 0
0811 18:57:13.059078 (+ 200us) write_op.cc:270] Start()
0811 18:57:13.059129 (+ 51us) write_op.cc:276] Timestamp: P: 1786474625054870 usec, L: 0
0811 18:57:13.059318 (+ 189us) write_op.cc:270] Start()
0811 18:57:13.059350 (+ 32us) write_op.cc:276] Timestamp: P: 1786474625083925 usec, L: 0
0811 18:57:13.059547 (+ 197us) write_op.cc:270] Start()
0811 18:57:13.059576 (+ 29us) write_op.cc:276] Timestamp: P: 1786474625108071 usec, L: 0
0811 18:57:13.059774 (+ 198us) write_op.cc:270] Start()
0811 18:57:13.059805 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625125339 usec, L: 0
0811 18:57:13.060061 (+ 256us) write_op.cc:270] Start()
0811 18:57:13.060092 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625139115 usec, L: 0
0811 18:57:13.060399 (+ 307us) write_op.cc:270] Start()
0811 18:57:13.060434 (+ 35us) write_op.cc:276] Timestamp: P: 1786474625142066 usec, L: 0
0811 18:57:13.060628 (+ 194us) write_op.cc:270] Start()
0811 18:57:13.060658 (+ 30us) write_op.cc:276] Timestamp: P: 1786474625144990 usec, L: 0
0811 18:57:13.060933 (+ 275us) write_op.cc:270] Start()
0811 18:57:13.060966 (+ 33us) write_op.cc:276] Timestamp: P: 1786474625149520 usec, L: 0
0811 18:57:13.061207 (+ 241us) write_op.cc:270] Start()
0811 18:57:13.061238 (+ 31us) write_op.cc:276] Timestamp: P: 1786474625191592 usec, L: 0
0811 18:57:13.061496 (+ 258us) write_op.cc:270] Start()
0811 18:57:13.061522 (+ 26us) write_op.cc:276] Timestamp: P: 1786474625268767 usec, L: 0
0811 18:57:13.061818 (+ 296us) write_op.cc:270] Start()
0811 18:57:13.061855 (+ 37us) write_op.cc:276] Timestamp: P: 1786474625491202 usec, L: 0
0811 18:57:13.062153 (+ 298us) write_op.cc:270] Start()
0811 18:57:13.062186 (+ 33us) write_op.cc:276] Timestamp: P: 178647
I20260811 18:57:14.002063 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 190) took 6749 ms. Trace:
I20260811 18:57:14.027637 4162 raft_consensus.cc:348] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.045637 4169 rpcz_store.cc:276] 0811 18:57:07.250231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:07.250307 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:13.689743 (+6439436us) ts_tablet_manager.cc:1145] Deleting tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
0811 18:57:13.689757 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:13.797657 (+107900us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.906071 (+108414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:13.999806 (+ 93735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5380109}
I20260811 18:57:14.046185 4162 raft_consensus.cc:374] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.046880 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 207) took 2675 ms. Trace:
I20260811 18:57:14.046888 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (table default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]) successfully deleted
I20260811 18:57:14.047044 4169 rpcz_store.cc:276] 0811 18:57:11.371448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375371 (+ 3923us) service_pool.cc:227] Handling call
0811 18:57:14.046501 (+2671130us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:14.046517 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.046691 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2386499}
I20260811 18:57:14.046180 4163 raft_consensus.cc:374] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:14.047775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0190cbd859274fcfa09c6fab899c4843 already in progress: creating tablet
I20260811 18:57:14.048267 4162 raft_consensus.cc:729] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.049067 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 208) took 2675 ms. Trace:
I20260811 18:57:14.055322 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet c08436f33db043b1bc6395cd92102e59 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.055609 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:14.056332 3549 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 309) took 2166 ms. Trace:
I20260811 18:57:14.056463 3549 rpcz_store.cc:276] 0811 18:57:11.889901 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.472638 (+1582737us) service_pool.cc:227] Handling call
0811 18:57:13.540976 (+ 68338us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f412e6 00007f61ef234126 00007f61ef233e55 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:13.541815 (+ 839us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:13.541974 (+ 159us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:13.542045 (+ 71us) catalog_manager.cc:2761] Locking tablets
0811 18:57:13.843289 (+301244us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:14.055320 (+212031us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:14.055418 (+ 98us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:14.055458 (+ 40us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:14.056209 (+ 751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:13.980964 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:13.981078 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:13.981085 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:13.981087 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:13.981221 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:13.981231 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:13.981236 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:13.981357 (+ 121us) tablet.cc:699] Row locks acquired
0811 18:57:13.981360 (+ 3us) write_op.cc:260] PREPARE: finished
0811 18:57:13.981404 (+ 44us) write_op.cc:270] Start()
0811 18:57:13.981432 (+ 28us) write_op.cc:276] Timestamp: P: 1786474633981368 usec, L: 0
0811 18:57:13.981435 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:13.981612 (+ 177us) log.cc:842] Serialized 725 byte log entry
0811 18:57:14.053225 (+ 71613us) op_driver.cc:444] REPLICATION: finished
0811 18:57:14.054581 (+ 1356us) write_op.cc:301] APPLY: starting
0811 18:57:14.054601 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:14.054670 (+ 69us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:14.054746 (+ 76us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:14.054918 (+ 172us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:14.054968 (+ 50us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:14.054980 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:14.055077 (+ 97us) log.cc:842] Serialized 29 byte log entry
0811 18:57:14.055148 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:14.055171 (+ 23us) write_op.cc:454] Released schema lock
0811 18:57:14.055181 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:14.055277 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:14.055284 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":156432768,"child_traces":[["op",{"apply.queue_time_us":1301,"apply.run_cpu_time_us":781,"apply.run_wall_time_us":780,"num_ops":2,"prepare.queue_time_us":136521,"prepare.run_cpu_time_us":779,"prepare.run_wall_time_us":11884,"replication_time_us":71765}]]}
I20260811 18:57:14.057389 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 405773849fb94800be61ad8a7a6055a4
I20260811 18:57:14.057691 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c87170e0a947438da8d0091a78e401c4" } new_table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" modify_external_catalogs: true
I20260811 18:57:14.058051 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 405773849fb94800be61ad8a7a6055a4 deleted and replaced by tablet 0aee6beb2e70421083841520733087f8
I20260811 18:57:14.058564 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cb87feb6db445d792ba8021cbe75510" } new_table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" modify_external_catalogs: true
I20260811 18:57:14.054224 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_021124838f7841c49f01816fb948d73c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.061638 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_021124838f7841c49f01816fb948d73c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.064527 4638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:14.068084 4638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:14.078564 4163 raft_consensus.cc:729] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.079010 4163 consensus_queue.cc:260] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.083388 4163 ts_tablet_manager.cc:1428] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.103s user 0.002s sys 0.002s
I20260811 18:57:14.083566 4169 rpcz_store.cc:276] 0811 18:57:11.371593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375616 (+ 4023us) service_pool.cc:227] Handling call
0811 18:57:14.047331 (+2671715us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:14.047343 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.047446 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2387237}
I20260811 18:57:14.086212 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d" } modify_external_catalogs: true
I20260811 18:57:14.086710 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_511d4c7dbd2b41ae95fc5f910147870d" } modify_external_catalogs: true
I20260811 18:57:14.089480 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 6ce8db230c7e41b6a4792273a0f07e6e from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:14.090432 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6ce8db230c7e41b6a4792273a0f07e6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ce8db230c7e41b6a4792273a0f07e6e already in progress: creating tablet
I20260811 18:57:14.060243 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.069248 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.081662 4556 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.095513 4556 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.095630 4162 consensus_queue.cc:260] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.096680 4162 ts_tablet_manager.cc:1428] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.128s user 0.000s sys 0.004s
I20260811 18:57:14.096927 4556 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.101701 4162 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.103099 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:14.104030 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 9bc5448a555944dfba8ccae09e25cbe9 deleted and replaced by tablet d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:14.104806 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f3fb616fae4249b4a043ddb879c7db57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:14.105059 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:14.105944 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3fb616fae4249b4a043ddb879c7db57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.108853 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_98051d104e9e4d9b976577088d6126a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.109674 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98051d104e9e4d9b976577088d6126a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.113193 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.113700 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 60274115fe164d85badcf2e72d640f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ad616256b9949ceaf8b300f660d00c5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.114888 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c08436f33db043b1bc6395cd92102e59
I20260811 18:57:14.115520 3547 catalog_manager.cc:6565] ReplaceTablet: tablet c08436f33db043b1bc6395cd92102e59 deleted and replaced by tablet 059b325df1db4fde9d4b26b559121316
I20260811 18:57:14.113711 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.091609 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:14.117043 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.119222 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d16e5449864d18a0a01e39d25ad8dd" } new_table_name: "default.Table_88ad2bea931c4ea0843ace52fcb146b3" modify_external_catalogs: true
I20260811 18:57:14.120790 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f640c60e81b4a369458b1497a269c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.121217 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.090320 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1a8b47adc7454c64a44013e919b3a485" } modify_external_catalogs: true
I20260811 18:57:14.121871 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1a8b47adc7454c64a44013e919b3a485" } modify_external_catalogs: true
I20260811 18:57:14.125041 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a5043728e7c842ea851193559d898d80" } new_table_name: "default.Table_06e3c71319514b82b5e73f430a685e0f" modify_external_catalogs: true
I20260811 18:57:14.127823 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.128259 3891 raft_consensus.cc:2454] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.129204 3676 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.130024 4576 raft_consensus.cc:2781] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.130182 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f640c60e81b4a369458b1497a269c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.130244 4576 raft_consensus.cc:482] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.130582 4576 raft_consensus.cc:3037] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.131678 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.132234 4163 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.135795 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.137131 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 209) took 2758 ms. Trace:
I20260811 18:57:14.137382 4169 rpcz_store.cc:276] 0811 18:57:11.371709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375789 (+ 4080us) service_pool.cc:227] Handling call
0811 18:57:14.083974 (+2708185us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:14.118444 (+ 34470us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.130507 (+ 12063us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2423826}
I20260811 18:57:14.138285 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 210) took 2766 ms. Trace:
I20260811 18:57:14.138458 4169 rpcz_store.cc:276] 0811 18:57:11.371814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.375940 (+ 4126us) service_pool.cc:227] Handling call
0811 18:57:14.137817 (+2761877us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:14.137832 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.138143 (+ 311us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2477617}
I20260811 18:57:14.139111 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 220) took 1357 ms. Trace:
I20260811 18:57:14.140012 4169 rpcz_store.cc:276] 0811 18:57:12.781970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.785648 (+ 3678us) service_pool.cc:227] Handling call
0811 18:57:14.138781 (+1353133us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:14.138797 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.139021 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1351340}
I20260811 18:57:14.140717 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 218) took 1359 ms. Trace:
I20260811 18:57:14.140918 4169 rpcz_store.cc:276] 0811 18:57:12.781641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.786564 (+ 4923us) service_pool.cc:227] Handling call
0811 18:57:14.140300 (+1353736us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:14.140310 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.140505 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35,"tablet-delete.queue_time_us":1352864}
I20260811 18:57:14.141752 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 219) took 1359 ms. Trace:
W20260811 18:57:14.144634 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 460e03cc14444c83b6c08ca194ca5a8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 460e03cc14444c83b6c08ca194ca5a8d already in progress: creating tablet
W20260811 18:57:14.145123 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dde27d8cc3eb4d09879e717fc90305fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dde27d8cc3eb4d09879e717fc90305fd already in progress: creating tablet
W20260811 18:57:14.145437 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1a55a42d1e64f04ad7f433614a3a1aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1a55a42d1e64f04ad7f433614a3a1aa already in progress: creating tablet
W20260811 18:57:14.145774 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ed3346bd12d04c45a511308c427856e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed3346bd12d04c45a511308c427856e5 already in progress: creating tablet
I20260811 18:57:14.148324 4163 tablet_bootstrap.cc:654] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.142179 4169 rpcz_store.cc:276] 0811 18:57:12.781813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.785678 (+ 3865us) service_pool.cc:227] Handling call
0811 18:57:14.141342 (+1355664us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:14.141354 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.141490 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1351677}
I20260811 18:57:14.151463 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 226) took 1267 ms. Trace:
I20260811 18:57:14.151716 4169 rpcz_store.cc:276] 0811 18:57:12.883525 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.883592 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:14.150846 (+1267254us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:14.150862 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.151300 (+ 438us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1248187}
W20260811 18:57:14.153930 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 94943cfbe41a47b3a36c81957a3adaeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94943cfbe41a47b3a36c81957a3adaeb already in progress: creating tablet
W20260811 18:57:14.154470 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f61d3822fc2d4b569f223238a6464e0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f61d3822fc2d4b569f223238a6464e0a already in progress: creating tablet
W20260811 18:57:14.154924 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f335e50ae57547ed80b144129faaffbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f335e50ae57547ed80b144129faaffbb already in progress: creating tablet
W20260811 18:57:14.156575 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7251204325dd424192a6f58e18e71c34 has no consensus state
I20260811 18:57:14.157183 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44e9cbdec47545899467895fd869b389" } modify_external_catalogs: true
I20260811 18:57:14.157533 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_44e9cbdec47545899467895fd869b389" } modify_external_catalogs: true
W20260811 18:57:14.159238 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbba1923ce0049448fe4e28f2dd258dc consensus state has no leader
I20260811 18:57:14.159881 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5345e1fe57284913bcf778715a827515" } new_table_name: "default.Table_2e738d6898cb440ca15ea7cca8f58b15" modify_external_catalogs: true
I20260811 18:57:14.161495 3550 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd] alter complete (version 1)
I20260811 18:57:14.163079 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1b41e9593787430fb7137bed80d6f078"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.164193 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b41e9593787430fb7137bed80d6f078 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.170094 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ef393c341bb54baf87176db5b5254ec7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef393c341bb54baf87176db5b5254ec7 already in progress: creating tablet
W20260811 18:57:14.170740 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d106e0e1f4b845429b10f4e100f69a8b because the tablet was not found. No further retry: Not found: Tablet not found: d106e0e1f4b845429b10f4e100f69a8b
W20260811 18:57:14.171175 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c already in progress: creating tablet
I20260811 18:57:14.173969 4729 consensus_queue.cc:1035] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:14.174511 4169 tablet_replica.cc:330] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.175050 4169 raft_consensus.cc:2229] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.175987 3891 raft_consensus.cc:1261] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260811 18:57:14.177224 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 490f34d7a403416fbab64e46973746de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 490f34d7a403416fbab64e46973746de already in progress: creating tablet
W20260811 18:57:14.178653 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 125757b8c1a6448b841c63339f392339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125757b8c1a6448b841c63339f392339 already in progress: creating tablet
I20260811 18:57:14.179186 4729 consensus_queue.cc:1035] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260811 18:57:14.173527 4162 tablet_bootstrap.cc:654] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.190479 4163 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.190907 4163 ts_tablet_manager.cc:1397] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.004s
I20260811 18:57:14.191689 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:14.192538 4163 raft_consensus.cc:348] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.193017 4163 raft_consensus.cc:374] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.193179 4163 raft_consensus.cc:729] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.196250 4163 consensus_queue.cc:260] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.198835 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.198469 4163 ts_tablet_manager.cc:1428] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:14.202958 4163 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.207110 4163 tablet_bootstrap.cc:654] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.211108 4576 raft_consensus.cc:504] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.213176 4576 leader_election.cc:288] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.216956 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.238520 4169 raft_consensus.cc:2258] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.239652 4169 ts_tablet_manager.cc:1905] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.241153 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ace00158b744bd8a97dc1c47fc6049c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.241576 3888 raft_consensus.cc:3037] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.246127 4162 tablet_bootstrap.cc:492] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.246594 4162 ts_tablet_manager.cc:1397] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.145s user 0.009s sys 0.000s
I20260811 18:57:14.248323 4162 raft_consensus.cc:348] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.248987 4162 raft_consensus.cc:374] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.250488 4162 raft_consensus.cc:729] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.266387 4658 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.266702 4658 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.268030 4658 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:14.264408 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 43526.5us
I20260811 18:57:14.269130 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.275563 3744 raft_consensus.cc:2454] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.276484 3824 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.276793 4163 tablet_bootstrap.cc:492] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.277025 4658 raft_consensus.cc:2781] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.277364 4163 ts_tablet_manager.cc:1397] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.010s
I20260811 18:57:14.277794 4658 raft_consensus.cc:482] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.278021 4658 raft_consensus.cc:3037] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:14.278913 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23f41a2638e74ba19c1c527b9fc796df consensus state has no leader
I20260811 18:57:14.280632 4739 raft_consensus.cc:482] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.281013 4739 raft_consensus.cc:504] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.283249 4739 leader_election.cc:288] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.283289 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.284137 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdc11a6352eb4aac9ff657cf245aceee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.285270 4162 consensus_queue.cc:260] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.286157 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c70d9db064e45af91fd9901d85e211f consensus state has no leader
I20260811 18:57:14.286360 4162 ts_tablet_manager.cc:1428] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.039s user 0.001s sys 0.002s
I20260811 18:57:14.286867 4162 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.287636 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6d87a15f366a4e118c84e294c3b9926e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.288818 3676 leader_election.cc:341] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 2852f9e5f67c4025836018af553388a5
I20260811 18:57:14.288992 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd862d7d75f24a05aa54d761e710cd50" } modify_external_catalogs: true
I20260811 18:57:14.289295 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd862d7d75f24a05aa54d761e710cd50" } modify_external_catalogs: true
I20260811 18:57:14.290139 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_858da7e74d4141a19ced6fdc29c33d89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.291023 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_858da7e74d4141a19ced6fdc29c33d89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.294792 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d87a15f366a4e118c84e294c3b9926e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:14.298964 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6f36ab6bb774257b5c2692c64eff63d consensus state has no leader
I20260811 18:57:14.299612 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e746326e8bff404aa5b3bc1fac099489"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.300535 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e746326e8bff404aa5b3bc1fac099489 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.306668 4162 tablet_bootstrap.cc:654] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.324653 4169 ts_tablet_manager.cc:1918] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.327580 4167 ts_tablet_manager.cc:1918] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.332760 4167 log.cc:1196] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
I20260811 18:57:14.333642 4167 ts_tablet_manager.cc:1939] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:14.328423 4658 raft_consensus.cc:504] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.336668 4658 leader_election.cc:288] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.337692 4169 log.cc:1196] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:14.338109 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60274115fe164d85badcf2e72d640f54" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:14.338562 3744 raft_consensus.cc:3037] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.339172 4169 ts_tablet_manager.cc:1939] T a0ba169c32ef48b984e649b211ee2af1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.342064 4163 raft_consensus.cc:348] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.342542 4163 raft_consensus.cc:374] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.342720 4163 raft_consensus.cc:729] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.343040 4163 consensus_queue.cc:260] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.344162 4163 ts_tablet_manager.cc:1428] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.066s user 0.003s sys 0.000s
I20260811 18:57:14.344658 4163 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.348503 4163 tablet_bootstrap.cc:654] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.358245 4162 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.358500 4162 ts_tablet_manager.cc:1397] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.002s
I20260811 18:57:14.359221 4739 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.359579 4739 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.360391 4739 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.361632 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.362684 3891 raft_consensus.cc:2454] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.360550 4162 raft_consensus.cc:348] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.363386 3676 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.363440 4162 raft_consensus.cc:374] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.363688 4162 raft_consensus.cc:729] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.363910 4739 raft_consensus.cc:2781] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.364087 4739 raft_consensus.cc:482] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.364061 4162 consensus_queue.cc:260] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.364507 4739 raft_consensus.cc:3037] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.366189 4162 ts_tablet_manager.cc:1428] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:14.366746 4162 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.368081 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:14.369165 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_994d2d4469b54597a9aa33c59dca9d84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.369988 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_994d2d4469b54597a9aa33c59dca9d84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.373051 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.375102 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b07958bac6344da1bbf3d7d35f1ac551"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.376061 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b07958bac6344da1bbf3d7d35f1ac551 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.379679 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet e7b3a14979e84b1e8ec6ed3f49b93b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.387741 4162 tablet_bootstrap.cc:654] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.391525 4658 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.391824 4658 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.392771 4658 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.394115 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:14.395164 3824 leader_election.cc:341] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:14.400990 4163 tablet_bootstrap.cc:492] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.401284 4163 ts_tablet_manager.cc:1397] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.057s user 0.015s sys 0.001s
I20260811 18:57:14.402413 4163 raft_consensus.cc:348] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.403062 4163 raft_consensus.cc:374] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.403291 4163 raft_consensus.cc:729] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.403771 4163 consensus_queue.cc:260] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.408244 4163 ts_tablet_manager.cc:1428] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:14.410610 4163 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.414026 4163 tablet_bootstrap.cc:654] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.418205 3888 raft_consensus.cc:2454] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.419806 4169 tablet_replica.cc:330] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.420199 4169 raft_consensus.cc:2229] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.420634 4169 raft_consensus.cc:2258] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.421659 4169 ts_tablet_manager.cc:1905] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.426789 3744 raft_consensus.cc:2454] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:14.427402 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a0ba169c32ef48b984e649b211ee2af1 (table default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:14.427543 3824 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:14.427793 3824 raft_consensus.cc:2670] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:14.427989 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a902ad70a25546be926ff8f7bf2ed64f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a902ad70a25546be926ff8f7bf2ed64f already in progress: creating tablet
W20260811 18:57:14.428294 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 867259e37fe44c33a35823ec2d896754 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 867259e37fe44c33a35823ec2d896754 already in progress: creating tablet
W20260811 18:57:14.428843 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 405773849fb94800be61ad8a7a6055a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 405773849fb94800be61ad8a7a6055a4 already in progress: creating tablet
I20260811 18:57:14.433521 3676 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.433717 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 176) took 7466 ms. Trace:
I20260811 18:57:14.434047 4576 raft_consensus.cc:2781] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.435012 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:14.435946 4576 raft_consensus.cc:686] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.436350 4576 consensus_queue.cc:237] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.434077 4167 rpcz_store.cc:276] 0811 18:57:06.966943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:06.967502 (+ 559us) service_pool.cc:227] Handling call
0811 18:57:13.996346 (+7028844us) ts_tablet_manager.cc:1145] Deleting tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97
0811 18:57:13.996369 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.207991 (+211622us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:14.327310 (+119319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:14.433566 (+106256us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7028515}
I20260811 18:57:14.437835 3548 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:14.438325 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 198) took 6039 ms. Trace:
I20260811 18:57:14.438681 4167 rpcz_store.cc:276] 0811 18:57:08.399130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.399224 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:14.437961 (+6038737us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:14.437975 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.438193 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6020530}
I20260811 18:57:14.439425 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 199) took 6024 ms. Trace:
I20260811 18:57:14.439726 4167 rpcz_store.cc:276] 0811 18:57:08.414579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.414663 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:14.439074 (+6024411us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:14.439085 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.439259 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5991913}
I20260811 18:57:14.440969 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 (table default.Table_68b0781f65bb450a9b74324ca7d6d5ed [id=a2cbbc78d7284cb4be302f06b79a329b]) successfully deleted
W20260811 18:57:14.441370 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0190cbd859274fcfa09c6fab899c4843 already in progress: creating tablet
I20260811 18:57:14.441895 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 200) took 5995 ms. Trace:
W20260811 18:57:14.441895 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6ce8db230c7e41b6a4792273a0f07e6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ce8db230c7e41b6a4792273a0f07e6e already in progress: creating tablet
I20260811 18:57:14.442255 4167 rpcz_store.cc:276] 0811 18:57:08.446364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.446549 (+ 185us) service_pool.cc:227] Handling call
0811 18:57:14.440011 (+5993462us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:14.440021 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.441710 (+ 1689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5963396}
I20260811 18:57:14.443230 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.443403 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 201) took 5992 ms. Trace:
I20260811 18:57:14.443591 4167 rpcz_store.cc:276] 0811 18:57:08.451041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:08.451112 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:14.443083 (+5991971us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:14.443097 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.443310 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":553,"tablet-delete.queue_time_us":5966469}
W20260811 18:57:14.443910 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 460e03cc14444c83b6c08ca194ca5a8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 460e03cc14444c83b6c08ca194ca5a8d already in progress: creating tablet
W20260811 18:57:14.445024 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dde27d8cc3eb4d09879e717fc90305fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dde27d8cc3eb4d09879e717fc90305fd already in progress: creating tablet
I20260811 18:57:14.445751 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 211) took 2762 ms. Trace:
I20260811 18:57:14.446125 4167 rpcz_store.cc:276] 0811 18:57:11.682595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682898 (+ 303us) service_pool.cc:227] Handling call
0811 18:57:14.445021 (+2762123us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:14.445034 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.445153 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761830}
I20260811 18:57:14.446455 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.447355 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 210) took 2764 ms. Trace:
I20260811 18:57:14.447983 4167 rpcz_store.cc:276] 0811 18:57:11.682450 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682805 (+ 355us) service_pool.cc:227] Handling call
0811 18:57:14.447168 (+2764363us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:14.447178 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.447293 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1656622}
W20260811 18:57:14.448906 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1a55a42d1e64f04ad7f433614a3a1aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1a55a42d1e64f04ad7f433614a3a1aa already in progress: creating tablet
I20260811 18:57:14.449204 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 209) took 2766 ms. Trace:
I20260811 18:57:14.449536 4167 rpcz_store.cc:276] 0811 18:57:11.682208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:11.682674 (+ 466us) service_pool.cc:227] Handling call
0811 18:57:14.448937 (+2766263us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:14.448949 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.449089 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1658311}
W20260811 18:57:14.449767 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 94943cfbe41a47b3a36c81957a3adaeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94943cfbe41a47b3a36c81957a3adaeb already in progress: creating tablet
W20260811 18:57:14.451191 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ed3346bd12d04c45a511308c427856e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed3346bd12d04c45a511308c427856e5 already in progress: creating tablet
I20260811 18:57:14.455888 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 58c515c153df4317b3fb427983ee15e0 (DEFAULT_TABLE table=default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456067 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1374a80d15444cb0faff339275aa8d (DEFAULT_TABLE table=default.Table_9292c8eb06fb4a0d8e33b6796398b740 [id=e0ef202b868f47269e4f2a63cb5a390a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456451 3861 tablet_service.cc:1467] Processing CreateTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 (DEFAULT_TABLE table=default.Table_cec1a44bf54f48019648c9338236fcc0 [id=5600dcaaf83c4f978c3d47f79e01568c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.456986 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 40311cd1f327461e94eb443d83a16480 (DEFAULT_TABLE table=default.Table_4dfa50c8631f4b70b6b743837d5a9032 [id=01ec228d80ec40c8a1778d09edcd61b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.461556 3710 tablet_service.cc:1467] Processing CreateTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 (DEFAULT_TABLE table=default.Table_cec1a44bf54f48019648c9338236fcc0 [id=5600dcaaf83c4f978c3d47f79e01568c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.462311 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d4c26da86b49b8a37f2289f4ef6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.468864 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 217) took 1665 ms. Trace:
I20260811 18:57:14.471963 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 58c515c153df4317b3fb427983ee15e0 (DEFAULT_TABLE table=default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.472681 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c515c153df4317b3fb427983ee15e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.477416 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1374a80d15444cb0faff339275aa8d (DEFAULT_TABLE table=default.Table_9292c8eb06fb4a0d8e33b6796398b740 [id=e0ef202b868f47269e4f2a63cb5a390a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.478400 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1374a80d15444cb0faff339275aa8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.479609 4163 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.480031 4163 ts_tablet_manager.cc:1397] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.004s
I20260811 18:57:14.469154 4167 rpcz_store.cc:276] 0811 18:57:12.803641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.803976 (+ 335us) service_pool.cc:227] Handling call
0811 18:57:14.468452 (+1664476us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:14.468468 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.468755 (+ 287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1664165}
I20260811 18:57:14.481886 4163 raft_consensus.cc:348] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.482456 4163 raft_consensus.cc:374] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.482745 4163 raft_consensus.cc:729] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.483498 4163 consensus_queue.cc:260] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.484460 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (DEFAULT_TABLE table=default.Table_77c2e3d67a494888be8b3405d4a06cf2 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.485064 4163 ts_tablet_manager.cc:1428] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260811 18:57:14.486851 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (DEFAULT_TABLE table=default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.486852 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 45fde25bcb40493aafb083e161aa6932 (DEFAULT_TABLE table=default.Table_575c5aca059048d68252cca6583a1f67 [id=4bc5d2d6894e40ebab164e5ffd6ac600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.486907 4163 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.490526 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3441d374bdb04681b2bd464f6ed910e2" } modify_external_catalogs: true
I20260811 18:57:14.491117 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3441d374bdb04681b2bd464f6ed910e2" } modify_external_catalogs: true
I20260811 18:57:14.494781 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6317dbacc890428caf5adac65ad7895d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:14.496156 4163 tablet_bootstrap.cc:654] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:14.496451 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6317dbacc890428caf5adac65ad7895d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.509548 4740 raft_consensus.cc:482] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.509869 4740 raft_consensus.cc:504] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.511914 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807cee88c18340dbb23c51cad54ab56d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.512475 3888 raft_consensus.cc:2454] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.513873 3676 leader_election.cc:302] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.519372 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267303174d744009a1803a363c1e431f" } new_table_name: "default.Table_2f38c7848c3140fbac5000b908fe7eae" modify_external_catalogs: true
I20260811 18:57:14.522282 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 807cee88c18340dbb23c51cad54ab56d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.522948 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.523453 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.524570 4740 leader_election.cc:288] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.525323 4729 raft_consensus.cc:2781] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.525534 4729 raft_consensus.cc:482] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.527379 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1374a80d15444cb0faff339275aa8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.528067 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d4c26da86b49b8a37f2289f4ef6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.528891 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3263da75f0d84528b7c8cc55dcaeaa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.529568 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.530232 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fde25bcb40493aafb083e161aa6932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.531581 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40311cd1f327461e94eb443d83a16480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.537883 4729 raft_consensus.cc:3037] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.538915 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 40311cd1f327461e94eb443d83a16480 (DEFAULT_TABLE table=default.Table_4dfa50c8631f4b70b6b743837d5a9032 [id=01ec228d80ec40c8a1778d09edcd61b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.540172 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (DEFAULT_TABLE table=default.Table_77c2e3d67a494888be8b3405d4a06cf2 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.540414 4739 raft_consensus.cc:504] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.540835 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (DEFAULT_TABLE table=default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.541962 4739 leader_election.cc:288] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.545497 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c515c153df4317b3fb427983ee15e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:14.547870 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f61d3822fc2d4b569f223238a6464e0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f61d3822fc2d4b569f223238a6464e0a already in progress: creating tablet
I20260811 18:57:14.547641 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ab2eee21264b1aa5ba00528cae3843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.550441 4162 tablet_bootstrap.cc:492] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.550386 3888 raft_consensus.cc:3037] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.559473 4163 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.562475 4162 ts_tablet_manager.cc:1397] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.188s user 0.008s sys 0.024s
I20260811 18:57:14.562772 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40311cd1f327461e94eb443d83a16480. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.564850 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 218) took 1644 ms. Trace:
I20260811 18:57:14.588819 4163 ts_tablet_manager.cc:1397] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.006s
I20260811 18:57:14.593355 4163 raft_consensus.cc:348] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.595299 4163 raft_consensus.cc:374] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.595528 4163 raft_consensus.cc:729] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.596906 4163 consensus_queue.cc:260] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.598273 4163 ts_tablet_manager.cc:1428] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260811 18:57:14.598860 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.599399 4163 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.603276 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.603308 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 45fde25bcb40493aafb083e161aa6932 (DEFAULT_TABLE table=default.Table_575c5aca059048d68252cca6583a1f67 [id=4bc5d2d6894e40ebab164e5ffd6ac600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.564057 4162 raft_consensus.cc:348] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.624786 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68084.7us
I20260811 18:57:14.626539 4658 raft_consensus.cc:482] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.626811 4658 raft_consensus.cc:504] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.627628 4658 leader_election.cc:288] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.628855 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7251204325dd424192a6f58e18e71c34 (DEFAULT_TABLE table=default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.629367 4658 consensus_queue.cc:1035] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:14.629688 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf (DEFAULT_TABLE table=default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.631393 4163 tablet_bootstrap.cc:654] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.639753 4169 ts_tablet_manager.cc:1918] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:14.640121 4169 log.cc:1196] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/60274115fe164d85badcf2e72d640f54
I20260811 18:57:14.607082 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3263da75f0d84528b7c8cc55dcaeaa46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.641050 4169 ts_tablet_manager.cc:1939] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.642127 4162 raft_consensus.cc:374] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.642315 4162 raft_consensus.cc:729] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.642558 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6432abd4fc5c4f9ca95971ca4795e1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.642916 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7251204325dd424192a6f58e18e71c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.642777 4162 consensus_queue.cc:260] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.643565 4162 ts_tablet_manager.cc:1428] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.081s user 0.000s sys 0.003s
I20260811 18:57:14.643985 4162 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:14.610565 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7251204325dd424192a6f58e18e71c34 (DEFAULT_TABLE table=default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:14.644974 4729 raft_consensus.cc:504] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.647147 4729 leader_election.cc:288] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.648236 4658 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.648427 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807cee88c18340dbb23c51cad54ab56d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:14.649047 3891 raft_consensus.cc:3037] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.650941 3888 raft_consensus.cc:2454] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.652063 4658 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.652944 3676 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.653651 4576 raft_consensus.cc:2781] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.653980 4576 raft_consensus.cc:686] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.655087 4576 consensus_queue.cc:237] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:14.608532 4167 rpcz_store.cc:276] 0811 18:57:12.919809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.920000 (+ 191us) service_pool.cc:227] Handling call
0811 18:57:14.480345 (+1560345us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:14.480359 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.564569 (+ 84210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1342147}
I20260811 18:57:14.660842 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:14.664582 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ef393c341bb54baf87176db5b5254ec7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef393c341bb54baf87176db5b5254ec7 already in progress: creating tablet
I20260811 18:57:14.664665 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cf2375b4c64dbfae4e20003545da60" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.665225 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6432abd4fc5c4f9ca95971ca4795e1cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.670487 4658 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.673517 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 219) took 1744 ms. Trace:
I20260811 18:57:14.675731 3744 raft_consensus.cc:2454] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.679462 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45fde25bcb40493aafb083e161aa6932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.679551 4167 rpcz_store.cc:276] 0811 18:57:12.920626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:12.920709 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:14.664452 (+1743743us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:14.664478 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.665183 (+ 705us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1524526}
I20260811 18:57:14.680342 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 220) took 1269 ms. Trace:
W20260811 18:57:14.681063 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f335e50ae57547ed80b144129faaffbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f335e50ae57547ed80b144129faaffbb already in progress: creating tablet
I20260811 18:57:14.681699 4167 rpcz_store.cc:276] 0811 18:57:13.410590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.411165 (+ 575us) service_pool.cc:227] Handling call
0811 18:57:14.680043 (+1268878us) ts_tablet_manager.cc:1145] Deleting tablet 494c09652d9e408880ca606ae0b30773
0811 18:57:14.680058 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.680217 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268527}
W20260811 18:57:14.682592 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 494c09652d9e408880ca606ae0b30773 because the tablet was not found. No further retry: Not found: Tablet not found: 494c09652d9e408880ca606ae0b30773
I20260811 18:57:14.683828 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7251204325dd424192a6f58e18e71c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:14.683848 4167 tablet_replica.cc:330] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:14.684314 3824 leader_election.cc:341] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:14.685207 3824 leader_election.cc:302] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.685832 4658 raft_consensus.cc:2781] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.685988 4658 raft_consensus.cc:482] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.687242 4167 raft_consensus.cc:2229] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.687434 4658 raft_consensus.cc:3037] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.687711 4167 raft_consensus.cc:2258] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.689836 4167 ts_tablet_manager.cc:1905] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:14.690739 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 125757b8c1a6448b841c63339f392339 (table default.Table_1996443a5de84ded8919fb5c51dc6554 [id=4a64bb1d12324474a39317c1210869fc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 125757b8c1a6448b841c63339f392339 consensus state has no leader
I20260811 18:57:14.704293 4756 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:14.704892 4756 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:14.707348 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 339 } } ignore_live_leader: false dest_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" is_pre_election: true
I20260811 18:57:14.708520 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 339 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:14.723606 3891 raft_consensus.cc:2454] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:14.731932 4169 tablet_replica.cc:330] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.732283 4169 raft_consensus.cc:2229] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:14.732703 4169 pending_rounds.cc:62] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:14.732843 4169 pending_rounds.cc:68] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400100693155840 op_type: NO_OP noop_request { }
I20260811 18:57:14.733148 4169 raft_consensus.cc:2866] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:14.733346 4169 raft_consensus.cc:2258] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.734015 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 60274115fe164d85badcf2e72d640f54 (table default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]) successfully deleted
I20260811 18:57:14.734490 4169 ts_tablet_manager.cc:1905] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:14.734826 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9bc5448a555944dfba8ccae09e25cbe9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bc5448a555944dfba8ccae09e25cbe9 already in progress: creating tablet
I20260811 18:57:14.735411 4739 raft_consensus.cc:482] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.738466 3676 leader_election.cc:302] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.738554 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
W20260811 18:57:14.740767 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 54461.4us
I20260811 18:57:14.741855 4163 tablet_bootstrap.cc:492] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:14.742309 4163 ts_tablet_manager.cc:1397] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.143s user 0.005s sys 0.008s
I20260811 18:57:14.744073 4163 raft_consensus.cc:348] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.744447 4163 raft_consensus.cc:374] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:14.744664 4163 raft_consensus.cc:729] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:14.745772 4729 raft_consensus.cc:2781] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.745254 4739 raft_consensus.cc:504] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.749279 4756 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:14.752321 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet fbba1923ce0049448fe4e28f2dd258dc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.753898 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508f0b39cd654aa991a2928a4b067762" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:14.754208 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 151df87806694f46be5013592c68c01a
I20260811 18:57:14.754308 3891 raft_consensus.cc:2454] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:14.745280 4163 consensus_queue.cc:260] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.760242 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.763062 4163 ts_tablet_manager.cc:1428] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.000s sys 0.003s
I20260811 18:57:14.771055 4163 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:14.778507 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 2f9fa42654cf4ec18220e5fb173ddef7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:14.779376 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0859cafbc76b4eae9fe4453842a95178" } modify_external_catalogs: true
I20260811 18:57:14.780402 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0859cafbc76b4eae9fe4453842a95178" } modify_external_catalogs: true
W20260811 18:57:14.780236 4381 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.124:44143, user_credentials={real_user=slave}} blocked reactor thread for 70946.5us
I20260811 18:57:14.786280 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_53cd8000542843c984903cd25e451bdf" } new_table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" modify_external_catalogs: true
I20260811 18:57:14.790138 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cdd3a653fc1b405cbfd89c5d98728e45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.791041 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cdd3a653fc1b405cbfd89c5d98728e45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.791046 4756 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:14.794299 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.795280 4163 tablet_bootstrap.cc:654] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.796810 4739 leader_election.cc:288] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:14.797367 4704 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:14.797686 4729 raft_consensus.cc:686] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:14.798054 4704 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: loading cluster ID for follower catalog manager: success
I20260811 18:57:14.799016 4729 consensus_queue.cc:237] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.801538 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.809217 3676 leader_election.cc:302] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.810732 4162 tablet_bootstrap.cc:654] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:14.818806 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4687b95728fc485fa0e842bc447eb83f
I20260811 18:57:14.829703 4169 ts_tablet_manager.cc:1918] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:14.830484 4169 log.cc:1196] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:14.831396 4169 ts_tablet_manager.cc:1939] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:14.831866 4763 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:14.832314 4763 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.833662 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.833858 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.834120 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.834833 4763 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:14.835325 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d62ce6ce67e045c3a28f6c71419187b5" } new_table_name: "default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3" modify_external_catalogs: true
I20260811 18:57:14.839375 4704 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: acquiring CA information for follower catalog manager: success
W20260811 18:57:14.840747 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 7215ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:14.842674 4658 raft_consensus.cc:504] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:14.848261 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53475.2us
I20260811 18:57:14.851058 4658 leader_election.cc:288] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:14.856086 3676 consensus_peers.cc:487] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0ba169c32ef48b984e649b211ee2af1. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:14.857503 4739 raft_consensus.cc:2781] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.859315 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80cf2375b4c64dbfae4e20003545da60" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:14.859601 3745 raft_consensus.cc:3037] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.863816 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.862087 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33979de5b48c4674a46c8da3b7151753" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:14.865533 4704 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:14.866279 4739 raft_consensus.cc:482] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.867288 3744 raft_consensus.cc:2454] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:14.868175 3824 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.871337 4739 raft_consensus.cc:3037] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.872457 4658 raft_consensus.cc:2781] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:14.872737 4658 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:14.872912 4658 raft_consensus.cc:3037] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:14.895556 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
W20260811 18:57:14.899354 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c08436f33db043b1bc6395cd92102e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c08436f33db043b1bc6395cd92102e59 already in progress: creating tablet
W20260811 18:57:14.906484 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 02808605ee6c4634af47fc3fd57e9cd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02808605ee6c4634af47fc3fd57e9cd8 already in progress: creating tablet
W20260811 18:57:14.910087 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8c07150e31994bdf9a00ac2e7a954691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c07150e31994bdf9a00ac2e7a954691 already in progress: creating tablet
I20260811 18:57:14.911738 4169 tablet_replica.cc:330] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:14.912189 4169 raft_consensus.cc:2229] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:14.912660 4169 raft_consensus.cc:2258] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:14.913969 4169 ts_tablet_manager.cc:1905] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:14.929283 3745 raft_consensus.cc:2454] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:14.930330 3824 leader_election.cc:302] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:14.930963 4763 raft_consensus.cc:2781] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:14.931341 4763 raft_consensus.cc:686] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:14.931890 4763 consensus_queue.cc:237] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:14.940135 4162 tablet_bootstrap.cc:492] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:14.940423 4162 ts_tablet_manager.cc:1397] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.297s user 0.000s sys 0.011s
W20260811 18:57:14.981016 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bd39dcc0bed4623a06f00a56b740dba has no consensus state
I20260811 18:57:14.983256 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ff1be026ca404eada53f60f257ea367c
I20260811 18:57:14.985773 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:14.987586 4770 tablet.cc:1722] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:14.989581 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b41e9593787430fb7137bed80d6f078" } new_table_name: "default.Table_b9bdd685e23249119bbeac4997cbbf9c" modify_external_catalogs: true
I20260811 18:57:14.993271 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c37e3fb34b3f446d854006b05dbe5a94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:14.994095 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c37e3fb34b3f446d854006b05dbe5a94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:14.998102 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]). Expected version 2 got 0
I20260811 18:57:14.999747 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59cd4e34981541759a761fe10bf57a2c" } new_table_name: "default.Table_2bb9c598e2de422b86a8304bbece96db" modify_external_catalogs: true
I20260811 18:57:15.010043 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:14.943213 4162 raft_consensus.cc:348] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.015707 4162 raft_consensus.cc:374] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.016063 4162 raft_consensus.cc:729] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.016567 4162 consensus_queue.cc:260] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.017505 4162 ts_tablet_manager.cc:1428] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.077s user 0.003s sys 0.002s
I20260811 18:57:15.018118 4162 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.031839 4658 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.032785 4768 tablet.cc:1722] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.033599 4658 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.041458 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 9007f2c1f860449cbfda3986b307656f
I20260811 18:57:15.051326 4163 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.051864 4163 ts_tablet_manager.cc:1397] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.281s user 0.027s sys 0.064s
I20260811 18:57:15.053409 4163 raft_consensus.cc:348] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.053687 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet ff1be026ca404eada53f60f257ea367c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.054081 4163 raft_consensus.cc:374] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.054320 4163 raft_consensus.cc:729] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.054740 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a311fc9dc1724795a63f4df79f8ebcbc" } modify_external_catalogs: true
I20260811 18:57:15.055140 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a311fc9dc1724795a63f4df79f8ebcbc" } modify_external_catalogs: true
I20260811 18:57:15.055167 4163 consensus_queue.cc:260] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.057701 4163 ts_tablet_manager.cc:1428] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260811 18:57:15.058290 4163 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.065850 4163 tablet_bootstrap.cc:654] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:15.071933 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e51a08e6dbf04e3fa58689ee0cd45db3 consensus state has no leader
W20260811 18:57:15.072274 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 8041ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:15.081316 4167 ts_tablet_manager.cc:1918] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:15.089502 4163 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.089821 4163 ts_tablet_manager.cc:1397] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.011s
I20260811 18:57:15.094084 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5409ed17441242119ff946163da07a21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.096004 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5409ed17441242119ff946163da07a21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.097157 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.096990 4163 raft_consensus.cc:348] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.097347 4163 raft_consensus.cc:374] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.097536 4163 raft_consensus.cc:729] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.097977 4163 consensus_queue.cc:260] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.102160 4163 ts_tablet_manager.cc:1428] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20260811 18:57:15.104463 4163 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.079878 4739 raft_consensus.cc:504] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.108778 4729 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.109357 4163 tablet_bootstrap.cc:654] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.112577 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "508f0b39cd654aa991a2928a4b067762" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.112903 3891 raft_consensus.cc:3037] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.135679 3891 raft_consensus.cc:2454] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.136343 4658 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.136672 4658 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.137583 4658 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.140025 3676 leader_election.cc:302] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.140658 4576 raft_consensus.cc:2781] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.141047 4167 log.cc:1196] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/60274115fe164d85badcf2e72d640f54
I20260811 18:57:15.141395 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33979de5b48c4674a46c8da3b7151753" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:15.142366 4739 leader_election.cc:288] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.142704 4576 raft_consensus.cc:686] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.143129 4576 consensus_queue.cc:237] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.145352 3740 raft_consensus.cc:3037] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.130249 4729 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.155591 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.156824 3891 raft_consensus.cc:2454] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.165807 3676 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.166419 4729 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.167591 4729 raft_consensus.cc:2781] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.167168 4167 ts_tablet_manager.cc:1939] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:15.170625 4729 raft_consensus.cc:482] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:15.171249 4729 raft_consensus.cc:3037] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:15.182710 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 76094.1us
I20260811 18:57:15.188937 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fbba1923ce0049448fe4e28f2dd258dc
I20260811 18:57:15.189574 3549 catalog_manager.cc:6565] ReplaceTablet: tablet fbba1923ce0049448fe4e28f2dd258dc deleted and replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:15.195484 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.198758 4163 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.199092 4163 ts_tablet_manager.cc:1397] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.095s user 0.015s sys 0.027s
I20260811 18:57:15.199980 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.200523 4163 raft_consensus.cc:348] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.200919 4163 raft_consensus.cc:374] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.201100 4163 raft_consensus.cc:729] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.201582 4163 consensus_queue.cc:260] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.206022 4163 ts_tablet_manager.cc:1428] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 18:57:15.206609 4163 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.208272 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.208591 4739 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.209043 4739 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.209928 4739 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.210899 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.211297 3888 raft_consensus.cc:2454] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.211891 4162 tablet_bootstrap.cc:654] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.211920 3676 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.212635 4739 raft_consensus.cc:2781] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.212817 4739 raft_consensus.cc:482] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:15.213002 4739 raft_consensus.cc:3037] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.213647 4163 tablet_bootstrap.cc:654] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.224627 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.225486 4163 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.225662 4163 ts_tablet_manager.cc:1397] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.003s
I20260811 18:57:15.226583 4163 raft_consensus.cc:348] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.226958 4163 raft_consensus.cc:374] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.227115 4163 raft_consensus.cc:729] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.227409 4163 consensus_queue.cc:260] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.230765 4163 ts_tablet_manager.cc:1428] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:57:15.231217 4163 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.234150 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f640c60e81b4a369458b1497a269c9c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.237787 4163 tablet_bootstrap.cc:654] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:15.242750 3676 consensus_peers.cc:487] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:15.255019 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 221) took 1843 ms. Trace:
I20260811 18:57:15.255188 4167 rpcz_store.cc:276] 0811 18:57:13.411019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.415153 (+ 4134us) service_pool.cc:227] Handling call
0811 18:57:14.682014 (+1266861us) ts_tablet_manager.cc:1145] Deleting tablet 60274115fe164d85badcf2e72d640f54
0811 18:57:14.682027 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:14.931971 (+249944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.031658 (+ 99687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.254922 (+223264us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1258213}
I20260811 18:57:15.255796 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 222) took 1783 ms. Trace:
I20260811 18:57:15.255910 4167 rpcz_store.cc:276] 0811 18:57:13.472609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.472686 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:15.255575 (+1782889us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:15.255589 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.255734 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1782491}
I20260811 18:57:15.256345 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 223) took 1692 ms. Trace:
I20260811 18:57:15.256459 4167 rpcz_store.cc:276] 0811 18:57:13.564094 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.564196 (+ 102us) service_pool.cc:227] Handling call
0811 18:57:15.256180 (+1691984us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:15.256192 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.256289 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1691456}
I20260811 18:57:15.256920 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 224) took 1669 ms. Trace:
I20260811 18:57:15.256950 4163 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.257036 4167 rpcz_store.cc:276] 0811 18:57:13.587553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.587626 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:15.256732 (+1669106us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:15.256744 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.256868 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1668714}
I20260811 18:57:15.257208 4163 ts_tablet_manager.cc:1397] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.000s
I20260811 18:57:15.257382 4167 tablet_replica.cc:330] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:15.257651 4167 raft_consensus.cc:2229] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:15.258265 4163 raft_consensus.cc:348] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.258643 4163 raft_consensus.cc:374] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.258733 4163 raft_consensus.cc:729] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.259123 4163 consensus_queue.cc:260] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.261822 4163 ts_tablet_manager.cc:1428] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:15.262316 4163 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.263171 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 60274115fe164d85badcf2e72d640f54 (table default.Table_fe796d2df98748a1a996fd9e9e0cf128 [id=fed9b2a1b8de4318bb4f5da4e7234f98]) successfully deleted
W20260811 18:57:15.263504 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06b8f2a97c0f46ea8c56b3de2b97e60c already in progress: creating tablet
W20260811 18:57:15.263736 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 490f34d7a403416fbab64e46973746de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 490f34d7a403416fbab64e46973746de already in progress: creating tablet
W20260811 18:57:15.264008 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 125757b8c1a6448b841c63339f392339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 125757b8c1a6448b841c63339f392339 already in progress: creating tablet
I20260811 18:57:15.265262 4167 raft_consensus.cc:2258] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.266079 4167 ts_tablet_manager.cc:1905] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.266777 4163 tablet_bootstrap.cc:654] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.270197 3740 raft_consensus.cc:2454] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:15.270977 3824 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.271376 4658 raft_consensus.cc:2781] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.271631 4658 raft_consensus.cc:686] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:15.271986 4658 consensus_queue.cc:237] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.282989 4162 tablet_bootstrap.cc:492] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.283253 4162 ts_tablet_manager.cc:1397] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.265s user 0.005s sys 0.010s
I20260811 18:57:15.284157 4162 raft_consensus.cc:348] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.284488 4162 raft_consensus.cc:374] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.284587 4162 raft_consensus.cc:729] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.284945 4162 consensus_queue.cc:260] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.285552 4162 ts_tablet_manager.cc:1428] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:15.286065 4162 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.286499 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet b0b199f4c9e14b468cb26cbd61fc8586 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.288072 4162 tablet_bootstrap.cc:654] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.288923 4658 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.289192 4658 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.297816 4658 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.299924 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:15.300658 3824 leader_election.cc:341] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:15.302894 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6070ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:15.313436 4163 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.313295 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.313704 4163 ts_tablet_manager.cc:1397] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.005s
W20260811 18:57:15.314193 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a303b89a5ef54a7ea92f2bef6298d619 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.315037 4163 raft_consensus.cc:348] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.315317 4162 tablet_bootstrap.cc:492] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.315367 4163 raft_consensus.cc:374] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.315585 4163 raft_consensus.cc:729] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.315593 4162 ts_tablet_manager.cc:1397] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260811 18:57:15.315976 4163 consensus_queue.cc:260] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.317094 4162 raft_consensus.cc:348] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.317512 4162 raft_consensus.cc:374] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.317725 4162 raft_consensus.cc:729] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.319545 4163 ts_tablet_manager.cc:1428] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:15.320020 4163 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.322670 4162 consensus_queue.cc:260] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.330422 4162 ts_tablet_manager.cc:1428] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.002s sys 0.002s
I20260811 18:57:15.330997 4162 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.330778 4729 raft_consensus.cc:504] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.331889 4729 leader_election.cc:288] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.332717 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ce1042ae974f4dbac964bd7ea16a94" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.333029 3745 raft_consensus.cc:2379] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
I20260811 18:57:15.333061 3891 raft_consensus.cc:3037] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.337256 3891 raft_consensus.cc:2454] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.346547 4163 tablet_bootstrap.cc:654] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.347111 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.347769 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.349640 3676 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.354107 4576 raft_consensus.cc:2781] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.354313 4576 raft_consensus.cc:686] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.354705 4576 consensus_queue.cc:237] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.355779 4163 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.356011 4163 ts_tablet_manager.cc:1397] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.007s
I20260811 18:57:15.357311 4163 raft_consensus.cc:348] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.357666 4163 raft_consensus.cc:374] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.357758 4163 raft_consensus.cc:729] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.358075 4163 consensus_queue.cc:260] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.361004 4739 raft_consensus.cc:504] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.361683 4163 ts_tablet_manager.cc:1428] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260811 18:57:15.362025 4163 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.362414 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed3346bd12d04c45a511308c427856e5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:15.362630 3891 raft_consensus.cc:3037] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.363121 4739 leader_election.cc:288] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.363901 4163 tablet_bootstrap.cc:654] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.365648 4162 tablet_bootstrap.cc:654] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.368017 3891 raft_consensus.cc:2454] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:15.368517 3676 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:15.368943 4739 raft_consensus.cc:2781] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:15.369321 4739 raft_consensus.cc:686] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:15.369887 4739 consensus_queue.cc:237] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.382799 4163 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.383031 4163 ts_tablet_manager.cc:1397] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.001s
I20260811 18:57:15.384140 4163 raft_consensus.cc:348] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.384400 4163 raft_consensus.cc:374] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.384517 4163 raft_consensus.cc:729] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.384783 4163 consensus_queue.cc:260] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.389657 4163 ts_tablet_manager.cc:1428] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 18:57:15.390017 4163 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.393390 4163 tablet_bootstrap.cc:654] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.401964 4169 ts_tablet_manager.cc:1918] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.402280 4169 log.cc:1196] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:15.412606 4162 tablet_bootstrap.cc:492] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:15.417853 4162 ts_tablet_manager.cc:1397] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.087s user 0.004s sys 0.004s
W20260811 18:57:15.419087 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73f05e1d5cb947fe9dbcc2e876a23788 has no consensus state
I20260811 18:57:15.421239 4162 raft_consensus.cc:348] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.423362 4162 raft_consensus.cc:374] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.423514 4162 raft_consensus.cc:729] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.424160 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.424989 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b48fafe85a504185911a1e7edb4fcbfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:15.431068 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 866ed614511649459b22a9fce55220b2 consensus state has no leader
I20260811 18:57:15.432998 4167 ts_tablet_manager.cc:1918] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.435113 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.435817 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 8336ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
W20260811 18:57:15.435827 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.447005 3554 catalog_manager.cc:5495] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:15.447556 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:15.456494 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0e1efffade0f486d8d627e56e0be6ff6 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]). Expected version 2 got 0
I20260811 18:57:15.458994 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (DEFAULT_TABLE table=default.Table_b9bdd685e23249119bbeac4997cbbf9c [id=0ea510ab7a724cadb94d2cfbc5fde4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.459403 4658 raft_consensus.cc:482] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.459563 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f05e1d5cb947fe9dbcc2e876a23788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.459622 4658 raft_consensus.cc:504] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:15.460351 4658 leader_election.cc:288] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.461732 4169 ts_tablet_manager.cc:1939] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:15.462133 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 247) took 1083 ms. Trace:
I20260811 18:57:15.462260 4169 rpcz_store.cc:276] 0811 18:57:14.378801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.379400 (+ 599us) service_pool.cc:227] Handling call
0811 18:57:14.911507 (+532107us) ts_tablet_manager.cc:1145] Deleting tablet e7b3a14979e84b1e8ec6ed3f49b93b00
0811 18:57:14.911526 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.243329 (+331803us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.401707 (+158378us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.462083 (+ 60376us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":531610}
I20260811 18:57:15.463114 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_005d29915a3b4449acfc665e917be061"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.463884 4169 tablet_replica.cc:330] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:15.463924 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_005d29915a3b4449acfc665e917be061 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.464148 4169 raft_consensus.cc:2229] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:15.464413 4169 pending_rounds.cc:62] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:15.464496 4169 pending_rounds.cc:68] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400104684687360 op_type: NO_OP noop_request { }
I20260811 18:57:15.464632 4169 raft_consensus.cc:2866] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:15.464761 4169 raft_consensus.cc:2258] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.465481 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]) successfully deleted
I20260811 18:57:15.463177 4163 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.466743 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a122fe2820f4d23b12edb970a94e958"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.467089 4163 ts_tablet_manager.cc:1397] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.012s
W20260811 18:57:15.467480 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a122fe2820f4d23b12edb970a94e958 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.467922 4658 raft_consensus.cc:482] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.468055 4658 raft_consensus.cc:504] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.468827 4658 leader_election.cc:288] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.469218 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" } modify_external_catalogs: true
I20260811 18:57:15.469556 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bbeefd35da1a4d1584009f5edafc4642" } modify_external_catalogs: true
I20260811 18:57:15.469789 4163 raft_consensus.cc:348] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.470093 4163 raft_consensus.cc:374] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.470247 4163 raft_consensus.cc:729] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.472847 4163 consensus_queue.cc:260] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.477246 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13fbac538d16421e921aa37f54683662" } modify_external_catalogs: true
I20260811 18:57:15.477602 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13fbac538d16421e921aa37f54683662" } modify_external_catalogs: true
I20260811 18:57:15.478901 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (DEFAULT_TABLE table=default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.479661 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45aff3e8cb0c4f7191e4522c6ef7d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.480172 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e738d6898cb440ca15ea7cca8f58b15" } new_table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" modify_external_catalogs: true
I20260811 18:57:15.484982 4169 ts_tablet_manager.cc:1905] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.461215 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a300a7aca2814da6b999eb4d865fea66" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.473461 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (DEFAULT_TABLE table=default.Table_b9bdd685e23249119bbeac4997cbbf9c [id=0ea510ab7a724cadb94d2cfbc5fde4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.500849 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f05e1d5cb947fe9dbcc2e876a23788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.476207 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (DEFAULT_TABLE table=default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.484750 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce8db230c7e41b6a4792273a0f07e6e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.491309 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 4f3c4536fc7440db83851fc8d7b3de10 (DEFAULT_TABLE table=default.Table_e75a55498d4c4072968ee03aaf48b831 [id=bd974444a6c546959eb6442a80543187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.493860 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 61cea7b0b85448f5854e8fd99ea5e792 (DEFAULT_TABLE table=default.Table_efa46b4a159b4805b21635ef65051022 [id=20ac4cbacf8f4ec8a8a41810f1da3355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.496317 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 3c96a264e017471ea114ca0c6a9500ef (DEFAULT_TABLE table=default.Table_98051d104e9e4d9b976577088d6126a2 [id=f5844f91d9ef446bbd4d34bdea7cba8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.499119 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.509636 4167 log.cc:1196] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a0ba169c32ef48b984e649b211ee2af1
I20260811 18:57:15.510620 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45aff3e8cb0c4f7191e4522c6ef7d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.511535 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61cea7b0b85448f5854e8fd99ea5e792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.512413 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f3c4536fc7440db83851fc8d7b3de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.514984 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968fa8b8912b40a8bdba77db1d39c5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.515893 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c96a264e017471ea114ca0c6a9500ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.520220 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3c96a264e017471ea114ca0c6a9500ef (DEFAULT_TABLE table=default.Table_98051d104e9e4d9b976577088d6126a2 [id=f5844f91d9ef446bbd4d34bdea7cba8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.521083 4163 ts_tablet_manager.cc:1428] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.041s user 0.003s sys 0.000s
W20260811 18:57:15.522327 3824 leader_election.cc:341] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:15.523008 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 4f3c4536fc7440db83851fc8d7b3de10 (DEFAULT_TABLE table=default.Table_e75a55498d4c4072968ee03aaf48b831 [id=bd974444a6c546959eb6442a80543187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.525804 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 61cea7b0b85448f5854e8fd99ea5e792 (DEFAULT_TABLE table=default.Table_efa46b4a159b4805b21635ef65051022 [id=20ac4cbacf8f4ec8a8a41810f1da3355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.526218 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.530076 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c96a264e017471ea114ca0c6a9500ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.531720 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0aee6beb2e70421083841520733087f8 (DEFAULT_TABLE table=default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.533450 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61cea7b0b85448f5854e8fd99ea5e792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.534245 3862 tablet_service.cc:1467] Processing CreateTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 (DEFAULT_TABLE table=default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.536696 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968fa8b8912b40a8bdba77db1d39c5c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.538470 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aee6beb2e70421083841520733087f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.545753 4163 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.545651 3857 tablet_service.cc:1467] Processing CreateTablet for tablet b57ed7814a1e440cba362613cbab32f8 (DEFAULT_TABLE table=default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.547295 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3f120758c6405dafd2ea0bf54c81d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.548861 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57ed7814a1e440cba362613cbab32f8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:15.550277 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6246ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:15.552578 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f3c4536fc7440db83851fc8d7b3de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.560801 4770 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.564872 4768 tablet.cc:1722] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:15.572599 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 0aee6beb2e70421083841520733087f8 (DEFAULT_TABLE table=default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:15.577533 3554 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.72.65:60889 (request call id 42) took 976 ms (client timeout 1000 ms). Trace:
W20260811 18:57:15.577886 3554 rpcz_store.cc:269] 0811 18:57:14.600868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.643635 (+ 42767us) service_pool.cc:227] Handling call
0811 18:57:14.782024 (+138389us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x615000033024. stack: 00007f61e4f0a66e 00007f61e4f0a430 00007f61e3f915ed 00000000008ea064 00000000008ea02d 00007f61f362327d 00007f61f3623373 00007f61f0445361 00007f61f7db866a 00007f61f7f3ad8f 00007f61ef22f946 00007f61ef22f675 00007f61eba469be 00007f61eba4579e 00007f61eba4bb25 00007f61eba4c54f
0811 18:57:15.577383 (+795359us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.542318 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.548372 (+ 6054us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.548380 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:15.548383 (+ 3us) tablet.cc:660] Decoding operations
0811 18:57:15.548495 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.548509 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.548515 (+ 6us) tablet.cc:683] Acquiring locks for 1 operations
0811 18:57:15.548661 (+ 146us) tablet.cc:699] Row locks acquired
0811 18:57:15.548666 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:15.548732 (+ 66us) write_op.cc:270] Start()
0811 18:57:15.548783 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635548691 usec, L: 0
0811 18:57:15.548789 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.549073 (+ 284us) log.cc:842] Serialized 527 byte log entry
0811 18:57:15.574469 (+ 25396us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.574713 (+ 244us) write_op.cc:301] APPLY: starting
0811 18:57:15.574728 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.574763 (+ 35us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.574813 (+ 50us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.574911 (+ 98us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.574950 (+ 39us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:15.574957 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:15.575049 (+ 92us) log.cc:842] Serialized 23 byte log entry
0811 18:57:15.575147 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.575180 (+ 33us) write_op.cc:454] Released schema lock
0811 18:57:15.575371 (+ 191us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.575458 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.575465 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":316642176,"child_traces":[["op",{"apply.queue_time_us":190,"apply.run_cpu_time_us":825,"apply.run_wall_time_us":825,"mutex_wait_us":1851,"num_ops":1,"prepare.queue_time_us":94063,"prepare.run_cpu_time_us":1294,"prepare.run_wall_time_us":35757,"raft.queue_time_us":15481,"raft.run_cpu_time_us":778,"raft.run_wall_time_us":779,"replication_time_us":25647}]]}
I20260811 18:57:15.586459 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd120e0ab3b34418960cefc5eb459cdb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.587258 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd120e0ab3b34418960cefc5eb459cdb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.598248 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b (DEFAULT_TABLE table=default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.598431 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:15.599031 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 2f9fa42654cf4ec18220e5fb173ddef7 deleted and replaced by tablet bb56d23ccabc4b02961249312257d9c4
I20260811 18:57:15.599340 3555 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:38308 (request call id 366) took 1122 ms. Trace:
I20260811 18:57:15.599799 3555 rpcz_store.cc:276] 0811 18:57:14.476954 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.778410 (+301456us) service_pool.cc:227] Handling call
0811 18:57:15.599149 (+820739us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.578678 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.578858 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.578865 (+ 7us) write_op.cc:435] Acquired schema lock
0811 18:57:15.578867 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:15.579045 (+ 178us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.579072 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.579078 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:15.579203 (+ 125us) tablet.cc:699] Row locks acquired
0811 18:57:15.579207 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:15.579265 (+ 58us) write_op.cc:270] Start()
0811 18:57:15.579327 (+ 62us) write_op.cc:276] Timestamp: P: 1786474635579230 usec, L: 0
0811 18:57:15.579333 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.585883 (+ 6550us) log.cc:842] Serialized 622 byte log entry
0811 18:57:15.596753 (+ 10870us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.596909 (+ 156us) write_op.cc:301] APPLY: starting
0811 18:57:15.596928 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.596992 (+ 64us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.597137 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.597292 (+ 155us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.597372 (+ 80us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0811 18:57:15.597384 (+ 12us) write_op.cc:312] APPLY: finished
0811 18:57:15.597504 (+ 120us) log.cc:842] Serialized 29 byte log entry
0811 18:57:15.597595 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.597621 (+ 26us) write_op.cc:454] Released schema lock
0811 18:57:15.597631 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.597701 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.597708 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":867,"apply.run_wall_time_us":866,"num_ops":2,"prepare.queue_time_us":128890,"prepare.run_cpu_time_us":1187,"prepare.run_wall_time_us":16420,"raft.queue_time_us":554,"raft.run_cpu_time_us":938,"raft.run_wall_time_us":938,"replication_time_us":17339}]]}
I20260811 18:57:15.601220 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c038c8ee6e694c6f99d5cb717ba90ca9" } modify_external_catalogs: true
I20260811 18:57:15.601536 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c038c8ee6e694c6f99d5cb717ba90ca9" } modify_external_catalogs: true
I20260811 18:57:15.614852 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:15.615085 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d32bb4eca047e4ba786c04b8bebf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.615641 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 372) took 1033 ms. Trace:
I20260811 18:57:15.615903 3550 rpcz_store.cc:276] 0811 18:57:14.582816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.054364 (+471548us) service_pool.cc:227] Handling call
0811 18:57:15.055369 (+ 1005us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:15.055520 (+ 151us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:15.055595 (+ 75us) catalog_manager.cc:2761] Locking tablets
0811 18:57:15.450766 (+395171us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:15.614438 (+163672us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:15.614539 (+ 101us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:15.614631 (+ 92us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:15.615493 (+ 862us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.595172 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:15.595306 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:15.595312 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:15.595314 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:15.595462 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:15.595476 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:15.595482 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:15.595660 (+ 178us) tablet.cc:699] Row locks acquired
0811 18:57:15.595665 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:15.595717 (+ 52us) write_op.cc:270] Start()
0811 18:57:15.605463 (+ 9746us) write_op.cc:276] Timestamp: P: 1786474635595679 usec, L: 0
0811 18:57:15.605470 (+ 7us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.605706 (+ 236us) log.cc:842] Serialized 725 byte log entry
0811 18:57:15.612811 (+ 7105us) op_driver.cc:444] REPLICATION: finished
0811 18:57:15.613014 (+ 203us) write_op.cc:301] APPLY: starting
0811 18:57:15.613031 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:15.613076 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:15.613118 (+ 42us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:15.613189 (+ 71us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:15.613236 (+ 47us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:15.613242 (+ 6us) write_op.cc:312] APPLY: finished
0811 18:57:15.614120 (+ 878us) log.cc:842] Serialized 29 byte log entry
0811 18:57:15.614204 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.614235 (+ 31us) write_op.cc:454] Released schema lock
0811 18:57:15.614243 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:15.614304 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:15.614307 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":152,"apply.run_cpu_time_us":631,"apply.run_wall_time_us":1368,"num_ops":2,"prepare.queue_time_us":143777,"raft.queue_time_us":2185,"raft.run_cpu_time_us":650,"raft.run_wall_time_us":651,"replication_time_us":7314}]]}
I20260811 18:57:15.619822 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91346cfabb664d0ebfc876dacc125c50" } modify_external_catalogs: true
I20260811 18:57:15.620389 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91346cfabb664d0ebfc876dacc125c50" } modify_external_catalogs: true
I20260811 18:57:15.631266 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 68a7051bc539468aafba30fbaf0669d4 (DEFAULT_TABLE table=default.Table_021124838f7841c49f01816fb948d73c [id=4a5d04ba5f3942a3942909ebd2bf5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.633623 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:15.634295 3553 catalog_manager.cc:6565] ReplaceTablet: tablet b0b199f4c9e14b468cb26cbd61fc8586 deleted and replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:15.637380 3865 tablet_service.cc:1467] Processing CreateTablet for tablet d00d943b778741b4aeec076ba46a9c40 (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.637948 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 with delete_type TABLET_DATA_DELETED (replaced by tablet bb56d23ccabc4b02961249312257d9c4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.639417 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a7051bc539468aafba30fbaf0669d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.640995 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_88ad2bea931c4ea0843ace52fcb146b3" } new_table_name: "default.Table_cbe771adb7364096802df77fa867acb0" modify_external_catalogs: true
I20260811 18:57:15.646188 4658 raft_consensus.cc:482] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.647890 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00d943b778741b4aeec076ba46a9c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.651384 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45] alter complete (version 2)
I20260811 18:57:15.652891 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 80cf2375b4c64dbfae4e20003545da60 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.659003 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2d444511ad44775b701b9ed7527e251"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.660531 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2d444511ad44775b701b9ed7527e251 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.664865 3710 tablet_service.cc:1467] Processing CreateTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 (DEFAULT_TABLE table=default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.663108 4576 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.669095 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aee6beb2e70421083841520733087f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.669348 4162 consensus_queue.cc:260] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.670281 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_220d78aeb25b4b64b594e0839928c35e" } new_table_name: "default.Table_ae88bb870eb143408638ffe3715fbf48" modify_external_catalogs: true
I20260811 18:57:15.653435 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd39dcc0bed4623a06f00a56b740dba (DEFAULT_TABLE table=default.Table_2f38c7848c3140fbac5000b908fe7eae [id=f312f8bcd459400b92a33532675d3991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.690121 4658 raft_consensus.cc:504] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.690943 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6bf41429faeb4785ad747d207e4641f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.691761 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bf41429faeb4785ad747d207e4641f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.697466 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd39dcc0bed4623a06f00a56b740dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.702185 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:15.703562 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.703970 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:15.705854 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_fce143fe6d0849a0a3af8939df804986 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:15.707434 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.707705 4163 tablet_bootstrap.cc:654] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.708560 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2f38c7848c3140fbac5000b908fe7eae" } new_table_name: "default.Table_28ed0eb238674836950a5639ebcbc6b9" modify_external_catalogs: true
I20260811 18:57:15.711599 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.715760 4658 leader_election.cc:288] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.719605 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:15.721724 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cec1a44bf54f48019648c9338236fcc0" } new_table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" modify_external_catalogs: true
I20260811 18:57:15.726428 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 059b325df1db4fde9d4b26b559121316 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.727077 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b325df1db4fde9d4b26b559121316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.743011 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.746801 3722 tablet_service.cc:1467] Processing CreateTablet for tablet b57ed7814a1e440cba362613cbab32f8 (DEFAULT_TABLE table=default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.747776 4729 consensus_queue.cc:1035] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.129s
I20260811 18:57:15.762185 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b (DEFAULT_TABLE table=default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.762996 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc3f120758c6405dafd2ea0bf54c81d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.778739 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 80cf2375b4c64dbfae4e20003545da60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.779232 4162 ts_tablet_manager.cc:1428] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.342s user 0.003s sys 0.000s
W20260811 18:57:15.781948 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6f36ab6bb774257b5c2692c64eff63d consensus state has no leader
I20260811 18:57:15.782800 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7b9dec45633841ac830523612293aabf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.783895 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b9dec45633841ac830523612293aabf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.784636 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:15.786123 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:15.787206 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3e16ca699564a61a49c4cb989ad865f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.788069 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3e16ca699564a61a49c4cb989ad865f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.789809 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:15.790756 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_133a44e94783442e8c27d4730310f996"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.791425 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_133a44e94783442e8c27d4730310f996 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.795080 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b57ed7814a1e440cba362613cbab32f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.796926 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.789825 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e8859199c3440ebb2b9664b65e0fb71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.800547 4167 ts_tablet_manager.cc:1939] T a0ba169c32ef48b984e649b211ee2af1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:15.805369 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.798955 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 508f0b39cd654aa991a2928a4b067762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.807847 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d32bb4eca047e4ba786c04b8bebf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.809309 3740 raft_consensus.cc:2454] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:15.809964 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde27d8cc3eb4d09879e717fc90305fd" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:15.810649 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.810949 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 68a7051bc539468aafba30fbaf0669d4 (DEFAULT_TABLE table=default.Table_021124838f7841c49f01816fb948d73c [id=4a5d04ba5f3942a3942909ebd2bf5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.811609 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a7051bc539468aafba30fbaf0669d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:15.814765 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23f41a2638e74ba19c1c527b9fc796df consensus state has no leader
I20260811 18:57:15.818182 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:15.818683 4162 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:15.829174 3824 leader_election.cc:302] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:15.829936 3824 leader_election.cc:341] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:15.830193 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:15.833873 4658 raft_consensus.cc:2781] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:15.834360 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d00d943b778741b4aeec076ba46a9c40 (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.834980 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00d943b778741b4aeec076ba46a9c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.836705 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9fa42654cf4ec18220e5fb173ddef7 with delete_type TABLET_DATA_DELETED (replaced by tablet bb56d23ccabc4b02961249312257d9c4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.836896 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 225) took 2193 ms. Trace:
I20260811 18:57:15.840718 4658 raft_consensus.cc:482] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:15.842557 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e8859199c3440ebb2b9664b65e0fb71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.848526 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.849548 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.850132 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3370
W20260811 18:57:15.850804 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71df3738a5cf4b12820dda131ced62e1 consensus state has no leader
I20260811 18:57:15.852005 3436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:15.852015 4167 rpcz_store.cc:276] 0811 18:57:13.608351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.608473 (+ 122us) service_pool.cc:227] Handling call
0811 18:57:15.257284 (+1648811us) ts_tablet_manager.cc:1145] Deleting tablet a0ba169c32ef48b984e649b211ee2af1
0811 18:57:15.257307 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.335251 (+ 77944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.432821 (+ 97570us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.801029 (+368208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1648348}
I20260811 18:57:15.852775 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 226) took 2228 ms. Trace:
I20260811 18:57:15.856858 4167 rpcz_store.cc:276] 0811 18:57:13.623956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.624038 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:15.852378 (+2228340us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:15.852393 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.852591 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2227921}
I20260811 18:57:15.857453 4167 tablet_replica.cc:330] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:15.858065 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e1efffade0f486d8d627e56e0be6ff6 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:15.858551 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f7d4c26da86b49b8a37f2289f4ef6614 consensus state has no leader
I20260811 18:57:15.860270 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1" } modify_external_catalogs: true
I20260811 18:57:15.860574 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12663a37e9cf40cf8d0c9950d6a729c1" } modify_external_catalogs: true
W20260811 18:57:15.861207 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bd39dcc0bed4623a06f00a56b740dba consensus state has no leader
I20260811 18:57:15.862002 3552 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:49260 (request call id 1) took 1936 ms. Trace:
I20260811 18:57:15.862218 3552 rpcz_store.cc:276] 0811 18:57:13.925736 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.861423 (+1935687us) service_pool.cc:227] Handling call
0811 18:57:15.861628 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260811 18:57:15.862906 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8426ccbe530446fcbaa162a03cbb1892 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.872663 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4981573f7a434b5da77a06d251ea684c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:15.873124 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:15.873612 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4981573f7a434b5da77a06d251ea684c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.877954 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878362 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878690 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.878931 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:15.881908 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a0ba169c32ef48b984e649b211ee2af1 (table default.Table_4fbcb57a32f54fc3a493bc5412e94051 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:15.887876 4163 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:15.888243 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b35ea225e3114cdc9ed09c78af2f272d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.889127 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b35ea225e3114cdc9ed09c78af2f272d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.890074 4658 raft_consensus.cc:3037] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:15.908994 4658 raft_consensus.cc:504] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.911847 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" } modify_external_catalogs: true
I20260811 18:57:15.912096 4163 ts_tablet_manager.cc:1397] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.366s user 0.002s sys 0.008s
I20260811 18:57:15.912228 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_31693dc5078f45c382fb5ed270c7b32d" } modify_external_catalogs: true
I20260811 18:57:15.836540 4576 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.921749 4658 leader_election.cc:288] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.922492 4658 raft_consensus.cc:482] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.922902 4658 raft_consensus.cc:504] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.924512 4658 leader_election.cc:288] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.924588 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b9dec45633841ac830523612293aabf" } modify_external_catalogs: true
I20260811 18:57:15.924875 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b9dec45633841ac830523612293aabf" } modify_external_catalogs: true
I20260811 18:57:15.924957 4658 raft_consensus.cc:482] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.925170 4658 raft_consensus.cc:504] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.926136 4658 leader_election.cc:288] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.927659 4162 tablet_bootstrap.cc:654] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.933044 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:15.933532 3888 raft_consensus.cc:2454] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:15.939993 4800 raft_consensus.cc:482] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:15.940514 4800 raft_consensus.cc:504] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.942162 4800 leader_election.cc:288] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:15.949836 4163 raft_consensus.cc:348] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.950331 4163 raft_consensus.cc:374] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.950632 4163 raft_consensus.cc:729] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.953994 4163 consensus_queue.cc:260] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:15.955219 4163 ts_tablet_manager.cc:1428] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.043s user 0.003s sys 0.000s
I20260811 18:57:15.960588 4169 ts_tablet_manager.cc:1918] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:15.960868 4169 log.cc:1196] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/80cf2375b4c64dbfae4e20003545da60
I20260811 18:57:15.961946 4169 ts_tablet_manager.cc:1939] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:15.962718 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 248) took 1516 ms. Trace:
I20260811 18:57:15.963003 4169 rpcz_store.cc:276] 0811 18:57:14.446125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.446200 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:15.463816 (+1017616us) ts_tablet_manager.cc:1145] Deleting tablet 80cf2375b4c64dbfae4e20003545da60
0811 18:57:15.463829 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.744741 (+280912us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.948833 (+204092us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:15.962654 (+ 13821us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1017208}
I20260811 18:57:15.963747 4163 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:15.963726 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 80cf2375b4c64dbfae4e20003545da60 (table default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]) successfully deleted
I20260811 18:57:15.975049 4169 tablet_replica.cc:330] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:15.975553 4169 raft_consensus.cc:2229] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:15.975911 4169 raft_consensus.cc:2258] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:15.976817 4169 ts_tablet_manager.cc:1905] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:15.978128 4167 raft_consensus.cc:2229] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:15.978477 4576 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:15.980146 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd39dcc0bed4623a06f00a56b740dba (DEFAULT_TABLE table=default.Table_2f38c7848c3140fbac5000b908fe7eae [id=f312f8bcd459400b92a33532675d3991]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.980304 4163 tablet_bootstrap.cc:654] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.980795 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd39dcc0bed4623a06f00a56b740dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.987360 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet e51a08e6dbf04e3fa58689ee0cd45db3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:15.978415 4740 consensus_queue.cc:1035] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.070s
I20260811 18:57:15.993281 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 059b325df1db4fde9d4b26b559121316 (DEFAULT_TABLE table=default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.993819 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b325df1db4fde9d4b26b559121316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.999471 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c6377231db8495796c02e71f7f71991"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.000353 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c6377231db8495796c02e71f7f71991 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.003286 4163 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.003536 4163 ts_tablet_manager.cc:1397] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.006s
I20260811 18:57:16.005201 4163 raft_consensus.cc:348] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.005666 4163 raft_consensus.cc:374] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.016821 3553 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137] alter complete (version 2)
I20260811 18:57:16.019117 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" } modify_external_catalogs: true
I20260811 18:57:16.019440 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acb61e38f2a0442688865698892cc5c9" } modify_external_catalogs: true
I20260811 18:57:16.020850 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" } modify_external_catalogs: true
I20260811 18:57:16.021136 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c68caefab77748b4821e0313f5f95a1a" } modify_external_catalogs: true
I20260811 18:57:16.023365 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0f6df7a291456d80538e2b76417873 (DEFAULT_TABLE table=default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.024147 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0f6df7a291456d80538e2b76417873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.018224 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_994d2d4469b54597a9aa33c59dca9d84" } new_table_name: "default.Table_51c66177bc494cd0a293e2959ff854e5" modify_external_catalogs: true
I20260811 18:57:16.026541 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e (DEFAULT_TABLE table=default.Table_994d2d4469b54597a9aa33c59dca9d84 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.026786 4739 consensus_queue.cc:1035] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.108s
I20260811 18:57:16.027352 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e544b4b1592c4d51a2ed0d6461edc75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.028558 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 (DEFAULT_TABLE table=default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.034554 3865 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c8d542ae2489ba466ffadd518db12 (DEFAULT_TABLE table=default.Table_5409ed17441242119ff946163da07a21 [id=97d17499bd684f0abf98e0582fa06f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.035338 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 (DEFAULT_TABLE table=default.Table_e746326e8bff404aa5b3bc1fac099489 [id=bebb3efbbc56416597abdf4a0b3952ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.036051 3856 tablet_service.cc:1467] Processing CreateTablet for tablet a847581412494e34b2c76e605ff11772 (DEFAULT_TABLE table=default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.036641 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 4814a3562ef14fb0bec79be8c3cff4d2 (DEFAULT_TABLE table=default.Table_858da7e74d4141a19ced6fdc29c33d89 [id=613069b1ba8145baa7e3614a2c5b59be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.037061 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 (DEFAULT_TABLE table=default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.038177 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.039359 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 (DEFAULT_TABLE table=default.Table_c37e3fb34b3f446d854006b05dbe5a94 [id=5fc5ade3f38c438d9b501127ac866b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.039497 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d (DEFAULT_TABLE table=default.Table_b07958bac6344da1bbf3d7d35f1ac551 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.040148 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7057b8a7bae249caaed304b5c66e6871 (DEFAULT_TABLE table=default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.041949 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5d495354c0994fb4a74c376078e41a26 (DEFAULT_TABLE table=default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.044173 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:16.045454 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 490f34d7a403416fbab64e46973746de from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.046118 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 799ba9a044e145ec89d0025c03f391ea from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.046635 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce94d37efd2245bf8cc71fdd90bd9735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047010 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7057b8a7bae249caaed304b5c66e6871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047389 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4814a3562ef14fb0bec79be8c3cff4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047499 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.047763 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d495354c0994fb4a74c376078e41a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.047923 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.048143 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ceca57cb245f4a3a6cb4b36f1306d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.049048 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b404e99e9414c67b3cd5aff12203a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.049296 4167 raft_consensus.cc:2258] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.050314 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cccebfc54484a9b9abffcce4e459fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.058468 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cce97bfdb1d41e685e091c8c5cc6946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.058406 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce8db230c7e41b6a4792273a0f07e6e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:16.063184 3552 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 22071cd82e0645939e21f0be945e0eef
I20260811 18:57:16.064849 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0b3ec38f9b55489ba00ed76f9e4456f5
W20260811 18:57:16.064539 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 52402.4us
I20260811 18:57:16.066149 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f97f37fa7c483cacaeed54f93e7a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.069151 4163 raft_consensus.cc:729] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.069744 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_77c2e3d67a494888be8b3405d4a06cf2" } new_table_name: "default.Table_655dd0bc17cc45d3a344bb5c02ff7f80" modify_external_catalogs: true
I20260811 18:57:16.070389 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:16.071002 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0e1efffade0f486d8d627e56e0be6ff6 deleted and replaced by tablet 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:16.075160 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c8d542ae2489ba466ffadd518db12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.076524 3870 tablet_service.cc:1467] Processing CreateTablet for tablet faf0594d55844ae49c99247ecd8b810f (DEFAULT_TABLE table=default.Table_6d87a15f366a4e118c84e294c3b9926e [id=5a23d06641cb447fb634032bd768498b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.078080 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9fa42654cf4ec18220e5fb173ddef7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.080222 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847581412494e34b2c76e605ff11772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.082827 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.082736 4163 consensus_queue.cc:260] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.085798 4169 ts_tablet_manager.cc:1918] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:16.092288 3714 consensus_meta.cc:287] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent flushing consensus metadata: real 0.572s user 0.002s sys 0.003s
W20260811 18:57:16.098001 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 7892ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:16.103914 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1efffade0f486d8d627e56e0be6ff6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.107035 4167 ts_tablet_manager.cc:1905] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.111280 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae200f3814e431a8288c30fe94bffcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.112900 3706 tablet_service.cc:1467] Processing CreateTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 (DEFAULT_TABLE table=default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.115787 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 4814a3562ef14fb0bec79be8c3cff4d2 (DEFAULT_TABLE table=default.Table_858da7e74d4141a19ced6fdc29c33d89 [id=613069b1ba8145baa7e3614a2c5b59be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.116480 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0f6df7a291456d80538e2b76417873 (DEFAULT_TABLE table=default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.116737 3745 raft_consensus.cc:3037] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.083921 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.141305 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf0594d55844ae49c99247ecd8b810f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.142432 4169 log.cc:1196] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/807cee88c18340dbb23c51cad54ab56d
I20260811 18:57:16.143404 4169 ts_tablet_manager.cc:1939] T 807cee88c18340dbb23c51cad54ab56d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.143437 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0f6df7a291456d80538e2b76417873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.144353 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 with delete_type TABLET_DATA_DELETED (replaced by tablet 27cda53b7ca54346a7a22bf26f5db990) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.147993 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 256) took 1624 ms. Trace:
I20260811 18:57:16.148334 4169 rpcz_store.cc:276] 0811 18:57:14.521479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.522038 (+ 559us) service_pool.cc:227] Handling call
0811 18:57:15.974882 (+1452844us) ts_tablet_manager.cc:1145] Deleting tablet 807cee88c18340dbb23c51cad54ab56d
0811 18:57:15.974901 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:15.991220 (+ 16319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.085503 (+ 94283us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.145873 (+ 60370us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1452314}
I20260811 18:57:16.149242 4169 tablet_replica.cc:330] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.149734 4169 raft_consensus.cc:2229] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.150251 4169 raft_consensus.cc:2258] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.157544 4163 ts_tablet_manager.cc:1428] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.138s user 0.004s sys 0.001s
I20260811 18:57:16.160624 4739 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:16.166893 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 238706us
I20260811 18:57:16.168709 4169 ts_tablet_manager.cc:1905] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.182003 4797 raft_consensus.cc:482] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.182432 4797 raft_consensus.cc:504] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.184228 4163 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:16.189023 4797 leader_election.cc:288] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.163792 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 (DEFAULT_TABLE table=default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.175261 3718 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c8d542ae2489ba466ffadd518db12 (DEFAULT_TABLE table=default.Table_5409ed17441242119ff946163da07a21 [id=97d17499bd684f0abf98e0582fa06f5d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.161844 3678 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.269s (SENT)
I20260811 18:57:16.175555 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 (DEFAULT_TABLE table=default.Table_e746326e8bff404aa5b3bc1fac099489 [id=bebb3efbbc56416597abdf4a0b3952ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.197286 4739 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.174170 3705 tablet_service.cc:1467] Processing CreateTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e (DEFAULT_TABLE table=default.Table_994d2d4469b54597a9aa33c59dca9d84 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.201033 4739 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.203037 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce94d37efd2245bf8cc71fdd90bd9735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.212296 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_575c5aca059048d68252cca6583a1f67" } new_table_name: "default.Table_1b2c3c3b4a2948bbba5b4b556faf1015" modify_external_catalogs: true
I20260811 18:57:16.214608 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f97f37fa7c483cacaeed54f93e7a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.216109 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a847581412494e34b2c76e605ff11772 (DEFAULT_TABLE table=default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.219858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 (DEFAULT_TABLE table=default.Table_c37e3fb34b3f446d854006b05dbe5a94 [id=5fc5ade3f38c438d9b501127ac866b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.224730 4797 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.230674 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 (DEFAULT_TABLE table=default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.227612 4799 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.235260 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e544b4b1592c4d51a2ed0d6461edc75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.232570 4799 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.240844 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 7057b8a7bae249caaed304b5c66e6871 (DEFAULT_TABLE table=default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.242141 4797 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.243325 4797 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.245826 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a847581412494e34b2c76e605ff11772. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.251487 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44221.3us
#0 0x967ea3 in base::subtle::NoBarrier_Load(int const volatile*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/atomicops-internals-x86.h:200:10
I20260811 18:57:16.255136 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7057b8a7bae249caaed304b5c66e6871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.256587 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cccebfc54484a9b9abffcce4e459fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.258010 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cce97bfdb1d41e685e091c8c5cc6946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.261217 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5befe5c7d84b401e8323659cf14a3a17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.262097 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5befe5c7d84b401e8323659cf14a3a17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.268129 3745 raft_consensus.cc:2454] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:16.269307 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f62a01a774b4295a6dc3352d22d1ba4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.270452 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f62a01a774b4295a6dc3352d22d1ba4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.276307 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c8d542ae2489ba466ffadd518db12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.279762 4799 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.296530 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4814a3562ef14fb0bec79be8c3cff4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.300856 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:16.304724 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a81334e7da3741a9b24f8978312181b9
I20260811 18:57:16.305757 3891 raft_consensus.cc:2454] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:16.308293 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:16.308539 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d18dc31cde54577b8bcc329191b74ef" } new_table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" modify_external_catalogs: true
I20260811 18:57:16.309333 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_005d29915a3b4449acfc665e917be061" } new_table_name: "default.Table_94cadddacfc545b6af75e68ed02f34c1" modify_external_catalogs: true
W20260811 18:57:16.311510 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45fde25bcb40493aafb083e161aa6932 consensus state has no leader
I20260811 18:57:16.312011 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e51a08e6dbf04e3fa58689ee0cd45db3
I20260811 18:57:16.312876 3550 catalog_manager.cc:6565] ReplaceTablet: tablet e51a08e6dbf04e3fa58689ee0cd45db3 deleted and replaced by tablet 02799b864ad242f08fd89798c0fef4bf
I20260811 18:57:16.316053 4163 tablet_bootstrap.cc:654] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.316495 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.317281 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2058b5fb0d08491a9f2fa354276fc21b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.318223 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e9433d18ac7b46958942fc1f4b458354"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.318775 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
W20260811 18:57:16.318918 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9433d18ac7b46958942fc1f4b458354 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.319288 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7fc31b7c8d824f879807f105af51df48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.319895 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7fc31b7c8d824f879807f105af51df48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:16.321349 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:16.322000 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.322497 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5054093f72194ee8a604b8aa7fb6cf67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.323422 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:16.327219 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 799ba9a044e145ec89d0025c03f391ea deleted and replaced by tablet e1af33fd98c6486f975368ec90493db7
I20260811 18:57:16.332072 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef1abae384a9404089b0f6600e935a5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.332970 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef1abae384a9404089b0f6600e935a5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.335225 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dd7b9197d5d74b599220994b06e7830a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.335991 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd7b9197d5d74b599220994b06e7830a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.338006 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28b2e1b607a34ab2badc82e3c333a386"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.339003 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28b2e1b607a34ab2badc82e3c333a386 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:16.362550 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 266964us
I20260811 18:57:16.363533 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 807cee88c18340dbb23c51cad54ab56d (table default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]) successfully deleted
W20260811 18:57:16.365342 3824 leader_election.cc:341] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:16.373780 4803 log.cc:924] Time spent T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
I20260811 18:57:16.376866 3724 tablet_service.cc:1467] Processing CreateTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d (DEFAULT_TABLE table=default.Table_b07958bac6344da1bbf3d7d35f1ac551 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.388599 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b404e99e9414c67b3cd5aff12203a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.360265 3676 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:16.397364 3824 leader_election.cc:341] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:16.397998 3824 leader_election.cc:341] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.398448 3824 leader_election.cc:302] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:16.398816 4799 raft_consensus.cc:2781] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.399084 4799 raft_consensus.cc:686] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:16.399509 4799 consensus_queue.cc:237] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.400733 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf285cdf5235451bb1531cb294017101" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.401392 4739 raft_consensus.cc:2781] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.401582 4739 raft_consensus.cc:482] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.401762 4739 raft_consensus.cc:3037] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.410136 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ceca57cb245f4a3a6cb4b36f1306d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.411087 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.414824 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 117124us
I20260811 18:57:16.418111 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 02799b864ad242f08fd89798c0fef4bf
I20260811 18:57:16.422991 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f3fb616fae4249b4a043ddb879c7db57" } modify_external_catalogs: true
I20260811 18:57:16.423950 3861 tablet_service.cc:1467] Processing CreateTablet for tablet bb56d23ccabc4b02961249312257d9c4 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.425530 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45c439603b1140e998b5d9199026da49" } modify_external_catalogs: true
I20260811 18:57:16.425869 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45c439603b1140e998b5d9199026da49" } modify_external_catalogs: true
I20260811 18:57:16.424608 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f3fb616fae4249b4a043ddb879c7db57" } modify_external_catalogs: true
I20260811 18:57:16.428740 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet a81334e7da3741a9b24f8978312181b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.429047 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.428987 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0190cbd859274fcfa09c6fab899c4843 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:16.429910 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 473040a90e6941c38beb697316d2a27b has no consensus state
I20260811 18:57:16.430106 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d779d44f532f404cad704a689e72f1b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.431141 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b2718cd4a98742dfa8014a8a38c69c45 consensus state has no leader
W20260811 18:57:16.431806 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d779d44f532f404cad704a689e72f1b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.434716 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.437373 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (replaced by tablet 02799b864ad242f08fd89798c0fef4bf) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.438186 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.439306 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb56d23ccabc4b02961249312257d9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.414671 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5d495354c0994fb4a74c376078e41a26 (DEFAULT_TABLE table=default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.451838 3824 leader_election.cc:341] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.451839 4797 consensus_queue.cc:1035] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:16.458257 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.458932 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1efffade0f486d8d627e56e0be6ff6 with delete_type TABLET_DATA_DELETED (replaced by tablet 27cda53b7ca54346a7a22bf26f5db990) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:16.468437 3676 consensus_peers.cc:487] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807cee88c18340dbb23c51cad54ab56d. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:16.481539 4167 ts_tablet_manager.cc:1918] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.510494 4797 raft_consensus.cc:482] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.511442 4797 raft_consensus.cc:504] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.512590 4797 leader_election.cc:288] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:16.512748 4169 ts_tablet_manager.cc:1918] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.513032 4169 log.cc:1196] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/508f0b39cd654aa991a2928a4b067762
I20260811 18:57:16.513856 4169 ts_tablet_manager.cc:1939] T 508f0b39cd654aa991a2928a4b067762 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.514420 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 257) took 1992 ms. Trace:
I20260811 18:57:16.514581 4169 rpcz_store.cc:276] 0811 18:57:14.521734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.522645 (+ 911us) service_pool.cc:227] Handling call
0811 18:57:16.148838 (+1626193us) ts_tablet_manager.cc:1145] Deleting tablet 508f0b39cd654aa991a2928a4b067762
0811 18:57:16.148858 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.396710 (+247852us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.444751 (+ 48041us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.514281 (+ 69530us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1625678}
I20260811 18:57:16.515121 4169 tablet_replica.cc:330] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.515453 4169 raft_consensus.cc:2229] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.515688 4169 raft_consensus.cc:2258] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.516577 4169 ts_tablet_manager.cc:1905] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.521870 4163 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.522228 4163 ts_tablet_manager.cc:1397] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.364s user 0.005s sys 0.007s
I20260811 18:57:16.524169 4163 raft_consensus.cc:348] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.525245 4163 raft_consensus.cc:374] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.526067 4163 raft_consensus.cc:729] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.526538 4163 consensus_queue.cc:260] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.552806 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet c26851245cab4a2d80e94d4fe278fd50 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.555778 4163 ts_tablet_manager.cc:1428] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:16.485510 4739 raft_consensus.cc:504] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.485831 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d495354c0994fb4a74c376078e41a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.492626 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.492839 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet a81334e7da3741a9b24f8978312181b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.495676 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.506567 3716 tablet_service.cc:1467] Processing CreateTablet for tablet faf0594d55844ae49c99247ecd8b810f (DEFAULT_TABLE table=default.Table_6d87a15f366a4e118c84e294c3b9926e [id=5a23d06641cb447fb634032bd768498b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.509341 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:16.520027 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53202.5us
I20260811 18:57:16.527338 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (replaced by tablet 02799b864ad242f08fd89798c0fef4bf) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.527565 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.530315 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bb56d23ccabc4b02961249312257d9c4 (DEFAULT_TABLE table=default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.538254 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.573338 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" } modify_external_catalogs: true
I20260811 18:57:16.573722 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_133a44e94783442e8c27d4730310f996" } modify_external_catalogs: true
I20260811 18:57:16.573808 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1558f156b5eb46a1858ca7673e4a9747" } modify_external_catalogs: true
I20260811 18:57:16.574119 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_133a44e94783442e8c27d4730310f996" } modify_external_catalogs: true
I20260811 18:57:16.574906 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.575557 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b717724905d74a6a865cd6eb35c3a4d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.577358 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 508f0b39cd654aa991a2928a4b067762 (table default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]) successfully deleted
I20260811 18:57:16.573390 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9292c8eb06fb4a0d8e33b6796398b740" } new_table_name: "default.Table_647ae82e63464976abb7f26c9526a349" modify_external_catalogs: true
I20260811 18:57:16.586555 3676 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:16.601809 3676 consensus_peers.cc:487] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:16.613153 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.566743 4163 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:16.590197 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8426ccbe530446fcbaa162a03cbb1892" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
W20260811 18:57:16.577457 3824 leader_election.cc:341] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.566256 4169 ts_tablet_manager.cc:1918] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.622728 4729 consensus_queue.cc:1035] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260811 18:57:16.624408 4739 leader_election.cc:288] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.624818 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb56d23ccabc4b02961249312257d9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.628427 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf0594d55844ae49c99247ecd8b810f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.632085 3890 raft_consensus.cc:3037] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:16.635375 3824 leader_election.cc:341] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.641379 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f8eb041f3d704762868a048faec0e0fc
I20260811 18:57:16.643877 4169 log.cc:1196] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ed3346bd12d04c45a511308c427856e5
I20260811 18:57:16.644385 4163 tablet_bootstrap.cc:654] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.644815 3890 raft_consensus.cc:2454] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:16.647977 4576 raft_consensus.cc:2781] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.647969 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b65c4c4ef6ab40dcb8b22f25b9ea7ae8" } new_table_name: "default.Table_12d05bfe0d2348f0a14ef6b8ecf8414a" modify_external_catalogs: true
I20260811 18:57:16.648172 4576 raft_consensus.cc:482] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.649474 4167 log.cc:1196] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2852f9e5f67c4025836018af553388a5
I20260811 18:57:16.650022 4576 raft_consensus.cc:3037] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.650259 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd7b9197d5d74b599220994b06e7830a" } new_table_name: "default.Table_a15aed3f339749a7ab05c1774bc21112" modify_external_catalogs: true
I20260811 18:57:16.652334 4169 ts_tablet_manager.cc:1939] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.652145 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9f42592ab70540e59f42f7e97293a058"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:16.652841 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 258) took 2130 ms. Trace:
I20260811 18:57:16.653007 4169 rpcz_store.cc:276] 0811 18:57:14.521960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.523214 (+ 1254us) service_pool.cc:227] Handling call
0811 18:57:16.515035 (+1991821us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:16.515051 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.559161 (+ 44110us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.566111 (+ 6950us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.652774 (+ 86663us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1991378}
W20260811 18:57:16.653137 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f42592ab70540e59f42f7e97293a058 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.653600 4169 tablet_replica.cc:330] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.653988 4169 raft_consensus.cc:2229] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.654678 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a30bb384136f4a169a5751655437d5a3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:16.656500 4162 tablet_bootstrap.cc:492] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:16.656805 4162 ts_tablet_manager.cc:1397] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.877s user 0.012s sys 0.020s
W20260811 18:57:16.657828 3824 leader_election.cc:341] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:16.658957 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ed3346bd12d04c45a511308c427856e5 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
I20260811 18:57:16.659417 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:16.660856 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1e1374a80d15444cb0faff339275aa8d consensus state has no leader
I20260811 18:57:16.661491 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cce97bfdb1d41e685e091c8c5cc6946 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.667182 4162 raft_consensus.cc:348] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.667932 4162 raft_consensus.cc:374] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.670575 4576 raft_consensus.cc:504] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.675948 4576 leader_election.cc:288] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:16.676529 4162 raft_consensus.cc:729] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.677043 4162 consensus_queue.cc:260] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.680051 4167 ts_tablet_manager.cc:1939] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:16.684051 4169 raft_consensus.cc:2258] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.689421 4169 ts_tablet_manager.cc:1905] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.690826 4162 ts_tablet_manager.cc:1428] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260811 18:57:16.692134 4163 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.692625 4163 ts_tablet_manager.cc:1397] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.136s user 0.008s sys 0.004s
W20260811 18:57:16.706261 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_655dd0bc17cc45d3a344bb5c02ff7f80 [id=f51f38f3582f44db99007a265b665fd4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:16.708043 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.710078 4162 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:16.713802 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 97780.7us
I20260811 18:57:16.694177 4163 raft_consensus.cc:348] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.714766 4163 raft_consensus.cc:374] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.714962 4163 raft_consensus.cc:729] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.715477 4163 consensus_queue.cc:260] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.716989 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 228) took 2991 ms. Trace:
I20260811 18:57:16.717715 4163 ts_tablet_manager.cc:1428] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.004s sys 0.000s
I20260811 18:57:16.718243 4163 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:16.730176 3676 consensus_peers.cc:487] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 508f0b39cd654aa991a2928a4b067762. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:16.738080 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a9efd2f19d74a68a8536fd057377738 consensus state has no leader
I20260811 18:57:16.738751 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" } modify_external_catalogs: true
I20260811 18:57:16.739086 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3bd2f4a0463f483aa76c5d66c9f83d53" } modify_external_catalogs: true
I20260811 18:57:16.743220 4163 tablet_bootstrap.cc:654] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.745519 4167 rpcz_store.cc:276] 0811 18:57:13.689015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.689089 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:15.857307 (+2168218us) ts_tablet_manager.cc:1145] Deleting tablet 2852f9e5f67c4025836018af553388a5
0811 18:57:15.857324 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.434471 (+577147us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.481370 (+ 46899us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.680880 (+199510us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2167725}
I20260811 18:57:16.749091 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9f5c2bff2c124718a66c2f87c60ccbd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:16.749502 4169 ts_tablet_manager.cc:1918] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:16.749750 4169 log.cc:1196] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/151df87806694f46be5013592c68c01a
W20260811 18:57:16.749938 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f5c2bff2c124718a66c2f87c60ccbd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.751722 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2852f9e5f67c4025836018af553388a5 (table default.Table_8af423b4350249d6bae77e7e3c7d9fa4 [id=d9be1bf7e9d448b7bee83be50481af5e]) successfully deleted
W20260811 18:57:16.752960 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffa8a4a1c0cb4834be501bdd7d468472 has no consensus state
I20260811 18:57:16.753724 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b654b59f369e429097b918236d8ddd08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.754809 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b654b59f369e429097b918236d8ddd08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.761240 4162 tablet_bootstrap.cc:654] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.764739 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]). Expected version 2 got 1
I20260811 18:57:16.765727 4169 ts_tablet_manager.cc:1939] T 151df87806694f46be5013592c68c01a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:16.765352 3554 catalog_manager.cc:5495] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:16.766125 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:16.766912 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 262) took 2006 ms. Trace:
I20260811 18:57:16.767135 4169 rpcz_store.cc:276] 0811 18:57:14.759982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.760065 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:16.653383 (+1893318us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:16.653396 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.703050 (+ 49654us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.749303 (+ 46253us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:16.766861 (+ 17558us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1892920}
I20260811 18:57:16.768282 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 151df87806694f46be5013592c68c01a (table default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]) successfully deleted
I20260811 18:57:16.771859 4163 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:16.772176 4163 ts_tablet_manager.cc:1397] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.054s user 0.006s sys 0.000s
I20260811 18:57:16.773095 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 2c29af3615e74722a809d3e5ede9b9a1 (DEFAULT_TABLE table=default.Table_9c6377231db8495796c02e71f7f71991 [id=dd46ddba2d9843a981969eb5db1d4cea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.773329 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]). Expected version 1 got 0
I20260811 18:57:16.773849 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c29af3615e74722a809d3e5ede9b9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.774096 4163 raft_consensus.cc:348] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.774513 4163 raft_consensus.cc:374] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.774778 4163 raft_consensus.cc:729] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.775252 4163 consensus_queue.cc:260] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:16.777853 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (DEFAULT_TABLE table=default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.778596 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4de110bd3348f0b0dae21c9f2b6a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.783131 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 452ee450778e4d33adfa7e2745b5d970 (DEFAULT_TABLE table=default.Table_d3e16ca699564a61a49c4cb989ad865f [id=48f45fcc06e0418185841260a06cb98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.783846 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452ee450778e4d33adfa7e2745b5d970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.786734 4169 tablet_replica.cc:330] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:16.787227 4169 raft_consensus.cc:2229] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:16.788592 3676 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:16.789373 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 0bfb54175c834517b7af291c942333cd (DEFAULT_TABLE table=default.Table_6bf41429faeb4785ad747d207e4641f8 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.790208 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bfb54175c834517b7af291c942333cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.793938 4729 raft_consensus.cc:2781] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.794258 4729 raft_consensus.cc:686] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:16.797688 4729 consensus_queue.cc:237] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:16.799795 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 58323.2us
I20260811 18:57:16.801889 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 230) took 2913 ms. Trace:
I20260811 18:57:16.801913 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e23d79bc476d496eaa713465bcfb1abf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.802780 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e23d79bc476d496eaa713465bcfb1abf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.803128 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (DEFAULT_TABLE table=default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.804598 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 0bfb54175c834517b7af291c942333cd (DEFAULT_TABLE table=default.Table_6bf41429faeb4785ad747d207e4641f8 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.807047 3722 tablet_service.cc:1467] Processing CreateTablet for tablet eb82243777a9494b96959ced0c24de5a (DEFAULT_TABLE table=default.Table_fd120e0ab3b34418960cefc5eb459cdb [id=3dc6eeef579941eaaa83be543d1c01d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.808159 3860 tablet_service.cc:1467] Processing CreateTablet for tablet d73a98bd35bd444195fd32fe6c67a04f (DEFAULT_TABLE table=default.Table_6a122fe2820f4d23b12edb970a94e958 [id=356820f47d4942d0a4cce16978925421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.808773 3713 tablet_service.cc:1467] Processing CreateTablet for tablet d73a98bd35bd444195fd32fe6c67a04f (DEFAULT_TABLE table=default.Table_6a122fe2820f4d23b12edb970a94e958 [id=356820f47d4942d0a4cce16978925421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.809031 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a98bd35bd444195fd32fe6c67a04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.810204 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_241ec7c366e84652a6d9c0123d2149ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.811141 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_241ec7c366e84652a6d9c0123d2149ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.811115 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 (DEFAULT_TABLE table=default.Table_b2d444511ad44775b701b9ed7527e251 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.811794 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c365f79d4a4201ac323f16bddf3e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.812639 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa (DEFAULT_TABLE table=default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.813078 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa (DEFAULT_TABLE table=default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.813711 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 (DEFAULT_TABLE table=default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.814774 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 (DEFAULT_TABLE table=default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.812186 3863 tablet_service.cc:1467] Processing CreateTablet for tablet eb82243777a9494b96959ced0c24de5a (DEFAULT_TABLE table=default.Table_fd120e0ab3b34418960cefc5eb459cdb [id=3dc6eeef579941eaaa83be543d1c01d1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.816428 4812 log.cc:924] Time spent T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260811 18:57:16.822003 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 452ee450778e4d33adfa7e2745b5d970 (DEFAULT_TABLE table=default.Table_d3e16ca699564a61a49c4cb989ad865f [id=48f45fcc06e0418185841260a06cb98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.822288 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8c0444ec2242299482736a73551f80" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:16.822674 3888 raft_consensus.cc:3037] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.822917 4167 rpcz_store.cc:276] 0811 18:57:13.852271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.852339 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:16.746736 (+2894397us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:16.746749 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.765231 (+ 18482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2893716}
W20260811 18:57:16.824663 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 867259e37fe44c33a35823ec2d896754 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 867259e37fe44c33a35823ec2d896754 already in progress: creating tablet
I20260811 18:57:16.824841 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4de110bd3348f0b0dae21c9f2b6a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.829545 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb82243777a9494b96959ced0c24de5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.831524 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19243ab3d6744aaa87a133f1462cd2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.832860 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 19243ab3d6744aaa87a133f1462cd2fa
I20260811 18:57:16.833977 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27cda53b7ca54346a7a22bf26f5db990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.833974 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_704e71bf00da49609103fb3fde6afb2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.834748 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_704e71bf00da49609103fb3fde6afb2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.835880 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a98bd35bd444195fd32fe6c67a04f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.838873 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27cda53b7ca54346a7a22bf26f5db990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.838850 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 (DEFAULT_TABLE table=default.Table_b2d444511ad44775b701b9ed7527e251 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.839932 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.843851 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452ee450778e4d33adfa7e2745b5d970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.850556 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e18242f762d8407db0e2f8716890feb4 (DEFAULT_TABLE table=default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.850239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 229) took 2989 ms. Trace:
I20260811 18:57:16.851974 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:16.852300 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c365f79d4a4201ac323f16bddf3e62. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.853190 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.620s (SENT) [suppressed 9 similar messages]
I20260811 18:57:16.865577 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19243ab3d6744aaa87a133f1462cd2fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.867250 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 473040a90e6941c38beb697316d2a27b (DEFAULT_TABLE table=default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.865331 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c0b355a0d3984012be4e9406998fd4a0 (DEFAULT_TABLE table=default.Table_7e8859199c3440ebb2b9664b65e0fb71 [id=aa5ff7c0d063424992f9dc22c89aa620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.868180 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 2599dbd65a814601928373ab9910e3ca (DEFAULT_TABLE table=default.Table_b35ea225e3114cdc9ed09c78af2f272d [id=3a1d54898d9b4f918ab8ea53a0f1bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.876839 4169 pending_rounds.cc:62] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:16.882097 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18242f762d8407db0e2f8716890feb4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:16.895574 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 6026ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:16.900357 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2599dbd65a814601928373ab9910e3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.914697 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b355a0d3984012be4e9406998fd4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.877903 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 2c29af3615e74722a809d3e5ede9b9a1 (DEFAULT_TABLE table=default.Table_9c6377231db8495796c02e71f7f71991 [id=dd46ddba2d9843a981969eb5db1d4cea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:16.915635 3361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3370: Timed out: CheckLeaks RPC to 127.3.72.126:34821 timed out after 1.000s (SENT)
I20260811 18:57:16.915947 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3370
I20260811 18:57:16.869776 4167 rpcz_store.cc:276] 0811 18:57:13.851791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.852598 (+ 807us) service_pool.cc:227] Handling call
0811 18:57:16.835322 (+2982724us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:16.835339 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:16.841095 (+ 5756us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2982157}
I20260811 18:57:16.917130 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473040a90e6941c38beb697316d2a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.918853 3723 tablet_service.cc:1467] Processing CreateTablet for tablet e18242f762d8407db0e2f8716890feb4 (DEFAULT_TABLE table=default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.920295 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:16.920491 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb82243777a9494b96959ced0c24de5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.921787 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:16.922808 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:16.923664 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 8cce97bfdb1d41e685e091c8c5cc6946 deleted and replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa
I20260811 18:57:16.926376 4169 pending_rounds.cc:68] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106077646848 op_type: NO_OP noop_request { }
I20260811 18:57:16.926738 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c0b355a0d3984012be4e9406998fd4a0 (DEFAULT_TABLE table=default.Table_7e8859199c3440ebb2b9664b65e0fb71 [id=aa5ff7c0d063424992f9dc22c89aa620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.927363 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 2599dbd65a814601928373ab9910e3ca (DEFAULT_TABLE table=default.Table_b35ea225e3114cdc9ed09c78af2f272d [id=3a1d54898d9b4f918ab8ea53a0f1bfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.929191 4167 tablet_replica.cc:330] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:16.929527 4167 raft_consensus.cc:2229] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.929883 4167 raft_consensus.cc:2258] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.930706 4167 ts_tablet_manager.cc:1905] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.935238 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 473040a90e6941c38beb697316d2a27b (DEFAULT_TABLE table=default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.936170 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bfb54175c834517b7af291c942333cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.936776 4169 raft_consensus.cc:2866] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260811 18:57:16.939486 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a902ad70a25546be926ff8f7bf2ed64f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a902ad70a25546be926ff8f7bf2ed64f already in progress: creating tablet
I20260811 18:57:16.942937 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.943414 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.943672 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.951963 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_021124838f7841c49f01816fb948d73c" } new_table_name: "default.Table_27d36ce508354815add475acc3f872eb" modify_external_catalogs: true
I20260811 18:57:16.953199 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:16.953888 3549 catalog_manager.cc:6565] ReplaceTablet: tablet c26851245cab4a2d80e94d4fe278fd50 deleted and replaced by tablet 4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:16.961086 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:16.962442 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c6e0d1eeb63493bbea4cd78dd085a3c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.965147 4169 raft_consensus.cc:2258] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.965246 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 968fa8b8912b40a8bdba77db1d39c5c0 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.969695 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:16.970023 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2599dbd65a814601928373ab9910e3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.974848 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 151df87806694f46be5013592c68c01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:16.977389 4818 log.cc:924] Time spent T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260811 18:57:16.977720 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 987b9ac6859444e3a86c56af786cc84e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:16.982048 4169 ts_tablet_manager.cc:1905] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:16.982422 3678 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.013s (SENT)
I20260811 18:57:16.982672 3678 leader_election.cc:302] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:16.986173 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 50458.7us
I20260811 18:57:16.987360 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c29af3615e74722a809d3e5ede9b9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.988545 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18242f762d8407db0e2f8716890feb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.989219 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b355a0d3984012be4e9406998fd4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.990090 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 473040a90e6941c38beb697316d2a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.991477 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3cc27811ad29451daaebbf31fb3ad198"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:16.992157 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cc27811ad29451daaebbf31fb3ad198 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.992540 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f" } modify_external_catalogs: true
I20260811 18:57:16.993521 3888 raft_consensus.cc:2454] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:16.993988 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f" } modify_external_catalogs: true
I20260811 18:57:16.998663 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5befe5c7d84b401e8323659cf14a3a17" } new_table_name: "default.Table_c86ddee4faa543c79b02e248ac801fb3" modify_external_catalogs: true
I20260811 18:57:17.000433 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.001080 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d490ceda4c745668e9deb77c4a07fc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.010648 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:17.017896 4163 ts_tablet_manager.cc:1428] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.246s user 0.005s sys 0.000s
I20260811 18:57:17.018985 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b227bc7305f4b2dae28f04e7001553b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.022143 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6533d74b53c54546bdd082863b38257c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.022460 4163 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:17.022951 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6533d74b53c54546bdd082863b38257c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.024914 4163 tablet_bootstrap.cc:654] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.024923 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5f4d9f42e99c454e88964d343262e534"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.026041 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f4d9f42e99c454e88964d343262e534 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.029412 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68a7051bc539468aafba30fbaf0669d4 consensus state has no leader
I20260811 18:57:17.030081 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_457be7b012ba40deab01ec6480b82ede" } new_table_name: "default.Table_3c807c78cf1249abbf878e819573553d" modify_external_catalogs: true
I20260811 18:57:17.046979 4815 raft_consensus.cc:2726] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:17.065057 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:17.050594 3824 leader_election.cc:334] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.017s (SENT)
I20260811 18:57:17.068308 3824 leader_election.cc:302] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:17.068620 3824 raft_consensus.cc:2670] T 2852f9e5f67c4025836018af553388a5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:17.083979 4169 ts_tablet_manager.cc:1918] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.086759 4162 tablet_bootstrap.cc:492] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.090675 4162 ts_tablet_manager.cc:1397] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.393s user 0.010s sys 0.002s
I20260811 18:57:17.091281 4169 log.cc:1196] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/33979de5b48c4674a46c8da3b7151753
I20260811 18:57:17.092073 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.078861 4163 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.092720 4163 ts_tablet_manager.cc:1397] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.005s sys 0.000s
I20260811 18:57:17.094066 4163 raft_consensus.cc:348] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.094867 4163 raft_consensus.cc:374] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.095870 4163 raft_consensus.cc:729] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.096522 4169 ts_tablet_manager.cc:1939] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.096622 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b227bc7305f4b2dae28f04e7001553b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.097187 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.097419 4163 consensus_queue.cc:260] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.105831 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 263) took 2312 ms. Trace:
I20260811 18:57:17.106745 4811 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.106958 4811 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.107745 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet ed3346bd12d04c45a511308c427856e5 with delete_type TABLET_DATA_DELETED (replaced by tablet b2718cd4a98742dfa8014a8a38c69c45) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.107960 4811 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.105330 4163 ts_tablet_manager.cc:1428] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260811 18:57:17.109196 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.110363 3676 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.109295 4163 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.106417 4169 rpcz_store.cc:276] 0811 18:57:14.793877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.793950 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:16.786633 (+1992683us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:16.786648 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.063263 (+276615us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.083771 (+ 20508us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.103008 (+ 19237us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14349,"tablet-delete.queue_time_us":1992042}
I20260811 18:57:17.112529 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 33979de5b48c4674a46c8da3b7151753 (table default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]) successfully deleted
I20260811 18:57:17.112883 4169 tablet_replica.cc:330] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.113886 4169 raft_consensus.cc:2229] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.115831 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8c0444ec2242299482736a73551f80 with delete_type TABLET_DATA_DELETED (replaced by tablet e5fd9fce095e4a769dbb38c7903359cc) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.116180 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 33979de5b48c4674a46c8da3b7151753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.116426 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.117376 3824 leader_election.cc:341] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.118753 4169 raft_consensus.cc:2258] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.122947 4162 raft_consensus.cc:348] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.125545 4815 raft_consensus.cc:2781] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.126389 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce8db230c7e41b6a4792273a0f07e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.127413 4162 raft_consensus.cc:374] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.127590 4162 raft_consensus.cc:729] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.130231 4162 consensus_queue.cc:260] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.131098 4162 ts_tablet_manager.cc:1428] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
I20260811 18:57:17.131872 4815 raft_consensus.cc:686] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.133136 4162 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.132321 4815 consensus_queue.cc:237] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.145714 4829 tablet.cc:1722] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:17.148610 4169 ts_tablet_manager.cc:1905] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.155660 4163 tablet_bootstrap.cc:654] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.156774 4167 ts_tablet_manager.cc:1918] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.176676 4167 log.cc:1196] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8af3e74889b64c408c9ed9c54916f890
I20260811 18:57:17.177388 4162 tablet_bootstrap.cc:654] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.177742 4163 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.178392 4163 ts_tablet_manager.cc:1397] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.000s
I20260811 18:57:17.181421 4163 raft_consensus.cc:348] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.184401 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 has no consensus state
I20260811 18:57:17.186439 4163 raft_consensus.cc:374] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.187003 4163 raft_consensus.cc:729] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.187604 4163 consensus_queue.cc:260] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.188642 4167 ts_tablet_manager.cc:1939] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.189534 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 233) took 3332 ms. Trace:
I20260811 18:57:17.191222 4167 rpcz_store.cc:276] 0811 18:57:13.857415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.860602 (+ 3187us) service_pool.cc:227] Handling call
0811 18:57:16.916222 (+3055620us) ts_tablet_manager.cc:1145] Deleting tablet 8af3e74889b64c408c9ed9c54916f890
0811 18:57:16.916234 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.082066 (+165832us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.139745 (+ 57679us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.189486 (+ 49741us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3055186}
I20260811 18:57:17.191849 4167 tablet_replica.cc:330] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:17.192533 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8af3e74889b64c408c9ed9c54916f890 (table default.Table_deebcc7412e6466986509a6858c4bc70 [id=0daf74fc92b7432a9e5f94cb4f1ebea1]) successfully deleted
I20260811 18:57:17.193099 4167 raft_consensus.cc:2229] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:17.193574 4167 pending_rounds.cc:62] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:17.195363 4163 ts_tablet_manager.cc:1428] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260811 18:57:17.199059 4163 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.199108 4167 pending_rounds.cc:68] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400104139776000 op_type: NO_OP noop_request { }
I20260811 18:57:17.199554 4167 raft_consensus.cc:2866] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:17.203006 4167 raft_consensus.cc:2258] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.203881 4167 ts_tablet_manager.cc:1905] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.214500 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f5c2bff2c124718a66c2f87c60ccbd3" } new_table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" modify_external_catalogs: true
I20260811 18:57:17.216441 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.216744 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_66613580530e4ae8989a795dd5d5ef69" } new_table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" modify_external_catalogs: true
W20260811 18:57:17.217166 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.219020 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55c1a7d44717415ca76a2d0b7e94394e consensus state has no leader
I20260811 18:57:17.219591 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4927a23edf074354bdd4cf94e4b74eff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.220117 4163 tablet_bootstrap.cc:654] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:17.220292 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4927a23edf074354bdd4cf94e4b74eff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.230631 4169 ts_tablet_manager.cc:1918] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.231786 4169 log.cc:1196] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9f8c0444ec2242299482736a73551f80
W20260811 18:57:17.232758 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 42405aec21e3405486b6c658438b5fb2 has no consensus state
I20260811 18:57:17.233359 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.234020 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a80fc1f336804bea9495ec0bce4bd47a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.234431 4163 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.234593 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_2bb9c598e2de422b86a8304bbece96db [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 1 got 0
I20260811 18:57:17.234810 4163 ts_tablet_manager.cc:1397] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
I20260811 18:57:17.235877 4163 raft_consensus.cc:348] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.236315 4163 raft_consensus.cc:374] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.236398 4163 raft_consensus.cc:729] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.236315 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_46eb50be61c143f2930a20efd2cd56b8" } modify_external_catalogs: true
I20260811 18:57:17.236560 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_46eb50be61c143f2930a20efd2cd56b8" } modify_external_catalogs: true
I20260811 18:57:17.240893 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:17.241160 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
W20260811 18:57:17.241952 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a443fe209556489abb47f4faa5d696e5 consensus state has no leader
I20260811 18:57:17.242385 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a108c0f554824b94acc46ae9330d1115" } modify_external_catalogs: true
I20260811 18:57:17.242589 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a108c0f554824b94acc46ae9330d1115" } modify_external_catalogs: true
I20260811 18:57:17.242707 4163 consensus_queue.cc:260] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.236688 4169 ts_tablet_manager.cc:1939] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.238497 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.238705 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.243922 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 265) took 2410 ms. Trace:
I20260811 18:57:17.244237 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0d00075c1e094c1cb6a03b6760496fb5 [id=64fc5489749347609383603760522f27]). Expected version 4 got 0
I20260811 18:57:17.244118 4169 rpcz_store.cc:276] 0811 18:57:14.833448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833521 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:17.112572 (+2279051us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:17.112585 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.153756 (+ 41171us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.230436 (+ 76680us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.243853 (+ 13417us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2278745}
I20260811 18:57:17.244664 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9f8c0444ec2242299482736a73551f80 (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:17.244900 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 264) took 2411 ms. Trace:
I20260811 18:57:17.245033 4169 rpcz_store.cc:276] 0811 18:57:14.833276 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833733 (+ 457us) service_pool.cc:227] Handling call
0811 18:57:17.244670 (+2410937us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:17.244686 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.244829 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2410655}
I20260811 18:57:17.245239 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:17.246861 3864 tablet_service.cc:1467] Processing CreateTablet for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (DEFAULT_TABLE table=default.Table_b654b59f369e429097b918236d8ddd08 [id=1323bd99594a4382ba121cc11e79de31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.247479 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.251286 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 42405aec21e3405486b6c658438b5fb2 (DEFAULT_TABLE table=default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.251806 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42405aec21e3405486b6c658438b5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.254792 3862 tablet_service.cc:1467] Processing CreateTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 (DEFAULT_TABLE table=default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.255741 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8f5b87062349c4b19ec99fa0175f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.258652 3861 tablet_service.cc:1467] Processing CreateTablet for tablet f75dc515d7f3473b8e43f20255a70e3e (DEFAULT_TABLE table=default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.261547 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f75dc515d7f3473b8e43f20255a70e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.264439 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc07c28ccfd48ee9c33e49442c6047b (DEFAULT_TABLE table=default.Table_d779d44f532f404cad704a689e72f1b3 [id=cabc6cc20754488eb152c0d4f686dce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.264973 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc07c28ccfd48ee9c33e49442c6047b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.268030 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b591f65f304846aab2374c6ada0dd0ec" } modify_external_catalogs: true
I20260811 18:57:17.268342 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b591f65f304846aab2374c6ada0dd0ec" } modify_external_catalogs: true
I20260811 18:57:17.270187 3716 tablet_service.cc:1467] Processing CreateTablet for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (DEFAULT_TABLE table=default.Table_b654b59f369e429097b918236d8ddd08 [id=1323bd99594a4382ba121cc11e79de31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.270799 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.272143 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef613c8dc0524f318790523ba7652317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.272945 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef613c8dc0524f318790523ba7652317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.277019 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 42405aec21e3405486b6c658438b5fb2 (DEFAULT_TABLE table=default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.277539 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42405aec21e3405486b6c658438b5fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.280438 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d37e81d376e14907b13746cca831a96b (DEFAULT_TABLE table=default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.281710 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37e81d376e14907b13746cca831a96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.282807 4162 tablet_bootstrap.cc:492] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.283082 4162 ts_tablet_manager.cc:1397] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.152s user 0.004s sys 0.003s
I20260811 18:57:17.284245 4162 raft_consensus.cc:348] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.286705 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a811d4488d0e4541808a5e1aa44d663b (DEFAULT_TABLE table=default.Table_7fc31b7c8d824f879807f105af51df48 [id=a71b71af19a44182aea7e742062558ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.288004 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a811d4488d0e4541808a5e1aa44d663b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.291543 3871 tablet_service.cc:1467] Processing CreateTablet for tablet d73a43558dbb48889ab205108f4ce6be (DEFAULT_TABLE table=default.Table_e9433d18ac7b46958942fc1f4b458354 [id=d818dd73be204b458c789ff1d10f7b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.292455 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a43558dbb48889ab205108f4ce6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.298440 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (DEFAULT_TABLE table=default.Table_c86ddee4faa543c79b02e248ac801fb3 [id=7a869d9623bc4e07938b2d26688723a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.298882 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.300468 3860 tablet_service.cc:1467] Processing CreateTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.300882 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa8a4a1c0cb4834be501bdd7d468472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.309458 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.310956 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8304ac7709ca4392baec4cfa78ef5b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.285526 3710 tablet_service.cc:1467] Processing CreateTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 (DEFAULT_TABLE table=default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.314837 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8f5b87062349c4b19ec99fa0175f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.285928 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f75dc515d7f3473b8e43f20255a70e3e (DEFAULT_TABLE table=default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.318887 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f75dc515d7f3473b8e43f20255a70e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.288858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet d37e81d376e14907b13746cca831a96b (DEFAULT_TABLE table=default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.322700 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37e81d376e14907b13746cca831a96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.288802 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc07c28ccfd48ee9c33e49442c6047b (DEFAULT_TABLE table=default.Table_d779d44f532f404cad704a689e72f1b3 [id=cabc6cc20754488eb152c0d4f686dce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.326682 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc07c28ccfd48ee9c33e49442c6047b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.290843 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a811d4488d0e4541808a5e1aa44d663b (DEFAULT_TABLE table=default.Table_7fc31b7c8d824f879807f105af51df48 [id=a71b71af19a44182aea7e742062558ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.330935 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a811d4488d0e4541808a5e1aa44d663b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.334203 3870 tablet_service.cc:1467] Processing CreateTablet for tablet e1af33fd98c6486f975368ec90493db7 (DEFAULT_TABLE table=default.Table_34c1bb6042d54870bb236a2204ce71ed [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.334769 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1af33fd98c6486f975368ec90493db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.339053 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 38e24f8d4522466d81605ea37fa7064a (DEFAULT_TABLE table=default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.339640 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e24f8d4522466d81605ea37fa7064a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.353374 4162 raft_consensus.cc:374] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.353554 4162 raft_consensus.cc:729] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.353840 4162 consensus_queue.cc:260] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.356462 4162 ts_tablet_manager.cc:1428] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.073s user 0.002s sys 0.000s
I20260811 18:57:17.356819 4162 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.295095 4815 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.359694 4815 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.361088 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:17.361356 3888 raft_consensus.cc:2454] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:17.361824 3676 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.362473 4576 raft_consensus.cc:2781] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:17.362625 4576 raft_consensus.cc:482] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:17.362731 4576 raft_consensus.cc:3037] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.365971 4576 raft_consensus.cc:504] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.361522 4815 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.367650 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c70d9db064e45af91fd9901d85e211f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:17.367918 3888 raft_consensus.cc:3037] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.371268 3888 raft_consensus.cc:2454] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:17.371795 3676 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.294827 4828 tablet.cc:1722] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:17.296110 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (DEFAULT_TABLE table=default.Table_c86ddee4faa543c79b02e248ac801fb3 [id=7a869d9623bc4e07938b2d26688723a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.372954 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.373303 3861 tablet_service.cc:1467] Processing CreateTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 (DEFAULT_TABLE table=default.Table_ef1abae384a9404089b0f6600e935a5f [id=73a97b63e5db407a94ff739f295cc859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.373836 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4dc2df4c6464d71bb13fe25e0046758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.300797 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.376993 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8304ac7709ca4392baec4cfa78ef5b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.377282 3860 tablet_service.cc:1467] Processing CreateTablet for tablet edd393d9c4d24aab8a170e1329400656 (DEFAULT_TABLE table=default.Table_28b2e1b607a34ab2badc82e3c333a386 [id=2699d23349ac49f0b9a0b92dbab4e3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.377851 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd393d9c4d24aab8a170e1329400656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.381289 4834 raft_consensus.cc:2781] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.382160 4169 tablet_replica.cc:330] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.382483 4169 raft_consensus.cc:2229] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.306644 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.307274 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 (DEFAULT_TABLE table=default.Table_ef1abae384a9404089b0f6600e935a5f [id=73a97b63e5db407a94ff739f295cc859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.388381 4811 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.388607 4811 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.389472 4811 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.390733 4811 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.390919 4811 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.391202 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4687b95728fc485fa0e842bc447eb83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4687b95728fc485fa0e842bc447eb83f already in progress: creating tablet
I20260811 18:57:17.391515 4811 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.392371 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4dc2df4c6464d71bb13fe25e0046758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.393671 4811 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.393843 4811 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.394662 4811 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.398754 4169 raft_consensus.cc:2258] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.399091 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa8a4a1c0cb4834be501bdd7d468472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.400015 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.403847 3824 leader_election.cc:341] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.403198 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.404759 4837 consensus_queue.cc:1035] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:17.405395 4576 leader_election.cc:288] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.405364 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.399639 4163 ts_tablet_manager.cc:1428] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.165s user 0.000s sys 0.003s
I20260811 18:57:17.406525 4163 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.412838 4163 tablet_bootstrap.cc:654] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.312175 3723 tablet_service.cc:1467] Processing CreateTablet for tablet d73a43558dbb48889ab205108f4ce6be (DEFAULT_TABLE table=default.Table_e9433d18ac7b46958942fc1f4b458354 [id=d818dd73be204b458c789ff1d10f7b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.419502 4834 raft_consensus.cc:686] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.426565 4834 consensus_queue.cc:237] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.441962 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73a43558dbb48889ab205108f4ce6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.407384 4169 ts_tablet_manager.cc:1905] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.426561 3824 leader_election.cc:341] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:17.449270 3824 leader_election.cc:341] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.439671 3706 tablet_service.cc:1467] Processing CreateTablet for tablet edd393d9c4d24aab8a170e1329400656 (DEFAULT_TABLE table=default.Table_28b2e1b607a34ab2badc82e3c333a386 [id=2699d23349ac49f0b9a0b92dbab4e3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.454698 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 38e24f8d4522466d81605ea37fa7064a (DEFAULT_TABLE table=default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.461164 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e1af33fd98c6486f975368ec90493db7 (DEFAULT_TABLE table=default.Table_34c1bb6042d54870bb236a2204ce71ed [id=7736472828a3499c8ad8026e164b9759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.478238 4163 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.480417 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd393d9c4d24aab8a170e1329400656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.481036 4163 ts_tablet_manager.cc:1397] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.001s
I20260811 18:57:17.484153 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1af33fd98c6486f975368ec90493db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.485958 4163 raft_consensus.cc:348] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.489372 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e24f8d4522466d81605ea37fa7064a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.493955 4169 ts_tablet_manager.cc:1918] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.505352 4163 raft_consensus.cc:374] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.505528 4163 raft_consensus.cc:729] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.505607 4169 log.cc:1196] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:17.505860 4163 consensus_queue.cc:260] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.506520 4169 ts_tablet_manager.cc:1939] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.506734 4163 ts_tablet_manager.cc:1428] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260811 18:57:17.507102 4163 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.508490 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:17.508790 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 266) took 2675 ms. Trace:
I20260811 18:57:17.509109 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 0c6e0d1eeb63493bbea4cd78dd085a3c deleted and replaced by tablet b203df4124364737839bf4c1ae072988
I20260811 18:57:17.509017 4169 rpcz_store.cc:276] 0811 18:57:14.833664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.833906 (+ 242us) service_pool.cc:227] Handling call
0811 18:57:17.382031 (+2548125us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:17.382053 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.453737 (+ 71684us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.493765 (+ 40028us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.508739 (+ 14974us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15932,"tablet-delete.queue_time_us":2547734}
I20260811 18:57:17.509757 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 267) took 2506 ms. Trace:
I20260811 18:57:17.509898 4169 rpcz_store.cc:276] 0811 18:57:15.003529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.003605 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:17.509562 (+2505957us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:17.509578 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.509721 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2499269}
I20260811 18:57:17.510425 4169 tablet_replica.cc:330] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.510697 4169 raft_consensus.cc:2229] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.511020 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b1ce1042ae974f4dbac964bd7ea16a94 (table default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]) successfully deleted
W20260811 18:57:17.511279 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ff1be026ca404eada53f60f257ea367c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff1be026ca404eada53f60f257ea367c already in progress: creating tablet
I20260811 18:57:17.511299 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.511469 4169 raft_consensus.cc:2258] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.513587 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c18120da2d245f495c52a670047760d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.513669 4169 ts_tablet_manager.cc:1905] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.508924 4163 tablet_bootstrap.cc:654] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.530763 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:17.531634 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ba1e7c81d1bf4209b7089639928742f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.532305 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba1e7c81d1bf4209b7089639928742f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.534405 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.535074 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet dde27d8cc3eb4d09879e717fc90305fd with delete_type TABLET_DATA_DELETED (replaced by tablet 0b3ec38f9b55489ba00ed76f9e4456f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.537292 4169 ts_tablet_manager.cc:1918] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.537911 4169 log.cc:1196] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:17.538509 4169 ts_tablet_manager.cc:1939] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.539394 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 268) took 2442 ms. Trace:
I20260811 18:57:17.540277 4169 rpcz_store.cc:276] 0811 18:57:15.096785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.096854 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:17.510327 (+2413473us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:17.510370 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.518234 (+ 7864us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.537149 (+ 18915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.539320 (+ 2171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2412993}
W20260811 18:57:17.540405 3823 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:17.541034 3540 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:17.541244 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 9 similar messages]
I20260811 18:57:17.541302 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a300a7aca2814da6b999eb4d865fea66 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:17.541519 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260811 18:57:17.541569 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 269) took 2346 ms. Trace:
I20260811 18:57:17.541674 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:17.541684 4169 rpcz_store.cc:276] 0811 18:57:15.195062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.195145 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:17.541389 (+2346244us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:17.541403 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.541514 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2345657}
W20260811 18:57:17.542163 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fbba1923ce0049448fe4e28f2dd258dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbba1923ce0049448fe4e28f2dd258dc already in progress: creating tablet
I20260811 18:57:17.542802 4169 tablet_replica.cc:330] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:17.549517 3677 connection.cc:535] client connection to 127.3.72.126:34821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:17.555217 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.556661 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.556949 4162 tablet_bootstrap.cc:654] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.560159 4163 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.560523 4163 ts_tablet_manager.cc:1397] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.053s user 0.005s sys 0.002s
I20260811 18:57:17.561740 4163 raft_consensus.cc:348] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.562259 4163 raft_consensus.cc:374] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.562445 4163 raft_consensus.cc:729] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.562848 4163 consensus_queue.cc:260] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.563426 4169 raft_consensus.cc:2229] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.566402 4169 raft_consensus.cc:2258] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.567199 4163 ts_tablet_manager.cc:1428] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:17.570449 4163 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.571231 4169 ts_tablet_manager.cc:1905] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.578006 4848 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.553023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.3.72.65:56675)
0811 18:57:17.577456 (+ 24433us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.577652 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":24254,"threads_started":1}
W20260811 18:57:17.589577 4849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.560625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.0.0.1:39266)
0811 18:57:17.588826 (+ 28201us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.589079 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":27985,"threads_started":1}
I20260811 18:57:17.598001 4163 tablet_bootstrap.cc:654] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:17.598505 4846 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:17.543633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.3.72.66:51551)
0811 18:57:17.597611 (+ 53978us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:17.597885 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53795,"threads_started":1}
I20260811 18:57:17.600116 4167 ts_tablet_manager.cc:1918] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.600383 4167 log.cc:1196] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/807cee88c18340dbb23c51cad54ab56d
I20260811 18:57:17.602214 4167 ts_tablet_manager.cc:1939] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.603032 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 235) took 3649 ms. Trace:
I20260811 18:57:17.605762 4167 rpcz_store.cc:276] 0811 18:57:13.953621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:13.954017 (+ 396us) service_pool.cc:227] Handling call
0811 18:57:17.191770 (+3237753us) ts_tablet_manager.cc:1145] Deleting tablet 807cee88c18340dbb23c51cad54ab56d
0811 18:57:17.191784 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.455348 (+263564us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.598331 (+142983us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.602982 (+ 4651us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3237160}
I20260811 18:57:17.611608 4169 ts_tablet_manager.cc:1918] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:17.611771 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:17.612999 4163 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.613301 4163 ts_tablet_manager.cc:1397] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.003s
I20260811 18:57:17.613664 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 807cee88c18340dbb23c51cad54ab56d (table default.Table_420484ea1b7f491bb6f1c10bfffed8f3 [id=2d8084b177b44c4cbc14063dccfc21e7]) successfully deleted
I20260811 18:57:17.614270 4163 raft_consensus.cc:348] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.614717 4163 raft_consensus.cc:374] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.614884 4163 raft_consensus.cc:729] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.615218 4163 consensus_queue.cc:260] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.622715 4163 ts_tablet_manager.cc:1428] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:17.627497 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 237) took 3567 ms. Trace:
I20260811 18:57:17.629243 4811 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.629480 4811 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.630479 4811 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.634554 4163 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.636646 4163 tablet_bootstrap.cc:654] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.637943 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:17.638609 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:17.639076 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 968fa8b8912b40a8bdba77db1d39c5c0 deleted and replaced by tablet f287040b879642448367abe2b1c73cd9
I20260811 18:57:17.640048 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:17.640477 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 987b9ac6859444e3a86c56af786cc84e deleted and replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104
I20260811 18:57:17.640695 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.640774 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet d2d7d986900244ddbc05dbe73a708dcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.636878 4169 log.cc:1196] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/dde27d8cc3eb4d09879e717fc90305fd
I20260811 18:57:17.642307 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 987b9ac6859444e3a86c56af786cc84e with delete_type TABLET_DATA_DELETED (replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.642686 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.642961 4811 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.643177 4811 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.644061 4811 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.644886 4169 ts_tablet_manager.cc:1939] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.645633 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 270) took 2421 ms. Trace:
I20260811 18:57:17.645810 4169 rpcz_store.cc:276] 0811 18:57:15.224444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.224503 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:17.542714 (+2318211us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:17.542723 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.585119 (+ 42396us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.611413 (+ 26294us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.645588 (+ 34175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2275,"tablet-delete.queue_time_us":2317907}
I20260811 18:57:17.647040 4169 tablet_replica.cc:330] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.647380 4169 raft_consensus.cc:2229] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.647622 4169 raft_consensus.cc:2258] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.648427 4169 ts_tablet_manager.cc:1905] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.631584 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.633876 4739 consensus_queue.cc:1035] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:17.657620 3546 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 2dcfbf6f46e34163a230c0c82df7ecfa
I20260811 18:57:17.658651 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:17.660106 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.663522 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9c18120da2d245f495c52a670047760d
I20260811 18:57:17.664021 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 9c18120da2d245f495c52a670047760d deleted and replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba
I20260811 18:57:17.665032 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c18120da2d245f495c52a670047760d with delete_type TABLET_DATA_DELETED (replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.665721 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b2d444511ad44775b701b9ed7527e251" } new_table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" modify_external_catalogs: true
I20260811 18:57:17.666075 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5120347318f64721a3d6ef8cdc5564cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.666663 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8" } modify_external_catalogs: true
I20260811 18:57:17.666859 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b717724905d74a6a865cd6eb35c3a4d8" } modify_external_catalogs: true
I20260811 18:57:17.668159 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e2b67d7e59584a2bb877f79bdafed3a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.669376 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4efc43ae63254c67aac88d7bd9b6c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.669750 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d779d44f532f404cad704a689e72f1b3" } new_table_name: "default.Table_7de254c88a674097b5df7ea987172a9e" modify_external_catalogs: true
W20260811 18:57:17.671533 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.672057 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2b67d7e59584a2bb877f79bdafed3a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.673887 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5120347318f64721a3d6ef8cdc5564cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.682821 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24955fa6843f46dc883be60173049a16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.683917 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24955fa6843f46dc883be60173049a16 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.687554 4163 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.687769 4163 ts_tablet_manager.cc:1397] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.003s
I20260811 18:57:17.689112 4163 raft_consensus.cc:348] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.689465 4163 raft_consensus.cc:374] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.689560 4163 raft_consensus.cc:729] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.689882 4163 consensus_queue.cc:260] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:17.690140 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62c365f79d4a4201ac323f16bddf3e62 consensus state has no leader
I20260811 18:57:17.690616 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b07958bac6344da1bbf3d7d35f1ac551" } new_table_name: "default.Table_d9b7a2cfa85347bf88241286c4bf0083" modify_external_catalogs: true
I20260811 18:57:17.690915 4163 ts_tablet_manager.cc:1428] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
#1 0x7fc7c79f9671 in kudu::rw_semaphore::unlock() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/rw_semaphore.h:148:5
I20260811 18:57:17.709129 4167 rpcz_store.cc:276] 0811 18:57:14.059853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.059948 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:17.613322 (+3553374us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:17.613343 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.627364 (+ 14021us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3518185}
W20260811 18:57:17.709659 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:17.709728 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 236) took 3650 ms. Trace:
I20260811 18:57:17.709867 4167 rpcz_store.cc:276] 0811 18:57:14.059261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.069044 (+ 9783us) service_pool.cc:227] Handling call
0811 18:57:17.709537 (+3640493us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:17.709552 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.709683 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3614280}
I20260811 18:57:17.710393 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 238) took 3605 ms. Trace:
I20260811 18:57:17.710216 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7c4738e833404ad7b1947bb598f05842"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.710517 4167 rpcz_store.cc:276] 0811 18:57:14.104581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.104676 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:17.710099 (+3605423us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:17.710110 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.710356 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3604791}
W20260811 18:57:17.710934 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c4738e833404ad7b1947bb598f05842 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.712168 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 239) took 3595 ms. Trace:
I20260811 18:57:17.712184 4163 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.712307 4167 rpcz_store.cc:276] 0811 18:57:14.116675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.116785 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:17.712012 (+3595227us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:17.712023 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.712126 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3594737}
I20260811 18:57:17.712563 4167 tablet_replica.cc:330] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:17.712786 4167 raft_consensus.cc:2229] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.712962 4167 raft_consensus.cc:2258] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.713557 4167 ts_tablet_manager.cc:1905] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.714628 4163 tablet_bootstrap.cc:654] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.721661 4169 ts_tablet_manager.cc:1918] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.721896 4169 log.cc:1196] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:17.722477 4169 ts_tablet_manager.cc:1939] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.722945 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 282) took 2116 ms. Trace:
I20260811 18:57:17.723112 4169 rpcz_store.cc:276] 0811 18:57:15.606882 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.606945 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:17.646952 (+2040007us) ts_tablet_manager.cc:1145] Deleting tablet 2f9fa42654cf4ec18220e5fb173ddef7
0811 18:57:17.646969 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.717105 (+ 70136us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.721487 (+ 4382us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.722871 (+ 1384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2008832}
I20260811 18:57:17.723733 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2f9fa42654cf4ec18220e5fb173ddef7 (table default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]) successfully deleted
I20260811 18:57:17.727128 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 287) took 2053 ms. Trace:
I20260811 18:57:17.727312 4169 rpcz_store.cc:276] 0811 18:57:15.673272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.673489 (+ 217us) service_pool.cc:227] Handling call
0811 18:57:17.726825 (+2053336us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:17.726845 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.727037 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2023052}
I20260811 18:57:17.733035 4169 tablet_replica.cc:330] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.733368 4169 raft_consensus.cc:2229] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:17.733857 4169 raft_consensus.cc:2258] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:17.736295 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62c365f79d4a4201ac323f16bddf3e62 consensus state has no leader
I20260811 18:57:17.736512 4169 ts_tablet_manager.cc:1905] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:17.737748 3824 leader_election.cc:341] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.739167 4163 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.739347 4163 ts_tablet_manager.cc:1397] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.006s
I20260811 18:57:17.739845 4836 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.740278 4163 raft_consensus.cc:348] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.740018 4836 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.740561 4163 raft_consensus.cc:374] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.740692 4163 raft_consensus.cc:729] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.740998 4163 consensus_queue.cc:260] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.741461 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:17.741866 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:17.742167 3891 raft_consensus.cc:2454] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:17.742462 3824 leader_election.cc:341] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.742858 3676 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.744616 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.743551 4836 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.745906 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c18120da2d245f495c52a670047760d with delete_type TABLET_DATA_DELETED (replaced by tablet 6f6b2ca385374abeb709fccbdf6fa0ba) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.746163 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 987b9ac6859444e3a86c56af786cc84e with delete_type TABLET_DATA_DELETED (replaced by tablet 4ec1fa93c2bf4008bdff3f8e86fec104) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.746376 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:17.746907 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 405773849fb94800be61ad8a7a6055a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 405773849fb94800be61ad8a7a6055a4 already in progress: creating tablet
W20260811 18:57:17.747283 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 02808605ee6c4634af47fc3fd57e9cd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02808605ee6c4634af47fc3fd57e9cd8 already in progress: creating tablet
W20260811 18:57:17.747541 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9bc5448a555944dfba8ccae09e25cbe9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bc5448a555944dfba8ccae09e25cbe9 already in progress: creating tablet
W20260811 18:57:17.747853 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c08436f33db043b1bc6395cd92102e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c08436f33db043b1bc6395cd92102e59 already in progress: creating tablet
I20260811 18:57:17.748301 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:17.748373 4811 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.748587 4811 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.749480 4811 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:17.749675 4834 raft_consensus.cc:2781] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:17.749862 4834 raft_consensus.cc:482] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:17.749993 4834 raft_consensus.cc:3037] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.750029 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:17.750838 3824 leader_election.cc:341] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:17.754258 4834 raft_consensus.cc:504] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.758585 4834 leader_election.cc:288] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.759725 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d6daff4b7540309ff58755758e9752" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:17.760085 3891 raft_consensus.cc:3037] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:17.762885 4162 tablet_bootstrap.cc:492] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.763101 4162 ts_tablet_manager.cc:1397] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.406s user 0.003s sys 0.007s
I20260811 18:57:17.763994 4162 raft_consensus.cc:348] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.764323 4162 raft_consensus.cc:374] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.764418 4162 raft_consensus.cc:729] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.764680 4162 consensus_queue.cc:260] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.765097 3891 raft_consensus.cc:2454] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:17.765336 4162 ts_tablet_manager.cc:1428] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:17.765770 3676 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:17.759491 4163 ts_tablet_manager.cc:1428] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.020s user 0.001s sys 0.002s
I20260811 18:57:17.766301 4836 raft_consensus.cc:2781] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:17.766633 4836 raft_consensus.cc:686] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:17.767367 4836 consensus_queue.cc:237] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.772644 4163 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.775126 4163 tablet_bootstrap.cc:654] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.780707 4162 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.788740 4162 tablet_bootstrap.cc:654] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.797640 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_12d05bfe0d2348f0a14ef6b8ecf8414a" } new_table_name: "default.Table_5781967ec2ee41a5ab982d9d7394c274" modify_external_catalogs: true
I20260811 18:57:17.800720 4169 ts_tablet_manager.cc:1918] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.800926 4169 log.cc:1196] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:17.801226 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056e34af1fcd4642abd908808acf8a67" } new_table_name: "default.Table_42b47e0e7e404c8c976e7472833fd243" modify_external_catalogs: true
I20260811 18:57:17.801486 4169 ts_tablet_manager.cc:1939] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.802167 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 288) took 2110 ms. Trace:
I20260811 18:57:17.802317 4169 rpcz_store.cc:276] 0811 18:57:15.692252 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.692325 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:17.732912 (+2040587us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:17.732932 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.783044 (+ 50112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.800518 (+ 17474us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.802114 (+ 1596us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2028732}
I20260811 18:57:17.802876 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6ce8db230c7e41b6a4792273a0f07e6e (table default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]) successfully deleted
I20260811 18:57:17.805891 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 289) took 2098 ms. Trace:
I20260811 18:57:17.806080 4169 rpcz_store.cc:276] 0811 18:57:15.707111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.707175 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:17.805291 (+2098116us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:17.805310 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.805834 (+ 524us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097608}
W20260811 18:57:17.806728 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 already in progress: creating tablet
W20260811 18:57:17.808802 3824 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.183s (SENT)
I20260811 18:57:17.809406 4169 tablet_replica.cc:330] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.809731 4169 raft_consensus.cc:2229] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.810009 4169 raft_consensus.cc:2258] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.819176 4169 ts_tablet_manager.cc:1905] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.823889 4163 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.824146 4163 ts_tablet_manager.cc:1397] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.000s
I20260811 18:57:17.825385 4163 raft_consensus.cc:348] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.825719 4163 raft_consensus.cc:374] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.825835 4163 raft_consensus.cc:729] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.826181 4163 consensus_queue.cc:260] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.833021 4163 ts_tablet_manager.cc:1428] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:57:17.832902 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3e695533c42647128860db601dcc0b5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.833310 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1b071ee4d3e847099b98398e52574f8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.833607 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e695533c42647128860db601dcc0b5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.833879 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b071ee4d3e847099b98398e52574f8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:17.837958 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c99ceca57cb245f4a3a6cb4b36f1306d consensus state has no leader
I20260811 18:57:17.838361 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_241ec7c366e84652a6d9c0123d2149ac" } modify_external_catalogs: true
I20260811 18:57:17.838711 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_241ec7c366e84652a6d9c0123d2149ac" } modify_external_catalogs: true
I20260811 18:57:17.838925 4163 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.841444 4163 tablet_bootstrap.cc:654] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.863399 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 38e24f8d4522466d81605ea37fa7064a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:17.867286 3860 tablet_service.cc:1467] Processing CreateTablet for tablet b20872d80c9545e5822bfc82d496ce97 (DEFAULT_TABLE table=default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.867442 3555 catalog_manager.cc:5495] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:17.867950 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20872d80c9545e5822bfc82d496ce97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.867966 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:17.868697 3550 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4] alter complete (version 1)
I20260811 18:57:17.873369 3861 tablet_service.cc:1467] Processing CreateTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.873942 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14f8717bcd94d6d88295fd8be130ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.875392 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.876044 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.877990 3709 tablet_service.cc:1467] Processing CreateTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.878574 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14f8717bcd94d6d88295fd8be130ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.881398 4167 ts_tablet_manager.cc:1918] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.881587 4167 log.cc:1196] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7f640c60e81b4a369458b1497a269c9c
I20260811 18:57:17.882138 4167 ts_tablet_manager.cc:1939] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:17.884609 3723 tablet_service.cc:1467] Processing CreateTablet for tablet b20872d80c9545e5822bfc82d496ce97 (DEFAULT_TABLE table=default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.885171 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20872d80c9545e5822bfc82d496ce97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.886536 4162 tablet_bootstrap.cc:492] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:17.886708 4162 ts_tablet_manager.cc:1397] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.002s
I20260811 18:57:17.887591 4162 raft_consensus.cc:348] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.887853 4162 raft_consensus.cc:374] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.887960 4162 raft_consensus.cc:729] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.888260 4162 consensus_queue.cc:260] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.888685 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ce1042ae974f4dbac964bd7ea16a94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.888900 4162 ts_tablet_manager.cc:1428] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:17.889061 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:17.889232 4162 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:17.889873 3865 tablet_service.cc:1467] Processing CreateTablet for tablet cd1fc828f73642c88171d6ac827de90e (DEFAULT_TABLE table=default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.890389 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1fc828f73642c88171d6ac827de90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.894672 3862 tablet_service.cc:1467] Processing CreateTablet for tablet a0116997a1c64e658045c319f1c45861 (DEFAULT_TABLE table=default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.895289 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0116997a1c64e658045c319f1c45861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.898638 4822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:17.899127 4822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
W20260811 18:57:17.900777 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a9efd2f19d74a68a8536fd057377738 consensus state has no leader
I20260811 18:57:17.901180 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a82335ab20e412c93a5ac0938a96012" } modify_external_catalogs: true
I20260811 18:57:17.901257 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a (DEFAULT_TABLE table=default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.901399 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a82335ab20e412c93a5ac0938a96012" } modify_external_catalogs: true
I20260811 18:57:17.901850 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf7720e31d84ae2ab592da1e7a3132a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.906956 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 330801031cc74eb2a5bc2caf53a18ead (DEFAULT_TABLE table=default.Table_e23d79bc476d496eaa713465bcfb1abf [id=54b88a6b23fe45e8b0f235baf8f7234f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.907586 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330801031cc74eb2a5bc2caf53a18ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.909198 4169 ts_tablet_manager.cc:1918] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:17.909361 4169 log.cc:1196] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:17.909861 4169 ts_tablet_manager.cc:1939] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.910374 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 285) took 2263 ms. Trace:
I20260811 18:57:17.910494 4169 rpcz_store.cc:276] 0811 18:57:15.646846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.649334 (+ 2488us) service_pool.cc:227] Handling call
0811 18:57:17.809318 (+2159984us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:17.809337 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.859685 (+ 50348us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.909027 (+ 49342us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.910288 (+ 1261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2097471}
I20260811 18:57:17.910868 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe24bb6edb944008ff9cc73faa7374c (DEFAULT_TABLE table=default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b [id=c52d624d402741e8a390e01b9b5056b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.911229 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 290) took 2182 ms. Trace:
I20260811 18:57:17.911341 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe24bb6edb944008ff9cc73faa7374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.914820 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 4a64df7c295c4d9baded1b63a9980581 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.915359 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a64df7c295c4d9baded1b63a9980581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.916683 4729 raft_consensus.cc:482] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:17.916888 4729 raft_consensus.cc:504] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:17.917701 4729 leader_election.cc:288] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:17.918849 3872 tablet_service.cc:1467] Processing CreateTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 (DEFAULT_TABLE table=default.Table_3cc27811ad29451daaebbf31fb3ad198 [id=17fdd095e10d41fe8033aad4d1034af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.919374 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfea0cd619024204b9bff1e2dd62b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.922148 4163 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:17.922317 4163 ts_tablet_manager.cc:1397] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.003s
I20260811 18:57:17.922854 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 (DEFAULT_TABLE table=default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:17.923416 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec4bba841dd446aa3db751358ce7f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:17.923470 4163 raft_consensus.cc:348] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:17.923789 4163 raft_consensus.cc:374] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:17.923909 4163 raft_consensus.cc:729] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:17.924227 4163 consensus_queue.cc:260] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:17.924861 4163 ts_tablet_manager.cc:1428] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:17.925285 4163 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:17.911350 4169 rpcz_store.cc:276] 0811 18:57:15.728729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.729246 (+ 517us) service_pool.cc:227] Handling call
0811 18:57:17.910881 (+2181635us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:17.910898 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.911042 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2167388}
I20260811 18:57:17.927215 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 293) took 2132 ms. Trace:
I20260811 18:57:17.927313 4169 rpcz_store.cc:276] 0811 18:57:15.795173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795241 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:17.926974 (+2131733us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:17.926987 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.927148 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2129871}
I20260811 18:57:17.927646 4169 tablet_replica.cc:330] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.927878 4169 raft_consensus.cc:2229] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.928113 4169 raft_consensus.cc:2258] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.928720 4169 ts_tablet_manager.cc:1905] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.943271 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8426ccbe530446fcbaa162a03cbb1892 (table default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]) successfully deleted
W20260811 18:57:17.943646 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45aff3e8cb0c4f7191e4522c6ef7d156 already in progress: creating tablet
W20260811 18:57:17.943964 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 36724a83bf9844b9a27a88ca36a9bf50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36724a83bf9844b9a27a88ca36a9bf50 already in progress: creating tablet
W20260811 18:57:17.949676 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 122b7898fea44ef78f269b1505d8ddc4 consensus state has no leader
I20260811 18:57:17.951823 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8af423b4350249d6bae77e7e3c7d9fa4" } modify_external_catalogs: true
I20260811 18:57:17.952109 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8af423b4350249d6bae77e7e3c7d9fa4" } modify_external_catalogs: true
I20260811 18:57:17.953423 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f42592ab70540e59f42f7e97293a058" } modify_external_catalogs: true
I20260811 18:57:17.953629 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9f42592ab70540e59f42f7e97293a058" } modify_external_catalogs: true
I20260811 18:57:17.949302 4163 tablet_bootstrap.cc:654] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:17.976004 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:17.976848 4169 ts_tablet_manager.cc:1918] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:17.976995 4169 log.cc:1196] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5ae200f3814e431a8288c30fe94bffcf
I20260811 18:57:17.977480 4169 ts_tablet_manager.cc:1939] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:17.977833 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 294) took 2180 ms. Trace:
I20260811 18:57:17.977952 4169 rpcz_store.cc:276] 0811 18:57:15.797057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.797117 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:17.927562 (+2130445us) ts_tablet_manager.cc:1145] Deleting tablet 5ae200f3814e431a8288c30fe94bffcf
0811 18:57:17.927570 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.953769 (+ 26199us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.976686 (+ 22917us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.977796 (+ 1110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2121989}
W20260811 18:57:17.978492 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b53eaa03dd34a22a14fa65da3866fe4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:17.985392 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ebf7720e31d84ae2ab592da1e7a3132a
I20260811 18:57:17.987109 4169 tablet_replica.cc:330] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:17.987389 4169 raft_consensus.cc:2229] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:17.987356 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5ae200f3814e431a8288c30fe94bffcf (table default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]) successfully deleted
I20260811 18:57:17.991201 4169 raft_consensus.cc:2258] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:17.992760 4169 ts_tablet_manager.cc:1905] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:17.994378 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be30e8194fd44874a1fdd996128eb4d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:17.995195 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be30e8194fd44874a1fdd996128eb4d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.001415 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.004695 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:18.006580 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:17.950120 3676 leader_election.cc:341] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 151df87806694f46be5013592c68c01a
I20260811 18:57:18.012598 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.013012 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 (DEFAULT_TABLE table=default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.013206 3718 tablet_service.cc:1467] Processing CreateTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a (DEFAULT_TABLE table=default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.013746 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.014250 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe24bb6edb944008ff9cc73faa7374c (DEFAULT_TABLE table=default.Table_9a0c65f5bd8f4152a6fe2339d28ad59b [id=c52d624d402741e8a390e01b9b5056b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.015468 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2bb9c598e2de422b86a8304bbece96db" } new_table_name: "default.Table_559eff9548514310840eee6c9a02b364" modify_external_catalogs: true
I20260811 18:57:18.019548 3706 tablet_service.cc:1467] Processing CreateTablet for tablet cd1fc828f73642c88171d6ac827de90e (DEFAULT_TABLE table=default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:18.020144 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 66869.4us
I20260811 18:57:18.020678 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4a64df7c295c4d9baded1b63a9980581 (DEFAULT_TABLE table=default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.021322 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ec4bba841dd446aa3db751358ce7f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.021428 3715 tablet_service.cc:1467] Processing CreateTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 (DEFAULT_TABLE table=default.Table_3cc27811ad29451daaebbf31fb3ad198 [id=17fdd095e10d41fe8033aad4d1034af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.021898 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfea0cd619024204b9bff1e2dd62b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.021369 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 330801031cc74eb2a5bc2caf53a18ead (DEFAULT_TABLE table=default.Table_e23d79bc476d496eaa713465bcfb1abf [id=54b88a6b23fe45e8b0f235baf8f7234f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.023916 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330801031cc74eb2a5bc2caf53a18ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.025117 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe24bb6edb944008ff9cc73faa7374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.026053 4169 ts_tablet_manager.cc:1918] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.026217 4169 log.cc:1196] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d3d6daff4b7540309ff58755758e9752
I20260811 18:57:18.026664 4169 ts_tablet_manager.cc:1939] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.026796 4729 raft_consensus.cc:482] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.026988 4729 raft_consensus.cc:504] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.027750 4729 leader_election.cc:288] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.027892 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1fc828f73642c88171d6ac827de90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.030558 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a64df7c295c4d9baded1b63a9980581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.031039 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf7720e31d84ae2ab592da1e7a3132a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:18.034984 3676 leader_election.cc:341] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: dde27d8cc3eb4d09879e717fc90305fd
I20260811 18:57:18.038069 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.039134 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 291) took 2244 ms. Trace:
I20260811 18:57:18.041402 4162 tablet_bootstrap.cc:654] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.041671 4169 rpcz_store.cc:276] 0811 18:57:15.794573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795803 (+ 1230us) service_pool.cc:227] Handling call
0811 18:57:17.987021 (+2191218us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:17.987033 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.999212 (+ 12179us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.025917 (+ 26705us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.039069 (+ 13152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2176147}
I20260811 18:57:18.043712 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d3d6daff4b7540309ff58755758e9752 (table default.Table_923c4369260d436fafb1aff17a2ec8d4 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]) successfully deleted
I20260811 18:57:18.044154 4169 tablet_replica.cc:330] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.044416 4169 raft_consensus.cc:2229] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.044665 4169 raft_consensus.cc:2258] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.045290 4169 ts_tablet_manager.cc:1905] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.068277 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 240) took 3827 ms. Trace:
I20260811 18:57:18.068542 4167 rpcz_store.cc:276] 0811 18:57:14.120397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.120500 (+ 103us) service_pool.cc:227] Handling call
0811 18:57:17.712502 (+3592002us) ts_tablet_manager.cc:1145] Deleting tablet 7f640c60e81b4a369458b1497a269c9c
0811 18:57:17.712511 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:17.836379 (+123868us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.881229 (+ 44850us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:17.947309 (+ 66080us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3591489}
I20260811 18:57:18.069460 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7f640c60e81b4a369458b1497a269c9c (table default.Table_7e851b9ff216483bb8768c88e4b6e1b9 [id=5c9e64742a0a4d80b8035ccb50459a2e]) successfully deleted
I20260811 18:57:18.069514 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 241) took 3875 ms. Trace:
I20260811 18:57:18.069650 4167 rpcz_store.cc:276] 0811 18:57:14.194541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.198569 (+ 4028us) service_pool.cc:227] Handling call
0811 18:57:18.069270 (+3870701us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:18.069288 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.069436 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3870186}
W20260811 18:57:18.070292 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8c07150e31994bdf9a00ac2e7a954691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c07150e31994bdf9a00ac2e7a954691 already in progress: creating tablet
I20260811 18:57:18.070740 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a0116997a1c64e658045c319f1c45861 (DEFAULT_TABLE table=default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.071374 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0116997a1c64e658045c319f1c45861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.075193 4167 tablet_replica.cc:330] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.075587 4167 raft_consensus.cc:2229] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.076617 4167 pending_rounds.cc:62] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:18.076879 4167 pending_rounds.cc:68] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317400108963278848 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "e7b3a14979e84b1e8ec6ed3f49b93b00" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_44e9cbdec47545899467895fd869b389" dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" new_extra_config { } }
W20260811 18:57:18.078389 4167 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:38286: Aborted: Op aborted
I20260811 18:57:18.079641 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38286 (request call id 281) took 2202 ms. Trace:
I20260811 18:57:18.080503 4163 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.081655 3676 consensus_peers.cc:487] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f8c0444ec2242299482736a73551f80. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.082139 4167 rpcz_store.cc:276] 0811 18:57:15.876834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.879172 (+ 2338us) service_pool.cc:227] Handling call
0811 18:57:18.079598 (+2200426us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:15.977183 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:15.977349 (+ 166us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:15.977379 (+ 30us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:15.977385 (+ 6us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:15.977433 (+ 48us) alter_schema_op.cc:126] START. Timestamp: 1786474635977363
0811 18:57:15.977437 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:15.977655 (+ 218us) log.cc:842] Serialized 173 byte log entry
0811 18:57:18.078255 (+2100600us) op_driver.cc:444] REPLICATION: finished
0811 18:57:18.078289 (+ 34us) op_driver.cc:372] HandleFailure(Aborted: Op aborted)
0811 18:57:18.078299 (+ 10us) alter_schema_op.cc:169] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":273,"prepare.run_cpu_time_us":6887,"prepare.run_wall_time_us":233422,"replication_time_us":2100769,"thread_start_us":219309,"threads_started":1,"wal-append.queue_time_us":190675,"wal-append.run_cpu_time_us":2476,"wal-append.run_wall_time_us":1215875}]]}
I20260811 18:57:18.082857 4163 ts_tablet_manager.cc:1397] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.158s user 0.007s sys 0.011s
I20260811 18:57:18.083889 4163 raft_consensus.cc:348] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.083961 4162 tablet_bootstrap.cc:492] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.085772 4162 ts_tablet_manager.cc:1397] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.197s user 0.006s sys 0.000s
I20260811 18:57:18.086961 4162 raft_consensus.cc:348] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.087371 4162 raft_consensus.cc:374] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.087529 4162 raft_consensus.cc:729] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.088001 4162 consensus_queue.cc:260] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.090145 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.056s (SENT) [suppressed 13 similar messages]
I20260811 18:57:18.062252 4169 ts_tablet_manager.cc:1918] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.093575 4169 log.cc:1196] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cf285cdf5235451bb1531cb294017101
I20260811 18:57:18.093896 4163 raft_consensus.cc:374] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.095602 4162 ts_tablet_manager.cc:1428] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:18.094172 4169 ts_tablet_manager.cc:1939] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.096565 4163 raft_consensus.cc:729] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.097486 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--webserver_interface=127.3.72.126
--webserver_port=40953
--builtin_ntp_servers=127.3.72.84:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:57:18.097678 4163 consensus_queue.cc:260] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.098412 4163 ts_tablet_manager.cc:1428] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.001s
I20260811 18:57:18.098798 4163 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.114094 4167 raft_consensus.cc:2258] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.115231 4167 ts_tablet_manager.cc:1905] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.096984 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 292) took 2301 ms. Trace:
I20260811 18:57:18.115634 4169 rpcz_store.cc:276] 0811 18:57:15.795014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.795475 (+ 461us) service_pool.cc:227] Handling call
0811 18:57:18.044047 (+2248572us) ts_tablet_manager.cc:1145] Deleting tablet cf285cdf5235451bb1531cb294017101
0811 18:57:18.044063 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.058852 (+ 14789us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.062113 (+ 3261us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.096942 (+ 34829us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2212969}
I20260811 18:57:18.116355 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cf285cdf5235451bb1531cb294017101 (table default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]) successfully deleted
I20260811 18:57:18.116849 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 295) took 2305 ms. Trace:
I20260811 18:57:18.117020 4169 rpcz_store.cc:276] 0811 18:57:15.811013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.811429 (+ 416us) service_pool.cc:227] Handling call
0811 18:57:18.116694 (+2305265us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:18.116708 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.116813 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2243388}
W20260811 18:57:18.117552 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ad424c8843ff434ba0449a86849bb5a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad424c8843ff434ba0449a86849bb5a0 already in progress: creating tablet
I20260811 18:57:18.118010 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 311) took 2043 ms. Trace:
I20260811 18:57:18.118175 4169 rpcz_store.cc:276] 0811 18:57:16.074900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.074986 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:18.117869 (+2042883us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:18.117881 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.117978 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2034890}
W20260811 18:57:18.118693 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b3ec38f9b55489ba00ed76f9e4456f5 already in progress: creating tablet
I20260811 18:57:18.119047 4169 tablet_replica.cc:330] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.119314 4169 raft_consensus.cc:2229] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.119557 4169 raft_consensus.cc:2258] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.120249 4169 ts_tablet_manager.cc:1905] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.123118 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5120347318f64721a3d6ef8cdc5564cd" } new_table_name: "default.Table_a8d2d56f43544d68adbbd2848fc00dde" modify_external_catalogs: true
I20260811 18:57:18.114424 4162 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.129352 4162 tablet_bootstrap.cc:654] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.136024 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 38e24f8d4522466d81605ea37fa7064a
I20260811 18:57:18.136510 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 38e24f8d4522466d81605ea37fa7064a deleted and replaced by tablet c17847a358e544b4b4e51f65463a9679
I20260811 18:57:18.139307 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.140015 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.152510 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 4687b95728fc485fa0e842bc447eb83f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.154834 4162 tablet_bootstrap.cc:492] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.155055 4162 ts_tablet_manager.cc:1397] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.000s
I20260811 18:57:18.155886 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 8426ccbe530446fcbaa162a03cbb1892 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.156379 4162 raft_consensus.cc:348] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.156782 4162 raft_consensus.cc:374] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.156905 4162 raft_consensus.cc:729] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.157186 4162 consensus_queue.cc:260] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.159427 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01f64f05d1ee44029680c5423fd0d81b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.160223 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01f64f05d1ee44029680c5423fd0d81b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.164161 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a30bb384136f4a169a5751655437d5a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.165304 4162 ts_tablet_manager.cc:1428] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:18.171603 4162 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.115921 4163 tablet_bootstrap.cc:654] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.175804 4167 ts_tablet_manager.cc:1918] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:18.177737 4167 log.cc:1196] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e7b3a14979e84b1e8ec6ed3f49b93b00
I20260811 18:57:18.181793 4162 tablet_bootstrap.cc:654] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.184104 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7bfc22ccd3e44c679f5e951318bb0af3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.184824 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bfc22ccd3e44c679f5e951318bb0af3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.185266 4167 ts_tablet_manager.cc:1939] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.186743 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet cf285cdf5235451bb1531cb294017101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:03 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.185990 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 242) took 3813 ms. Trace:
I20260811 18:57:18.187811 4167 rpcz_store.cc:276] 0811 18:57:14.372534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.372776 (+ 242us) service_pool.cc:227] Handling call
0811 18:57:18.075052 (+3702276us) ts_tablet_manager.cc:1145] Deleting tablet e7b3a14979e84b1e8ec6ed3f49b93b00
0811 18:57:18.075066 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.137895 (+ 62829us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.175624 (+ 37729us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.185925 (+ 10301us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3701810}
I20260811 18:57:18.188669 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e7b3a14979e84b1e8ec6ed3f49b93b00 (table default.Table_44e9cbdec47545899467895fd869b389 [id=255f252ebd6245bd928387a45609ddcb]) successfully deleted
I20260811 18:57:18.189182 4167 tablet_replica.cc:330] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.189548 4167 raft_consensus.cc:2229] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.190058 4167 raft_consensus.cc:2258] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.191397 4167 ts_tablet_manager.cc:1905] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.200928 4162 tablet_bootstrap.cc:492] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.201267 4162 ts_tablet_manager.cc:1397] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.001s
I20260811 18:57:18.202358 4162 raft_consensus.cc:348] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.202826 4162 raft_consensus.cc:374] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.202965 4162 raft_consensus.cc:729] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.203378 4162 consensus_queue.cc:260] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.205153 4163 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.205395 4163 ts_tablet_manager.cc:1397] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.003s
I20260811 18:57:18.206312 4163 raft_consensus.cc:348] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.206992 4163 raft_consensus.cc:374] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.207154 4163 raft_consensus.cc:729] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.207463 4163 consensus_queue.cc:260] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.209573 4163 ts_tablet_manager.cc:1428] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:57:18.209903 4163 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.213186 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:18.214327 4162 ts_tablet_manager.cc:1428] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260811 18:57:18.215492 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_914c8b90fadf48e88b75d13897462d84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.216181 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_914c8b90fadf48e88b75d13897462d84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.217484 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 4ec1fa93c2bf4008bdff3f8e86fec104
I20260811 18:57:18.217856 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c1631122dcd4c738fcb59926b7873fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.218482 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c1631122dcd4c738fcb59926b7873fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.225946 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.230751 4162 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.236996 4162 tablet_bootstrap.cc:654] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.244478 4167 ts_tablet_manager.cc:1918] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:18.244792 4167 log.cc:1196] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:18.245517 4167 ts_tablet_manager.cc:1939] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.246374 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 243) took 3803 ms. Trace:
I20260811 18:57:18.248509 4167 rpcz_store.cc:276] 0811 18:57:14.442751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.442818 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:18.188709 (+3745891us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:18.188723 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.231670 (+ 42947us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.244295 (+ 12625us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.246305 (+ 2010us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3745275}
I20260811 18:57:18.249394 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b1ce1042ae974f4dbac964bd7ea16a94 (table default.Table_8c56b0614a7843bea868d1e3fd95de68 [id=550b42f2eb864745ad5e20c1c2aef171]) successfully deleted
I20260811 18:57:18.250067 4167 tablet_replica.cc:330] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.250497 4167 raft_consensus.cc:2229] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.251147 4167 raft_consensus.cc:2258] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.252122 4162 tablet_bootstrap.cc:492] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.252434 4162 ts_tablet_manager.cc:1397] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260811 18:57:18.253576 4162 raft_consensus.cc:348] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.253968 4162 raft_consensus.cc:374] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.254149 4162 raft_consensus.cc:729] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.254599 4162 consensus_queue.cc:260] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.255684 4162 ts_tablet_manager.cc:1428] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:18.256786 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eb8f5b87062349c4b19ec99fa0175f24
I20260811 18:57:18.257472 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b071ee4d3e847099b98398e52574f8e" } new_table_name: "default.Table_3294569c49f84a2291010174baee10af" modify_external_catalogs: true
I20260811 18:57:18.258460 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cce97bfdb1d41e685e091c8c5cc6946 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.259224 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.259500 4162 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:18.259820 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.264590 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.264971 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae200f3814e431a8288c30fe94bffcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.266274 4167 ts_tablet_manager.cc:1905] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.271451 4836 raft_consensus.cc:482] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.271720 4836 raft_consensus.cc:504] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.272640 4836 leader_election.cc:288] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:18.274482 3676 leader_election.cc:341] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:18.279433 4162 tablet_bootstrap.cc:654] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.224073 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.290581 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_655dd0bc17cc45d3a344bb5c02ff7f80" } new_table_name: "default.Table_d7a06761b8a145519b7bb645e7a73657" modify_external_catalogs: true
I20260811 18:57:18.293627 4163 tablet_bootstrap.cc:654] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.297685 4167 ts_tablet_manager.cc:1918] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.299079 4163 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.299118 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 866ed614511649459b22a9fce55220b2 consensus state has no leader
I20260811 18:57:18.299449 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 62c365f79d4a4201ac323f16bddf3e62 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.300606 4163 ts_tablet_manager.cc:1397] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.001s
I20260811 18:57:18.303570 4163 raft_consensus.cc:348] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.305039 4163 raft_consensus.cc:374] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.304862 4162 tablet_bootstrap.cc:492] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.305424 4162 ts_tablet_manager.cc:1397] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.000s
I20260811 18:57:18.306609 4162 raft_consensus.cc:348] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.307067 4162 raft_consensus.cc:374] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.307226 4162 raft_consensus.cc:729] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.307597 4162 consensus_queue.cc:260] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.310904 4162 ts_tablet_manager.cc:1428] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260811 18:57:18.311223 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a994b3f4d7d4c92946ad8d595e6ad2b has no consensus state
I20260811 18:57:18.311421 4162 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.313988 4162 tablet_bootstrap.cc:654] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.317000 4167 log.cc:1196] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/151df87806694f46be5013592c68c01a
I20260811 18:57:18.263348 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.276921 4811 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.317469 4811 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.318225 4811 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.254122 4169 ts_tablet_manager.cc:1918] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.319798 4169 log.cc:1196] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a30bb384136f4a169a5751655437d5a3
I20260811 18:57:18.324113 4169 ts_tablet_manager.cc:1939] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.319074 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.325189 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1996443a5de84ded8919fb5c51dc6554" } new_table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" modify_external_catalogs: true
I20260811 18:57:18.325757 4167 ts_tablet_manager.cc:1939] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.326331 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 253) took 3555 ms. Trace:
I20260811 18:57:18.329249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 309) took 2262 ms. Trace:
W20260811 18:57:18.324985 3824 leader_election.cc:341] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.324286 4163 raft_consensus.cc:729] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.331158 4163 consensus_queue.cc:260] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.332197 4163 ts_tablet_manager.cc:1428] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.029s user 0.001s sys 0.001s
I20260811 18:57:18.332403 4169 rpcz_store.cc:276] 0811 18:57:16.066582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.066985 (+ 403us) service_pool.cc:227] Handling call
0811 18:57:18.118969 (+2051984us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:18.118980 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.196966 (+ 77986us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.253979 (+ 57013us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.329165 (+ 75186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1980251}
I20260811 18:57:18.337167 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_460a6647324c49b78cd79e59e974732b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.337883 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_460a6647324c49b78cd79e59e974732b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.326467 4167 rpcz_store.cc:276] 0811 18:57:14.770962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.773526 (+ 2564us) service_pool.cc:227] Handling call
0811 18:57:18.249976 (+3476450us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:18.249992 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.282999 (+ 33007us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.297530 (+ 14531us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.326275 (+ 28745us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3447994}
I20260811 18:57:18.339077 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 254) took 3492 ms. Trace:
I20260811 18:57:18.338392 4811 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.343849 4167 rpcz_store.cc:276] 0811 18:57:14.846824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.847111 (+ 287us) service_pool.cc:227] Handling call
0811 18:57:18.338807 (+3491696us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:18.338828 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.339002 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3471806}
I20260811 18:57:18.344732 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 255) took 3359 ms. Trace:
I20260811 18:57:18.344911 4167 rpcz_store.cc:276] 0811 18:57:14.985433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:14.985496 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:18.344523 (+3359027us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:18.344537 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.344669 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3358452}
I20260811 18:57:18.345329 4811 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.345588 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a30bb384136f4a169a5751655437d5a3 (table default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]) successfully deleted
I20260811 18:57:18.348589 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 151df87806694f46be5013592c68c01a (table default.Table_fd862d7d75f24a05aa54d761e710cd50 [id=c050b996a88945629835909bca551fea]) successfully deleted
W20260811 18:57:18.348901 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4687b95728fc485fa0e842bc447eb83f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4687b95728fc485fa0e842bc447eb83f already in progress: creating tablet
I20260811 18:57:18.352317 4163 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.346956 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.348222 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a300a7aca2814da6b999eb4d865fea66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.355731 4811 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.355922 4811 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.356240 4811 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:18.357203 3824 leader_election.cc:341] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.358134 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:18.358937 3824 leader_election.cc:341] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.359243 4811 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.359439 4169 tablet_replica.cc:330] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.359834 4169 raft_consensus.cc:2229] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.360222 4169 raft_consensus.cc:2258] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.361132 4169 ts_tablet_manager.cc:1905] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.361928 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 256) took 3161 ms. Trace:
W20260811 18:57:18.361927 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ff1be026ca404eada53f60f257ea367c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff1be026ca404eada53f60f257ea367c already in progress: creating tablet
I20260811 18:57:18.364181 4167 rpcz_store.cc:276] 0811 18:57:15.200923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.200985 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:18.361687 (+3160702us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:18.361707 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.361855 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3153180}
I20260811 18:57:18.365846 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.370777 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9e896cd16f14d14800b4a3f697d2ad4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:18.388897 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fbba1923ce0049448fe4e28f2dd258dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbba1923ce0049448fe4e28f2dd258dc already in progress: creating tablet
I20260811 18:57:18.389110 4167 tablet_replica.cc:330] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.389436 4167 raft_consensus.cc:2229] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.389720 4167 raft_consensus.cc:2258] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.390517 4167 ts_tablet_manager.cc:1905] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.393463 4163 tablet_bootstrap.cc:654] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.402908 4162 tablet_bootstrap.cc:492] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.414955 4811 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.415158 4811 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.415870 4811 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:18.415964 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 has no consensus state
I20260811 18:57:18.416006 4811 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.416160 4811 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.416656 4811 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.418043 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.418632 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.419354 4163 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:18.419464 3824 leader_election.cc:341] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.419670 4163 ts_tablet_manager.cc:1397] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.083s user 0.003s sys 0.003s
W20260811 18:57:18.419823 3824 leader_election.cc:341] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.420197 4162 ts_tablet_manager.cc:1397] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.109s user 0.006s sys 0.000s
I20260811 18:57:18.420768 4163 raft_consensus.cc:348] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.427528 4162 raft_consensus.cc:348] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.427956 4162 raft_consensus.cc:374] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.428081 4162 raft_consensus.cc:729] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.428134 4163 raft_consensus.cc:374] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.428406 4163 raft_consensus.cc:729] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.428787 4163 consensus_queue.cc:260] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.430799 4873 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.431241 4163 ts_tablet_manager.cc:1428] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260811 18:57:18.434026 4873 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:18.434849 4163 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.439610 4163 tablet_bootstrap.cc:654] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.446384 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.447527 4162 consensus_queue.cc:260] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.456595 3709 tablet_service.cc:1467] Processing CreateTablet for tablet ae7da0868b114052ba421801b76fb33c (DEFAULT_TABLE table=default.Table_e2b67d7e59584a2bb877f79bdafed3a1 [id=c417185e41c94dad8cc5e5f5937adb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.456804 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (DEFAULT_TABLE table=default.Table_a8d2d56f43544d68adbbd2848fc00dde [id=9c62165c055f4acea600e53873f88772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.457106 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7da0868b114052ba421801b76fb33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.457937 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
#2 0x7fc7bfa1f90d in std::unique_lock<kudu::rw_semaphore>::unlock() ../../../include/c++/8/bits/std_mutex.h:323:17
I20260811 18:57:18.462456 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.462994 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1e244addf5462499fcadaa3332cba1 (DEFAULT_TABLE table=default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.463568 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1e244addf5462499fcadaa3332cba1. 1 dirs total, 0 dirs full, 0 dirs failed
#3 0x7fc7bfa1f873 in std::unique_lock<kudu::rw_semaphore>::~unique_lock() ../../../include/c++/8/bits/std_mutex.h:232:4
#4 0x7fc7bfa14063 in kudu::tablet::AlterSchemaOpState::~AlterSchemaOpState() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:47:3
I20260811 18:57:18.472391 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d (DEFAULT_TABLE table=default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.473073 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0540e4e43ea34fbfbc66b6035c60684d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.474762 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 114397bea09444479df113552c22e154 (DEFAULT_TABLE table=default.Table_ba1e7c81d1bf4209b7089639928742f1 [id=cfba5a388f8f406d8b2a9ae1fc397585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.475261 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114397bea09444479df113552c22e154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.460816 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet d00d943b778741b4aeec076ba46a9c40 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.478448 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (DEFAULT_TABLE table=default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.479125 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b2ca385374abeb709fccbdf6fa0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.482776 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd (DEFAULT_TABLE table=default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.483397 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4059e5907b1e488ab9e1d04e563f96cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.491619 3723 tablet_service.cc:1467] Processing CreateTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 (DEFAULT_TABLE table=default.Table_24955fa6843f46dc883be60173049a16 [id=c5663aaa82454e73a920034172f2a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.492022 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c4738e833404ad7b1947bb598f05842" } modify_external_catalogs: true
I20260811 18:57:18.492380 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c4738e833404ad7b1947bb598f05842" } modify_external_catalogs: true
I20260811 18:57:18.492666 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.495688 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet c99ceca57cb245f4a3a6cb4b36f1306d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.495966 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b203df4124364737839bf4c1ae072988 (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.496462 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b203df4124364737839bf4c1ae072988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.497618 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" } modify_external_catalogs: true
I20260811 18:57:18.497871 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_523277d801974e708b0f7e9c401bb36a" } modify_external_catalogs: true
I20260811 18:57:18.507228 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_459873f5dbe2412c80b73ca35dce90ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.508004 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_459873f5dbe2412c80b73ca35dce90ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.455566 4162 ts_tablet_manager.cc:1428] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.029s user 0.003s sys 0.000s
I20260811 18:57:18.521431 4169 ts_tablet_manager.cc:1918] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:18.521569 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:18.525424 4811 raft_consensus.cc:482] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.525673 4811 raft_consensus.cc:504] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#5 0x7fc7bfab3cb2 in kudu::tablet::AlterSchemaOpState::~AlterSchemaOpState() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:46:25
I20260811 18:57:18.527040 4811 leader_election.cc:288] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.527339 4811 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:18.527613 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 10311ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
W20260811 18:57:18.528066 3824 leader_election.cc:341] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.527832 4811 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.530330 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 consensus state has no leader
I20260811 18:57:18.531165 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" } modify_external_catalogs: true
I20260811 18:57:18.531589 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bf28eaf7393d4d508363fbc8dd91686b" } modify_external_catalogs: true
W20260811 18:57:18.531718 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 81044.4us
I20260811 18:57:18.533528 4873 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.533833 4873 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.534233 4873 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.534961 4873 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.535987 4811 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.536703 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a" } modify_external_catalogs: true
I20260811 18:57:18.537536 3870 tablet_service.cc:1467] Processing CreateTablet for tablet ae7da0868b114052ba421801b76fb33c (DEFAULT_TABLE table=default.Table_e2b67d7e59584a2bb877f79bdafed3a1 [id=c417185e41c94dad8cc5e5f5937adb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.538113 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae7da0868b114052ba421801b76fb33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.539124 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (DEFAULT_TABLE table=default.Table_a8d2d56f43544d68adbbd2848fc00dde [id=9c62165c055f4acea600e53873f88772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.539528 4163 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.539916 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a80fc1f336804bea9495ec0bce4bd47a" } modify_external_catalogs: true
I20260811 18:57:18.542541 4169 log.cc:1196] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:18.542249 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a55a42d1e64f04ad7f433614a3a1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.544478 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.544632 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.544768 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 114397bea09444479df113552c22e154 (DEFAULT_TABLE table=default.Table_ba1e7c81d1bf4209b7089639928742f1 [id=cfba5a388f8f406d8b2a9ae1fc397585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.552103 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.552510 4163 ts_tablet_manager.cc:1397] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.105s user 0.006s sys 0.021s
I20260811 18:57:18.554267 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.554440 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.554585 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1e244addf5462499fcadaa3332cba1 (DEFAULT_TABLE table=default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.559940 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114397bea09444479df113552c22e154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.567266 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d (DEFAULT_TABLE table=default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.569414 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f287040b879642448367abe2b1c73cd9 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.565476 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd (DEFAULT_TABLE table=default.Table_4efc43ae63254c67aac88d7bd9b6c6a2 [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.570201 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (DEFAULT_TABLE table=default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.571931 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1e244addf5462499fcadaa3332cba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.570428 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 8426ccbe530446fcbaa162a03cbb1892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:18.576421 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48359.5us
W20260811 18:57:18.579263 3678 leader_election.cc:334] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.370s (SENT)
I20260811 18:57:18.579573 3678 leader_election.cc:302] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:18.580960 4169 ts_tablet_manager.cc:1939] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:18.581364 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f287040b879642448367abe2b1c73cd9 (DEFAULT_TABLE table=default.Table_00f2f8bec82f44f3a42420818d7dc6a0 [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.581665 4162 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.581679 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 312) took 2504 ms. Trace:
I20260811 18:57:18.583541 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0540e4e43ea34fbfbc66b6035c60684d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.584934 3863 tablet_service.cc:1467] Processing CreateTablet for tablet b203df4124364737839bf4c1ae072988 (DEFAULT_TABLE table=default.Table_cdd3a653fc1b405cbfd89c5d98728e45 [id=f7fa32f5e00247d789399dcebea75999]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.585078 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4059e5907b1e488ab9e1d04e563f96cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.581923 4163 raft_consensus.cc:348] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.589954 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a0c48e6299546a2921a7a7db03f1032"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.590917 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a0c48e6299546a2921a7a7db03f1032 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.591106 3872 tablet_service.cc:1467] Processing CreateTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 (DEFAULT_TABLE table=default.Table_24955fa6843f46dc883be60173049a16 [id=c5663aaa82454e73a920034172f2a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.596182 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b203df4124364737839bf4c1ae072988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.596837 4163 raft_consensus.cc:374] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.597133 4169 rpcz_store.cc:276] 0811 18:57:16.077159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.077396 (+ 237us) service_pool.cc:227] Handling call
0811 18:57:18.343737 (+2266341us) ts_tablet_manager.cc:1145] Deleting tablet 0e1efffade0f486d8d627e56e0be6ff6
0811 18:57:18.343751 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.405050 (+ 61299us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.442153 (+ 37103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.581506 (+139353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2199216}
I20260811 18:57:18.597265 4836 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.598573 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.598869 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:18.599550 3553 catalog_manager.cc:6565] ReplaceTablet: tablet 62c365f79d4a4201ac323f16bddf3e62 deleted and replaced by tablet 4093564a88264ed59c945523c9ddc9b3
I20260811 18:57:18.599690 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f287040b879642448367abe2b1c73cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.600567 3554 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c41131a66fdf4c178644890cff824752
I20260811 18:57:18.600899 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b4de110bd3348f0b0dae21c9f2b6a19 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.601001 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f287040b879642448367abe2b1c73cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.601388 4163 raft_consensus.cc:729] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.601469 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:18.601845 4169 tablet_replica.cc:330] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:18.601979 3547 catalog_manager.cc:6565] ReplaceTablet: tablet c99ceca57cb245f4a3a6cb4b36f1306d deleted and replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b
I20260811 18:57:18.604775 4836 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.605844 4836 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.616293 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b2ca385374abeb709fccbdf6fa0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.616948 4163 consensus_queue.cc:260] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:18.612833 3678 raft_consensus.cc:2670] T 8efb663060ef457cbece962fce701eb1 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:18.618117 4163 ts_tablet_manager.cc:1428] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.065s user 0.004s sys 0.000s
I20260811 18:57:18.620610 4163 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:18.622628 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 45784.2us
I20260811 18:57:18.625051 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8eb041f3d704762868a048faec0e0fc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.626040 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.634853 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.638154 4162 tablet_bootstrap.cc:654] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.638326 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0e1efffade0f486d8d627e56e0be6ff6 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
I20260811 18:57:18.640241 4169 raft_consensus.cc:2229] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.645839 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.650442 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6bf41429faeb4785ad747d207e4641f8" } new_table_name: "default.Table_eff26ab40dd947859e4f83ff55b793d4" modify_external_catalogs: true
I20260811 18:57:18.653733 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a93ee06ed76342e082697ded810c4c99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.654800 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:18.654924 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a93ee06ed76342e082697ded810c4c99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.657825 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b" } modify_external_catalogs: true
I20260811 18:57:18.658298 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2058b5fb0d08491a9f2fa354276fc21b" } modify_external_catalogs: true
I20260811 18:57:18.659364 4167 ts_tablet_manager.cc:1918] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:18.660079 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.660952 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c1631122dcd4c738fcb59926b7873fc" } new_table_name: "default.Table_e6a28da9c6fa458d9daa95fdbd30920c" modify_external_catalogs: true
I20260811 18:57:18.661505 4167 log.cc:1196] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/80cf2375b4c64dbfae4e20003545da60
I20260811 18:57:18.662837 4167 ts_tablet_manager.cc:1939] T 80cf2375b4c64dbfae4e20003545da60 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.664314 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 268) took 2912 ms. Trace:
I20260811 18:57:18.667500 4167 rpcz_store.cc:276] 0811 18:57:15.751514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.751728 (+ 214us) service_pool.cc:227] Handling call
0811 18:57:18.388974 (+2637246us) ts_tablet_manager.cc:1145] Deleting tablet 80cf2375b4c64dbfae4e20003545da60
0811 18:57:18.388995 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.574334 (+185339us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.658873 (+ 84539us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.664251 (+ 5378us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2609983}
I20260811 18:57:18.668161 4167 tablet_replica.cc:330] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.668439 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 80cf2375b4c64dbfae4e20003545da60 (table default.Table_174aeae5133245c3ba894064a8d08dd6 [id=050584e79512491ab8829d8ac3e519c7]) successfully deleted
I20260811 18:57:18.673262 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.674924 4169 raft_consensus.cc:2258] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.679265 4167 raft_consensus.cc:2229] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.679437 4162 tablet_bootstrap.cc:492] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:18.682511 3824 leader_election.cc:341] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:18.683153 3824 leader_election.cc:341] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.684267 4162 ts_tablet_manager.cc:1397] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.108s user 0.017s sys 0.000s
W20260811 18:57:18.685386 3824 leader_election.cc:341] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.686444 4162 raft_consensus.cc:348] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.687206 4162 raft_consensus.cc:374] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.687420 4162 raft_consensus.cc:729] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.688145 4162 consensus_queue.cc:260] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.690297 4169 ts_tablet_manager.cc:1905] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.696566 4162 ts_tablet_manager.cc:1428] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.012s user 0.000s sys 0.004s
I20260811 18:57:18.698390 4162 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.700106 4163 tablet_bootstrap.cc:654] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.700568 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:18.704296 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.706924 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:18.707239 3888 raft_consensus.cc:2454] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:18.707970 4162 tablet_bootstrap.cc:654] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.715167 4167 pending_rounds.cc:62] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:18.715397 4167 pending_rounds.cc:68] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400105548996608 op_type: NO_OP noop_request { }
I20260811 18:57:18.715610 4167 raft_consensus.cc:2866] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:18.715799 4167 raft_consensus.cc:2258] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.716709 4167 ts_tablet_manager.cc:1905] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.733115 4162 tablet_bootstrap.cc:492] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.733368 4162 ts_tablet_manager.cc:1397] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260811 18:57:18.734704 4162 raft_consensus.cc:348] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.735117 4162 raft_consensus.cc:374] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.735266 4162 raft_consensus.cc:729] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.735638 4162 consensus_queue.cc:260] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.738811 4836 consensus_queue.cc:1035] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:18.740077 3676 consensus_peers.cc:487] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d6daff4b7540309ff58755758e9752. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.740566 3676 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:18.740900 4836 raft_consensus.cc:2781] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:18.741057 4836 raft_consensus.cc:482] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:18.741159 4836 raft_consensus.cc:3037] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.743978 4162 ts_tablet_manager.cc:1428] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260811 18:57:18.744951 4836 raft_consensus.cc:504] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.746186 4836 leader_election.cc:288] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:18.748098 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_d7a06761b8a145519b7bb645e7a73657 [id=f51f38f3582f44db99007a265b665fd4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:18.751127 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.754065 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet e215759d48c24d8cb99bbc3fc9009a66 with delete_type TABLET_DATA_DELETED (replaced by tablet c08436f33db043b1bc6395cd92102e59) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.753607 4162 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.758466 4729 raft_consensus.cc:482] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.758708 4729 raft_consensus.cc:504] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.759444 4729 leader_election.cc:288] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.761664 4162 tablet_bootstrap.cc:654] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.768472 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:18.769016 3549 catalog_manager.cc:6565] ReplaceTablet: tablet d00d943b778741b4aeec076ba46a9c40 deleted and replaced by tablet ccc63b46a4e346f79d1e03a82490794c
I20260811 18:57:18.770617 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.775532 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.775663 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fdae8856d30a46259e91345b60181c2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.776585 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdae8856d30a46259e91345b60181c2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.780112 4167 ts_tablet_manager.cc:1918] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:18.780325 4167 log.cc:1196] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/508f0b39cd654aa991a2928a4b067762
I20260811 18:57:18.784185 4167 ts_tablet_manager.cc:1939] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.784718 4836 raft_consensus.cc:482] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.784929 4836 raft_consensus.cc:504] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.786584 4836 leader_election.cc:288] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.787133 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a443fe209556489abb47f4faa5d696e5
I20260811 18:57:18.788350 4162 tablet_bootstrap.cc:492] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.788548 4162 ts_tablet_manager.cc:1397] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.004s
I20260811 18:57:18.788784 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.789446 4162 raft_consensus.cc:348] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.789729 4162 raft_consensus.cc:374] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.789806 4162 raft_consensus.cc:729] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.790048 4162 consensus_queue.cc:260] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.790736 4162 ts_tablet_manager.cc:1428] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:57:18.791313 4162 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.793320 4162 tablet_bootstrap.cc:654] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.793757 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea0cc7a576d64d1c934364316cae3c70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.794492 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea0cc7a576d64d1c934364316cae3c70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.796490 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cd4eb198addc42f2b6d860ef805c589f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.797047 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd4eb198addc42f2b6d860ef805c589f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.798214 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.802116 4162 tablet_bootstrap.cc:492] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.802330 4162 ts_tablet_manager.cc:1397] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260811 18:57:18.788267 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 269) took 3026 ms. Trace:
I20260811 18:57:18.803227 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a55a42d1e64f04ad7f433614a3a1aa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:18.803522 3891 raft_consensus.cc:3037] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.803390 4162 raft_consensus.cc:348] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.803674 4162 raft_consensus.cc:374] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.803812 4162 raft_consensus.cc:729] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.804034 4162 consensus_queue.cc:260] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.807471 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.807706 4162 ts_tablet_manager.cc:1428] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:18.809289 4162 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.811378 4162 tablet_bootstrap.cc:654] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.815207 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c17847a358e544b4b4e51f65463a9679
I20260811 18:57:18.815478 4836 raft_consensus.cc:482] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.815723 4836 raft_consensus.cc:504] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.816637 4836 leader_election.cc:288] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.817271 4162 tablet_bootstrap.cc:492] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.817459 4162 ts_tablet_manager.cc:1397] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260811 18:57:18.818387 4162 raft_consensus.cc:348] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.818691 4162 raft_consensus.cc:374] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.818778 4162 raft_consensus.cc:729] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.821239 4162 consensus_queue.cc:260] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.822000 4162 ts_tablet_manager.cc:1428] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260811 18:57:18.822206 3676 consensus_peers.cc:487] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:18.822427 4162 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:18.822635 3676 leader_election.cc:341] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 0e1efffade0f486d8d627e56e0be6ff6
W20260811 18:57:18.823030 3676 leader_election.cc:341] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 5ae200f3814e431a8288c30fe94bffcf
W20260811 18:57:18.823429 3676 consensus_peers.cc:487] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8426ccbe530446fcbaa162a03cbb1892. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:18.824383 3676 leader_election.cc:341] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a30bb384136f4a169a5751655437d5a3
W20260811 18:57:18.827483 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bfb54175c834517b7af291c942333cd consensus state has no leader
I20260811 18:57:18.828184 4162 tablet_bootstrap.cc:654] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:18.828405 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e0747ef88574c68ae074732cb7095b9 has no consensus state
I20260811 18:57:18.830037 3891 raft_consensus.cc:2454] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:18.833000 4169 ts_tablet_manager.cc:1918] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:18.833196 4169 log.cc:1196] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a81334e7da3741a9b24f8978312181b9
I20260811 18:57:18.803264 4167 rpcz_store.cc:276] 0811 18:57:15.761398 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.761966 (+ 568us) service_pool.cc:227] Handling call
0811 18:57:18.668051 (+2906085us) ts_tablet_manager.cc:1145] Deleting tablet 508f0b39cd654aa991a2928a4b067762
0811 18:57:18.668065 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.731296 (+ 63231us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.779910 (+ 48614us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.788214 (+ 8304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2860772}
I20260811 18:57:18.833299 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cafb4240a8584ec184a99a633b46eeb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.833686 4169 ts_tablet_manager.cc:1939] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:18.833997 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cafb4240a8584ec184a99a633b46eeb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.834059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 315) took 2407 ms. Trace:
I20260811 18:57:18.835347 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 508f0b39cd654aa991a2928a4b067762 (table default.Table_1f27ea0d4f084ac081cca10255b06c11 [id=2f2a76641ad54c32a942c854342bfb11]) successfully deleted
I20260811 18:57:18.835791 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51c66177bc494cd0a293e2959ff854e5" } new_table_name: "default.Table_b3ee7e916a0c4d2180027e84bc055f54" modify_external_catalogs: true
I20260811 18:57:18.837062 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e746326e8bff404aa5b3bc1fac099489" } new_table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" modify_external_catalogs: true
I20260811 18:57:18.838871 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e12ffa777d684216bfc19d52f2dad4be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.839435 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e12ffa777d684216bfc19d52f2dad4be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.839572 4163 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:18.839742 4163 ts_tablet_manager.cc:1397] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.221s user 0.003s sys 0.007s
I20260811 18:57:18.840553 4163 raft_consensus.cc:348] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.840816 4163 raft_consensus.cc:374] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.840935 4163 raft_consensus.cc:729] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.841137 4167 tablet_replica.cc:330] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:18.841274 4163 consensus_queue.cc:260] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.841562 4167 raft_consensus.cc:2229] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.841861 4167 raft_consensus.cc:2258] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.842094 4163 ts_tablet_manager.cc:1428] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 18:57:18.842585 4163 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:18.842834 4167 ts_tablet_manager.cc:1905] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.841189 4162 tablet_bootstrap.cc:492] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.844800 4162 ts_tablet_manager.cc:1397] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.001s
I20260811 18:57:18.846065 4162 raft_consensus.cc:348] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.846590 4162 raft_consensus.cc:374] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.846721 4162 raft_consensus.cc:729] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.847083 4162 consensus_queue.cc:260] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.850492 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_786c886ed1164646855f804b8397bed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.851224 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_786c886ed1164646855f804b8397bed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.854278 4836 raft_consensus.cc:482] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.854518 4836 raft_consensus.cc:504] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.855386 4836 leader_election.cc:288] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.858248 4162 ts_tablet_manager.cc:1428] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:18.860041 4162 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.865285 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9c6377231db8495796c02e71f7f71991" } new_table_name: "default.Table_4d1fcb681d504d57b9d288e08be657c2" modify_external_catalogs: true
I20260811 18:57:18.871320 3676 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:18.871716 4836 raft_consensus.cc:2781] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:18.871963 4836 raft_consensus.cc:686] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:18.872354 4836 consensus_queue.cc:237] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.874595 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.875335 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.876334 4162 tablet_bootstrap.cc:654] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.885305 4167 ts_tablet_manager.cc:1918] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:18.888446 3676 leader_election.cc:341] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: cf285cdf5235451bb1531cb294017101
I20260811 18:57:18.901913 4162 tablet_bootstrap.cc:492] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:18.902988 4162 ts_tablet_manager.cc:1397] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.004s
I20260811 18:57:18.904223 4162 raft_consensus.cc:348] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.904804 4162 raft_consensus.cc:374] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.904947 4162 raft_consensus.cc:729] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.905308 4162 consensus_queue.cc:260] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.906167 4162 ts_tablet_manager.cc:1428] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:18.906738 4162 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:18.909170 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" } modify_external_catalogs: true
I20260811 18:57:18.909438 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4c14312f53aa4eb892770f9f6ef4bdb9" } modify_external_catalogs: true
I20260811 18:57:18.910676 4162 tablet_bootstrap.cc:654] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.911484 4878 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.911692 4878 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.911966 4167 log.cc:1196] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2f9fa42654cf4ec18220e5fb173ddef7
I20260811 18:57:18.912518 4167 ts_tablet_manager.cc:1939] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:18.912704 4878 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:18.912765 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:18.913239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 271) took 3115 ms. Trace:
I20260811 18:57:18.913631 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 3744b3d608754de68c4c57b072d1dc15 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:18.913944 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:18.913378 4167 rpcz_store.cc:276] 0811 18:57:15.797696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.799952 (+ 2256us) service_pool.cc:227] Handling call
0811 18:57:18.833809 (+3033857us) ts_tablet_manager.cc:1145] Deleting tablet 2f9fa42654cf4ec18220e5fb173ddef7
0811 18:57:18.833832 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.880755 (+ 46923us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.885174 (+ 4419us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.913161 (+ 27987us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2996941}
W20260811 18:57:18.915128 3824 leader_election.cc:341] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:18.915853 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 276) took 3040 ms. Trace:
I20260811 18:57:18.916008 4167 rpcz_store.cc:276] 0811 18:57:15.875490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.877707 (+ 2217us) service_pool.cc:227] Handling call
0811 18:57:18.915544 (+3037837us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:18.915562 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.915764 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3037337}
I20260811 18:57:18.917372 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2f9fa42654cf4ec18220e5fb173ddef7 (table default.Table_2e54e73ee78a4aeca60980925f339f11 [id=04e3deb0d47446198b908bf0c3c07b14]) successfully deleted
I20260811 18:57:18.919209 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:18.925678 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 135a8c34b93544d98383fc3103966a54 (DEFAULT_TABLE table=default.Table_7bfc22ccd3e44c679f5e951318bb0af3 [id=e07603a929ca4721b5b978a215d52fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.926368 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135a8c34b93544d98383fc3103966a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.930511 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:18.930963 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:18.931169 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9722572533de436cb9149e6230570382 (DEFAULT_TABLE table=default.Table_3e695533c42647128860db601dcc0b5f [id=d1ff7e5df21346dc9c4f609331322f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931172 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 930b6e8839e24ec9a69854349bc3ecf8 (DEFAULT_TABLE table=default.Table_3294569c49f84a2291010174baee10af [id=d7d11af2eee54ab880a15c525b7ad0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931352 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 (DEFAULT_TABLE table=default.Table_be30e8194fd44874a1fdd996128eb4d4 [id=bd75749ddd8d4224a75817f8d95f59d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931749 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b6e8839e24ec9a69854349bc3ecf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.931816 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 (DEFAULT_TABLE table=default.Table_be30e8194fd44874a1fdd996128eb4d4 [id=bd75749ddd8d4224a75817f8d95f59d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.931907 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d81e93739f44d9e99b6ebc647e1d0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.932164 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d81e93739f44d9e99b6ebc647e1d0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.933574 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_704e71bf00da49609103fb3fde6afb2c" } modify_external_catalogs: true
I20260811 18:57:18.933818 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_704e71bf00da49609103fb3fde6afb2c" } modify_external_catalogs: true
I20260811 18:57:18.934617 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9722572533de436cb9149e6230570382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.934707 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 930b6e8839e24ec9a69854349bc3ecf8 (DEFAULT_TABLE table=default.Table_3294569c49f84a2291010174baee10af [id=d7d11af2eee54ab880a15c525b7ad0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.935177 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930b6e8839e24ec9a69854349bc3ecf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.935537 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d9421bc220774f66b90bf537a47f1e13 with delete_type TABLET_DATA_DELETED (replaced by tablet d19675f757674450aceb5942e1a51c99) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:18.935812 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:18.938383 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:18.941054 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 9722572533de436cb9149e6230570382 (DEFAULT_TABLE table=default.Table_3e695533c42647128860db601dcc0b5f [id=d1ff7e5df21346dc9c4f609331322f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.941684 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9722572533de436cb9149e6230570382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.946482 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 135a8c34b93544d98383fc3103966a54 (DEFAULT_TABLE table=default.Table_7bfc22ccd3e44c679f5e951318bb0af3 [id=e07603a929ca4721b5b978a215d52fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.947064 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135a8c34b93544d98383fc3103966a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.947840 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 (DEFAULT_TABLE table=default.Table_0b53eaa03dd34a22a14fa65da3866fe4 [id=45f3f13a059e4e2e8624e5530abca5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.948282 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc6cce2c19746219b8b0fe54b13dcd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.951049 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 (DEFAULT_TABLE table=default.Table_0b53eaa03dd34a22a14fa65da3866fe4 [id=45f3f13a059e4e2e8624e5530abca5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.951529 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc6cce2c19746219b8b0fe54b13dcd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.952986 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 (DEFAULT_TABLE table=default.Table_914c8b90fadf48e88b75d13897462d84 [id=cf8fdaf8a86f4648880083c2ccd856d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.953912 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0e0d139c924f1dafe42da0bedada72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.955392 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a (DEFAULT_TABLE table=default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 [id=dc139df6f9814c7891ddbd2d85d16e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.955894 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54fa32f8fa1144c4bd9dcd00a550001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.959128 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a (DEFAULT_TABLE table=default.Table_ffad2d401c8f42f3b7a65afbb71b7c13 [id=dc139df6f9814c7891ddbd2d85d16e01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.959580 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54fa32f8fa1144c4bd9dcd00a550001a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.960742 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0747ef88574c68ae074732cb7095b9 (DEFAULT_TABLE table=default.Table_e6a28da9c6fa458d9daa95fdbd30920c [id=a2e4dc5b6af440d7bdf3d7b9dfffc9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.961301 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0747ef88574c68ae074732cb7095b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:18.965018 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b404e99e9414c67b3cd5aff12203a53 consensus state has no leader
I20260811 18:57:18.967267 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 (DEFAULT_TABLE table=default.Table_914c8b90fadf48e88b75d13897462d84 [id=cf8fdaf8a86f4648880083c2ccd856d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.967857 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0e0d139c924f1dafe42da0bedada72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.976145 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2be6762e326448d7b677a5641e2b3103 (DEFAULT_TABLE table=default.Table_01f64f05d1ee44029680c5423fd0d81b [id=9fd91de4ece84d0d8ecf793ca643843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.976787 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be6762e326448d7b677a5641e2b3103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.977731 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:18.979178 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 2be6762e326448d7b677a5641e2b3103 (DEFAULT_TABLE table=default.Table_01f64f05d1ee44029680c5423fd0d81b [id=9fd91de4ece84d0d8ecf793ca643843a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.979772 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be6762e326448d7b677a5641e2b3103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.981065 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e (DEFAULT_TABLE table=default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f [id=c4fcc501bfd9482da8e0e24cae8380ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.981557 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.982908 4729 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.983134 4729 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:18.991577 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_655b56a4862144f1afd52a620c3715d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.992264 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e (DEFAULT_TABLE table=default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f [id=c4fcc501bfd9482da8e0e24cae8380ee]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:18.992352 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_655b56a4862144f1afd52a620c3715d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.992892 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.994258 4729 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:18.969805 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0747ef88574c68ae074732cb7095b9 (DEFAULT_TABLE table=default.Table_e6a28da9c6fa458d9daa95fdbd30920c [id=a2e4dc5b6af440d7bdf3d7b9dfffc9cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.995002 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd" } modify_external_catalogs: true
I20260811 18:57:18.995268 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b48fafe85a504185911a1e7edb4fcbfd" } modify_external_catalogs: true
I20260811 18:57:18.995821 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0747ef88574c68ae074732cb7095b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.996307 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.009454 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.010228 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:19.014598 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c29af3615e74722a809d3e5ede9b9a1 consensus state has no leader
I20260811 18:57:19.016919 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_278e0a9db8364b7cb00a0920de4b0561"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.017637 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_278e0a9db8364b7cb00a0920de4b0561 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.021121 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4" } modify_external_catalogs: true
I20260811 18:57:19.021400 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9e896cd16f14d14800b4a3f697d2ad4" } modify_external_catalogs: true
I20260811 18:57:19.028844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 277) took 3153 ms. Trace:
I20260811 18:57:19.029131 4167 rpcz_store.cc:276] 0811 18:57:15.875756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878248 (+ 2492us) service_pool.cc:227] Handling call
0811 18:57:18.990795 (+3112547us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:18.990814 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.028760 (+ 37946us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112287}
W20260811 18:57:19.029794 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 already in progress: creating tablet
I20260811 18:57:19.030038 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 278) took 3154 ms. Trace:
I20260811 18:57:19.030170 4167 rpcz_store.cc:276] 0811 18:57:15.875988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878531 (+ 2543us) service_pool.cc:227] Handling call
0811 18:57:19.029797 (+3151266us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:19.029813 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.029963 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3150981}
I20260811 18:57:19.030828 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 279) took 3154 ms. Trace:
I20260811 18:57:19.030942 4878 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.030948 4167 rpcz_store.cc:276] 0811 18:57:15.876245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.878871 (+ 2626us) service_pool.cc:227] Handling call
0811 18:57:19.030652 (+3151781us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:19.030664 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.030760 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3151592}
W20260811 18:57:19.031206 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45aff3e8cb0c4f7191e4522c6ef7d156 already in progress: creating tablet
I20260811 18:57:19.031195 4878 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.031467 4167 tablet_replica.cc:330] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:19.031527 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 36724a83bf9844b9a27a88ca36a9bf50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36724a83bf9844b9a27a88ca36a9bf50 already in progress: creating tablet
I20260811 18:57:19.031858 4167 raft_consensus.cc:2229] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.031980 4878 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.032291 4167 raft_consensus.cc:2258] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.032586 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4fbcb57a32f54fc3a493bc5412e94051" } new_table_name: "default.Table_69803efc857941408d32f7da625acfb6" modify_external_catalogs: true
I20260811 18:57:19.038678 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.039171 3891 raft_consensus.cc:2454] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.043840 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0540e4e43ea34fbfbc66b6035c60684d
I20260811 18:57:19.046330 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:19.049227 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_460a6647324c49b78cd79e59e974732b" } new_table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" modify_external_catalogs: true
I20260811 18:57:18.834169 4169 rpcz_store.cc:276] 0811 18:57:16.426244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.426326 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:18.597574 (+2171248us) ts_tablet_manager.cc:1145] Deleting tablet a81334e7da3741a9b24f8978312181b9
0811 18:57:18.597588 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:18.755402 (+157814us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.832866 (+ 77464us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:18.834002 (+ 1136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2168629}
I20260811 18:57:19.068478 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 316) took 2639 ms. Trace:
I20260811 18:57:19.068742 4169 rpcz_store.cc:276] 0811 18:57:16.429380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.431284 (+ 1904us) service_pool.cc:227] Handling call
0811 18:57:19.068250 (+2636966us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:19.068263 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.068403 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633325}
I20260811 18:57:19.069381 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a81334e7da3741a9b24f8978312181b9 (table default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd]) successfully deleted
I20260811 18:57:19.069636 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 317) took 2634 ms. Trace:
I20260811 18:57:19.033030 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:19.070091 4162 tablet_bootstrap.cc:492] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.070254 4162 ts_tablet_manager.cc:1397] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.164s user 0.002s sys 0.003s
I20260811 18:57:19.070353 4167 ts_tablet_manager.cc:1905] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.071162 4162 raft_consensus.cc:348] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.071426 4162 raft_consensus.cc:374] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.050936 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.071463 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:19.071522 4162 raft_consensus.cc:729] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.071785 4162 consensus_queue.cc:260] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.072019 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 3744b3d608754de68c4c57b072d1dc15 deleted and replaced by tablet 91818d0cbe144a67a458a6648968bf5b
I20260811 18:57:19.072393 4162 ts_tablet_manager.cc:1428] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:19.072361 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.073033 4162 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.075124 4162 tablet_bootstrap.cc:654] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.075623 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3744b3d608754de68c4c57b072d1dc15 with delete_type TABLET_DATA_DELETED (replaced by tablet 91818d0cbe144a67a458a6648968bf5b) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:19.076364 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2c0f6df7a291456d80538e2b76417873 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c0f6df7a291456d80538e2b76417873 already in progress: creating tablet
I20260811 18:57:19.081122 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:19.083930 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab9634fb958f4d44840fa6f1709ce326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.084728 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9634fb958f4d44840fa6f1709ce326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.084753 4162 tablet_bootstrap.cc:492] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.084937 4162 ts_tablet_manager.cc:1397] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20260811 18:57:19.085656 4162 raft_consensus.cc:348] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.085914 4162 raft_consensus.cc:374] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.085983 4162 raft_consensus.cc:729] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.086222 4162 consensus_queue.cc:260] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.089118 4162 ts_tablet_manager.cc:1428] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:19.089505 4162 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.091794 4162 tablet_bootstrap.cc:654] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.092005 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.092320 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 331396c9962f48ed9ea6dc66f1660902 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:04 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.097836 3676 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.098197 4836 raft_consensus.cc:2781] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.098309 4836 raft_consensus.cc:482] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.098464 4836 raft_consensus.cc:3037] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.101418 4836 raft_consensus.cc:504] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.104949 4836 leader_election.cc:288] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.107244 4167 ts_tablet_manager.cc:1918] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:19.070809 3824 leader_election.cc:341] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.109892 4167 log.cc:1196] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:19.110749 4167 ts_tablet_manager.cc:1939] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.084514 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.111498 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 3744b3d608754de68c4c57b072d1dc15 with delete_type TABLET_DATA_DELETED (replaced by tablet 91818d0cbe144a67a458a6648968bf5b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.112164 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6381a951a966473c80670b0dd1aff062" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.112286 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a0116997a1c64e658045c319f1c45861
I20260811 18:57:19.112459 3891 raft_consensus.cc:3037] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.114286 4169 rpcz_store.cc:276] 0811 18:57:16.435177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.435234 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:19.069457 (+2634223us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:19.069469 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.069575 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2631868}
I20260811 18:57:19.114861 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 318) took 2676 ms. Trace:
I20260811 18:57:19.114941 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.114974 4169 rpcz_store.cc:276] 0811 18:57:16.437945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.438000 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:19.114627 (+2676627us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:19.114641 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.114799 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2676286}
I20260811 18:57:19.115288 4169 tablet_replica.cc:330] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:19.115299 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c1ceb39258a049f08a1f48c40df65d8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.115530 4169 raft_consensus.cc:2229] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.115623 3891 raft_consensus.cc:2454] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.115654 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 274) took 3278 ms. Trace:
I20260811 18:57:19.115728 4169 raft_consensus.cc:2258] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.115767 4167 rpcz_store.cc:276] 0811 18:57:15.837619 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.837757 (+ 138us) service_pool.cc:227] Handling call
0811 18:57:19.031364 (+3193607us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:19.031378 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.102634 (+ 71256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.107092 (+ 4458us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.115614 (+ 8522us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2983666}
W20260811 18:57:19.115974 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1ceb39258a049f08a1f48c40df65d8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.116441 4169 ts_tablet_manager.cc:1905] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.116447 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 280) took 3239 ms. Trace:
I20260811 18:57:19.116576 3676 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.116885 4836 raft_consensus.cc:2781] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.117024 4836 raft_consensus.cc:686] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.117277 4836 consensus_queue.cc:237] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.117453 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8426ccbe530446fcbaa162a03cbb1892 (table default.Table_a311fc9dc1724795a63f4df79f8ebcbc [id=dca60a562d6e4af49ec113002d510730]) successfully deleted
I20260811 18:57:19.118429 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 460e03cc14444c83b6c08ca194ca5a8d from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.119197 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c8efe53501a143ed90881e375a449e81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.119807 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8efe53501a143ed90881e375a449e81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.119771 4167 rpcz_store.cc:276] 0811 18:57:15.876676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:15.879125 (+ 2449us) service_pool.cc:227] Handling call
0811 18:57:19.116269 (+3237144us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:19.116281 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.116402 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3068006}
W20260811 18:57:19.120275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ad424c8843ff434ba0449a86849bb5a0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad424c8843ff434ba0449a86849bb5a0 already in progress: creating tablet
I20260811 18:57:19.120322 4167 tablet_replica.cc:330] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.120831 4167 raft_consensus.cc:2229] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.121033 4167 raft_consensus.cc:2258] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:19.121089 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 799ba9a044e145ec89d0025c03f391ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 799ba9a044e145ec89d0025c03f391ea already in progress: creating tablet
I20260811 18:57:19.121732 4167 ts_tablet_manager.cc:1905] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.122932 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.124433 4900 tablet.cc:1722] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:19.085913 4163 tablet_bootstrap.cc:654] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.140364 4898 tablet.cc:1722] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:19.140949 4836 raft_consensus.cc:482] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.141144 4836 raft_consensus.cc:504] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.143636 4836 leader_election.cc:288] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:19.144014 3676 leader_election.cc:341] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:19.146449 4162 tablet_bootstrap.cc:492] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.146608 4162 ts_tablet_manager.cc:1397] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.057s user 0.001s sys 0.004s
I20260811 18:57:19.147132 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:19.147970 4162 raft_consensus.cc:348] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.148475 4836 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.148631 4836 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.148896 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c70d9db064e45af91fd9901d85e211f (table default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c]). Expected version 2 got 0
I20260811 18:57:19.149348 4836 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.150272 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.150527 3888 raft_consensus.cc:2454] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.150800 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 27cda53b7ca54346a7a22bf26f5db990 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.150936 3676 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.151088 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.151324 4729 raft_consensus.cc:2781] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.151444 4729 raft_consensus.cc:482] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.151541 4729 raft_consensus.cc:3037] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.152042 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.152452 4162 raft_consensus.cc:374] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.152575 4162 raft_consensus.cc:729] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.152810 4162 consensus_queue.cc:260] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.153520 4162 ts_tablet_manager.cc:1428] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260811 18:57:19.155093 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.156153 4729 raft_consensus.cc:504] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.158159 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9421bc220774f66b90bf537a47f1e13" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.158478 3888 raft_consensus.cc:3037] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.159441 4162 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.161271 4162 tablet_bootstrap.cc:654] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.163533 4878 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.163769 4878 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.164691 4878 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.166679 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:19.167877 3824 leader_election.cc:341] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.169660 3888 raft_consensus.cc:2454] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.169759 4167 ts_tablet_manager.cc:1918] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:19.171077 3676 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.171479 4836 raft_consensus.cc:2781] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.171635 4836 raft_consensus.cc:686] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.171737 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f6a3985ef18b44138437cd5cd54f34ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.171927 4836 consensus_queue.cc:237] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.172427 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6a3985ef18b44138437cd5cd54f34ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.172859 4167 log.cc:1196] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0e1efffade0f486d8d627e56e0be6ff6
I20260811 18:57:19.173329 4167 ts_tablet_manager.cc:1939] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.174026 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 297) took 2874 ms. Trace:
I20260811 18:57:19.174152 4167 rpcz_store.cc:276] 0811 18:57:16.299180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.443931 (+144751us) service_pool.cc:227] Handling call
0811 18:57:19.120245 (+2676314us) ts_tablet_manager.cc:1145] Deleting tablet 0e1efffade0f486d8d627e56e0be6ff6
0811 18:57:19.120261 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.143647 (+ 23386us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.169617 (+ 25970us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.173950 (+ 4333us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659976}
I20260811 18:57:19.175276 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c25e17a6776443429e46c4aab57537c7
I20260811 18:57:19.175654 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_27d36ce508354815add475acc3f872eb" } new_table_name: "default.Table_821a790ed161401bb9758d71a0760916" modify_external_catalogs: true
I20260811 18:57:19.177147 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 296) took 2883 ms. Trace:
I20260811 18:57:19.177295 4167 rpcz_store.cc:276] 0811 18:57:16.293383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.442227 (+148844us) service_pool.cc:227] Handling call
0811 18:57:19.176897 (+2734670us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:19.176915 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.177066 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2606719}
W20260811 18:57:19.177825 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b3ec38f9b55489ba00ed76f9e4456f5 already in progress: creating tablet
I20260811 18:57:19.178033 4167 tablet_replica.cc:330] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.178313 4167 raft_consensus.cc:2229] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.178766 4167 raft_consensus.cc:2258] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.183473 4163 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.183697 4163 ts_tablet_manager.cc:1397] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.341s user 0.004s sys 0.001s
I20260811 18:57:19.184639 4163 raft_consensus.cc:348] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.184945 4163 raft_consensus.cc:374] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.185039 4163 raft_consensus.cc:729] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.185431 4163 consensus_queue.cc:260] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.186961 4167 ts_tablet_manager.cc:1905] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.191627 4162 tablet_bootstrap.cc:492] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.194418 4162 ts_tablet_manager.cc:1397] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.005s
I20260811 18:57:19.195443 4729 leader_election.cc:288] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.196556 4729 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.196767 4729 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.198889 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.199303 3888 raft_consensus.cc:2454] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.199752 3676 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.200045 4903 raft_consensus.cc:2781] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.200160 4903 raft_consensus.cc:482] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.200281 4903 raft_consensus.cc:3037] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.202863 4162 raft_consensus.cc:348] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.203164 4162 raft_consensus.cc:374] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.203270 4162 raft_consensus.cc:729] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.203316 4903 raft_consensus.cc:504] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.204298 4729 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.204766 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad616256b9949ceaf8b300f660d00c5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.205242 3888 raft_consensus.cc:3037] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.204557 4903 leader_election.cc:288] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.211236 4162 consensus_queue.cc:260] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.212651 4162 ts_tablet_manager.cc:1428] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260811 18:57:19.214303 3888 raft_consensus.cc:2454] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.214978 3676 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.215359 4729 raft_consensus.cc:2781] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.215627 4729 raft_consensus.cc:686] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.219717 4729 consensus_queue.cc:237] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:19.222463 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6432abd4fc5c4f9ca95971ca4795e1cf consensus state has no leader
I20260811 18:57:19.222905 4167 ts_tablet_manager.cc:1918] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:19.223053 4167 log.cc:1196] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a81334e7da3741a9b24f8978312181b9
W20260811 18:57:19.223558 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf5749d4eebd42319df1b52eeba4c062 has no consensus state
I20260811 18:57:19.223562 4167 ts_tablet_manager.cc:1939] T a81334e7da3741a9b24f8978312181b9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.224859 4162 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.226773 4162 tablet_bootstrap.cc:654] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.231180 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 300) took 2850 ms. Trace:
I20260811 18:57:19.231369 4167 rpcz_store.cc:276] 0811 18:57:16.380305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.471548 (+ 91243us) service_pool.cc:227] Handling call
0811 18:57:19.177915 (+2706367us) ts_tablet_manager.cc:1145] Deleting tablet a81334e7da3741a9b24f8978312181b9
0811 18:57:19.177931 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.197101 (+ 19170us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.222759 (+ 25658us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.231133 (+ 8374us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2607207}
I20260811 18:57:19.231935 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a81334e7da3741a9b24f8978312181b9 (table default.Table_c68caefab77748b4821e0313f5f95a1a [id=53b2edaad71944edb32a2b604deebbdd]) successfully deleted
I20260811 18:57:19.232138 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 301) took 2848 ms. Trace:
I20260811 18:57:19.232302 4167 rpcz_store.cc:276] 0811 18:57:16.383646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.495343 (+111697us) service_pool.cc:227] Handling call
0811 18:57:19.231945 (+2736602us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:19.231962 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.232102 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660828}
W20260811 18:57:19.232827 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2c0f6df7a291456d80538e2b76417873 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c0f6df7a291456d80538e2b76417873 already in progress: creating tablet
I20260811 18:57:19.232933 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 302) took 2839 ms. Trace:
I20260811 18:57:19.233071 4167 rpcz_store.cc:276] 0811 18:57:16.399319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.525560 (+126241us) service_pool.cc:227] Handling call
0811 18:57:19.232785 (+2707225us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:19.232799 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.232899 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660567}
I20260811 18:57:19.233578 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 303) took 2828 ms. Trace:
I20260811 18:57:19.233732 4167 rpcz_store.cc:276] 0811 18:57:16.404801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.538119 (+133318us) service_pool.cc:227] Handling call
0811 18:57:19.233438 (+2695319us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:19.233451 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.233544 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2660722}
W20260811 18:57:19.234123 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 799ba9a044e145ec89d0025c03f391ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 799ba9a044e145ec89d0025c03f391ea already in progress: creating tablet
I20260811 18:57:19.234297 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 304) took 2553 ms. Trace:
I20260811 18:57:19.234452 4167 rpcz_store.cc:276] 0811 18:57:16.681162 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.681233 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:19.234113 (+2552880us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:19.234125 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.234248 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2525839}
W20260811 18:57:19.234910 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f8eb041f3d704762868a048faec0e0fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f8eb041f3d704762868a048faec0e0fc already in progress: creating tablet
I20260811 18:57:19.235005 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 314) took 2389 ms. Trace:
I20260811 18:57:19.235133 4167 rpcz_store.cc:276] 0811 18:57:16.845875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.845952 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:19.234858 (+2388906us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:19.234872 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.234965 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2382677}
W20260811 18:57:19.235572 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 19243ab3d6744aaa87a133f1462cd2fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19243ab3d6744aaa87a133f1462cd2fa already in progress: creating tablet
I20260811 18:57:19.235721 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 321) took 2182 ms. Trace:
I20260811 18:57:19.235867 4167 rpcz_store.cc:276] 0811 18:57:17.053413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.053478 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:19.235561 (+2182083us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:19.235574 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.235680 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2170290}
I20260811 18:57:19.236425 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 323) took 2161 ms. Trace:
I20260811 18:57:19.236583 4167 rpcz_store.cc:276] 0811 18:57:17.074603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.075715 (+ 1112us) service_pool.cc:227] Handling call
0811 18:57:19.236271 (+2160556us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:19.236284 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.236392 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2143879}
I20260811 18:57:19.237933 4163 ts_tablet_manager.cc:1428] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.054s user 0.001s sys 0.002s
I20260811 18:57:19.240495 4163 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.242591 4167 tablet_replica.cc:330] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.242951 4167 raft_consensus.cc:2229] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.243124 4163 tablet_bootstrap.cc:654] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.243399 4167 raft_consensus.cc:2258] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.244514 4167 ts_tablet_manager.cc:1905] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.248392 4169 ts_tablet_manager.cc:1918] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:19.250890 4162 tablet_bootstrap.cc:492] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.251078 4162 ts_tablet_manager.cc:1397] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.001s
I20260811 18:57:19.252050 4162 raft_consensus.cc:348] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.252368 4162 raft_consensus.cc:374] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.252483 4162 raft_consensus.cc:729] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.252758 4162 consensus_queue.cc:260] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.263523 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.264228 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74385eb07bdb4cb39d7783a47ae5781c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.265708 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_42b47e0e7e404c8c976e7472833fd243" } new_table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" modify_external_catalogs: true
I20260811 18:57:19.267483 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a8d2d56f43544d68adbbd2848fc00dde" } new_table_name: "default.Table_079d99e744054f2a8a2d6ac002aca4e3" modify_external_catalogs: true
I20260811 18:57:19.268796 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea042ebbfb444a2086b87857b6775140"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.269384 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea042ebbfb444a2086b87857b6775140 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.270411 4162 ts_tablet_manager.cc:1428] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:19.270893 4162 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.271041 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_771bec99eb584420a45d22e9b038229a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.271579 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_771bec99eb584420a45d22e9b038229a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.272959 4162 tablet_bootstrap.cc:654] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.276786 4167 ts_tablet_manager.cc:1918] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260811 18:57:19.277145 4167 log.cc:1196] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:19.277866 4167 ts_tablet_manager.cc:1939] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.278824 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 324) took 2203 ms. Trace:
I20260811 18:57:19.278995 4167 rpcz_store.cc:276] 0811 18:57:17.074831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.088514 (+ 13683us) service_pool.cc:227] Handling call
0811 18:57:19.242496 (+2153982us) ts_tablet_manager.cc:1145] Deleting tablet 7b227bc7305f4b2dae28f04e7001553b
0811 18:57:19.242515 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.248911 (+ 6396us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.276602 (+ 27691us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.278741 (+ 2139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2145625}
I20260811 18:57:19.281126 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]) successfully deleted
I20260811 18:57:19.281164 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 322) took 2217 ms. Trace:
I20260811 18:57:19.281330 4167 rpcz_store.cc:276] 0811 18:57:17.063501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.064471 (+ 970us) service_pool.cc:227] Handling call
0811 18:57:19.280925 (+2216454us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:19.280939 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.281086 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2183572}
I20260811 18:57:19.281786 4167 tablet_replica.cc:330] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:19.281875 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cbf6ce167fb348be8d61372791f0cbca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbf6ce167fb348be8d61372791f0cbca already in progress: creating tablet
I20260811 18:57:19.282146 4167 raft_consensus.cc:2229] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.290845 4167 pending_rounds.cc:62] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.290934 4167 pending_rounds.cc:68] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106479046656 op_type: NO_OP noop_request { }
I20260811 18:57:19.291134 4167 raft_consensus.cc:2866] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.291268 4167 raft_consensus.cc:2258] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.291990 4167 ts_tablet_manager.cc:1905] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.289397 4162 tablet_bootstrap.cc:492] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.298739 4162 ts_tablet_manager.cc:1397] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.004s
W20260811 18:57:19.298847 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68a7051bc539468aafba30fbaf0669d4 consensus state has no leader
I20260811 18:57:19.300948 4162 raft_consensus.cc:348] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.301273 4162 raft_consensus.cc:374] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.301364 4162 raft_consensus.cc:729] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.302608 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_786c886ed1164646855f804b8397bed1" } modify_external_catalogs: true
I20260811 18:57:19.302575 4162 consensus_queue.cc:260] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.302836 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_786c886ed1164646855f804b8397bed1" } modify_external_catalogs: true
I20260811 18:57:19.309249 4162 ts_tablet_manager.cc:1428] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:19.309275 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet c2247194db704f9abd54a8a0df1ca566 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.312314 3553 catalog_manager.cc:5495] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:19.312834 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_86432f719d4b48c7ae9f1e0fd8095fcc [id=263eb849fd614011a9dd641aeb9126d0]). Expected version 1 got 0
I20260811 18:57:19.265894 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.315479 4169 log.cc:1196] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f8eb041f3d704762868a048faec0e0fc
I20260811 18:57:19.316437 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 91c8101778b54239b834a4f11ef4a08a (DEFAULT_TABLE table=default.Table_cafb4240a8584ec184a99a633b46eeb1 [id=471bee2cce33409c8c4b890f9ee1297c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.316704 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 91c8101778b54239b834a4f11ef4a08a (DEFAULT_TABLE table=default.Table_cafb4240a8584ec184a99a633b46eeb1 [id=471bee2cce33409c8c4b890f9ee1297c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.316902 4162 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.316968 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c8101778b54239b834a4f11ef4a08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.317215 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c8101778b54239b834a4f11ef4a08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.318661 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 (DEFAULT_TABLE table=default.Table_cd4eb198addc42f2b6d860ef805c589f [id=f2e5c5f123ca451f8f217e96692247ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.319044 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7991dabe263b44f6b7b7d1d43b96ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.320595 4162 tablet_bootstrap.cc:654] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.322036 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7f48893a8614cacb6393af837f050f3" } modify_external_catalogs: true
I20260811 18:57:19.322312 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7f48893a8614cacb6393af837f050f3" } modify_external_catalogs: true
I20260811 18:57:19.323366 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 92e49846ab714dd294acdd09d49a83bd (DEFAULT_TABLE table=default.Table_ea0cc7a576d64d1c934364316cae3c70 [id=fa447a325fe54e78a18b949dd3706dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.323865 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e49846ab714dd294acdd09d49a83bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.326689 3861 tablet_service.cc:1467] Processing CreateTablet for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (DEFAULT_TABLE table=default.Table_fdae8856d30a46259e91345b60181c2b [id=de9b6daef0ed416aa79b0dc0f4b02c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.327085 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.328863 3715 tablet_service.cc:1467] Processing CreateTablet for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (DEFAULT_TABLE table=default.Table_fdae8856d30a46259e91345b60181c2b [id=de9b6daef0ed416aa79b0dc0f4b02c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.331100 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 (DEFAULT_TABLE table=default.Table_cd4eb198addc42f2b6d860ef805c589f [id=f2e5c5f123ca451f8f217e96692247ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.331606 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7991dabe263b44f6b7b7d1d43b96ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.336303 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c0886d985c1343cab4196151b02eff8d (DEFAULT_TABLE table=default.Table_a93ee06ed76342e082697ded810c4c99 [id=a50bb8f0718e4073bfe4254492df7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.336808 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0886d985c1343cab4196151b02eff8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.339145 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 380c1f53b3ac4509a09b25a2800028af (DEFAULT_TABLE table=default.Table_6a0c48e6299546a2921a7a7db03f1032 [id=f7bad72dc09b4b97abea39686ec4ae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.339612 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 380c1f53b3ac4509a09b25a2800028af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.342659 3860 tablet_service.cc:1467] Processing CreateTablet for tablet bf5749d4eebd42319df1b52eeba4c062 (DEFAULT_TABLE table=default.Table_3428bf646eff41ebb8394b062d83cbe0 [id=14e90fa7300e4a5ea10d2da1e28e24a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.342679 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 92e49846ab714dd294acdd09d49a83bd (DEFAULT_TABLE table=default.Table_ea0cc7a576d64d1c934364316cae3c70 [id=fa447a325fe54e78a18b949dd3706dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.343199 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5749d4eebd42319df1b52eeba4c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.344399 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.346283 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92e49846ab714dd294acdd09d49a83bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.349143 3862 tablet_service.cc:1467] Processing CreateTablet for tablet d8aea8dc31914052832bdc83d638461f (DEFAULT_TABLE table=default.Table_459873f5dbe2412c80b73ca35dce90ed [id=a2bf91886ddb4b9bb10366b77185f531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.349561 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8aea8dc31914052832bdc83d638461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.351764 4167 ts_tablet_manager.cc:1918] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:19.353344 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4093564a88264ed59c945523c9ddc9b3 (DEFAULT_TABLE table=default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.353960 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4093564a88264ed59c945523c9ddc9b3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:19.355510 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 122b7898fea44ef78f269b1505d8ddc4 consensus state has no leader
I20260811 18:57:19.356000 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef613c8dc0524f318790523ba7652317" } modify_external_catalogs: true
I20260811 18:57:19.356283 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef613c8dc0524f318790523ba7652317" } modify_external_catalogs: true
I20260811 18:57:19.356526 4167 log.cc:1196] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ed3346bd12d04c45a511308c427856e5
I20260811 18:57:19.357426 4167 ts_tablet_manager.cc:1939] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:19.359372 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 325) took 2265 ms. Trace:
I20260811 18:57:19.359563 4167 rpcz_store.cc:276] 0811 18:57:17.092618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.092733 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:19.281687 (+2188954us) ts_tablet_manager.cc:1145] Deleting tablet ed3346bd12d04c45a511308c427856e5
0811 18:57:19.281699 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.295821 (+ 14122us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.345724 (+ 49903us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.358178 (+ 12454us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2173744}
I20260811 18:57:19.360860 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 320) took 2310 ms. Trace:
I20260811 18:57:19.361064 4167 rpcz_store.cc:276] 0811 18:57:17.050242 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.088459 (+ 38217us) service_pool.cc:227] Handling call
0811 18:57:19.359964 (+2271505us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:19.359981 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.360122 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2250579}
W20260811 18:57:19.361696 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a994b3f4d7d4c92946ad8d595e6ad2b consensus state has no leader
I20260811 18:57:19.362388 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.363080 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.361711 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 380c1f53b3ac4509a09b25a2800028af (DEFAULT_TABLE table=default.Table_6a0c48e6299546a2921a7a7db03f1032 [id=f7bad72dc09b4b97abea39686ec4ae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.368300 3708 tablet_service.cc:1467] Processing CreateTablet for tablet c0886d985c1343cab4196151b02eff8d (DEFAULT_TABLE table=default.Table_a93ee06ed76342e082697ded810c4c99 [id=a50bb8f0718e4073bfe4254492df7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.369599 3721 tablet_service.cc:1467] Processing CreateTablet for tablet bf5749d4eebd42319df1b52eeba4c062 (DEFAULT_TABLE table=default.Table_3428bf646eff41ebb8394b062d83cbe0 [id=14e90fa7300e4a5ea10d2da1e28e24a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.375262 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ed3346bd12d04c45a511308c427856e5 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:19.375808 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 48d32bb4eca047e4ba786c04b8bebf8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48d32bb4eca047e4ba786c04b8bebf8b already in progress: creating tablet
I20260811 18:57:19.375993 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ccc63b46a4e346f79d1e03a82490794c (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.386550 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 380c1f53b3ac4509a09b25a2800028af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.394599 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 4093564a88264ed59c945523c9ddc9b3 (DEFAULT_TABLE table=default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.395566 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d8aea8dc31914052832bdc83d638461f (DEFAULT_TABLE table=default.Table_459873f5dbe2412c80b73ca35dce90ed [id=a2bf91886ddb4b9bb10366b77185f531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.396596 4729 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.396944 4729 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.398375 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.398540 3851 tablet_service.cc:1467] Processing CreateTablet for tablet ccc63b46a4e346f79d1e03a82490794c (DEFAULT_TABLE table=default.Table_0d93e11fb1464fcbb4999abaebc3d611 [id=7ef504f80694466a804cb34703c396fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.398665 3888 raft_consensus.cc:2454] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.398991 4729 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.399276 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5749d4eebd42319df1b52eeba4c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.399340 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc63b46a4e346f79d1e03a82490794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.400497 3676 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.401006 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0886d985c1343cab4196151b02eff8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.403713 4167 tablet_replica.cc:330] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.405182 4729 raft_consensus.cc:2781] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.410454 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8aea8dc31914052832bdc83d638461f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.411262 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b (DEFAULT_TABLE table=default.Table_d9b7a2cfa85347bf88241286c4bf0083 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.411466 4729 raft_consensus.cc:482] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.411672 4729 raft_consensus.cc:3037] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.412068 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1800e9e2b064a6fb546b9d33cefa68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.414810 4167 raft_consensus.cc:2229] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:19.415454 3717 tablet_service.cc:1467] Processing CreateTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b (DEFAULT_TABLE table=default.Table_d9b7a2cfa85347bf88241286c4bf0083 [id=0a4e4826c2cf4250a34e7e26eece8e3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.418665 4836 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.419216 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4093564a88264ed59c945523c9ddc9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.423550 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1800e9e2b064a6fb546b9d33cefa68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.429425 4836 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.431373 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.431747 3888 raft_consensus.cc:2454] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.430301 4836 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.432411 4903 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.432546 3676 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.433583 4903 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.434616 4903 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.435858 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.436257 3888 raft_consensus.cc:2454] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.437341 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccc63b46a4e346f79d1e03a82490794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.441505 3676 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.443131 4836 raft_consensus.cc:2781] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.443397 4836 raft_consensus.cc:482] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.445219 4167 pending_rounds.cc:62] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.454763 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be1e5fa6b9304fd6bcb4304038be24f3" } modify_external_catalogs: true
I20260811 18:57:19.455173 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be1e5fa6b9304fd6bcb4304038be24f3" } modify_external_catalogs: true
I20260811 18:57:19.457351 4163 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.457697 4163 ts_tablet_manager.cc:1397] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.217s user 0.005s sys 0.000s
I20260811 18:57:19.445478 4836 raft_consensus.cc:3037] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.446506 4903 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.462863 4167 pending_rounds.cc:68] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400113725657088 op_type: NO_OP noop_request { }
I20260811 18:57:19.463454 4167 raft_consensus.cc:2866] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.463775 4167 raft_consensus.cc:2258] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.464704 4167 ts_tablet_manager.cc:1905] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.462394 4903 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.469857 4903 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.470263 4903 raft_consensus.cc:2781] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.471297 4903 raft_consensus.cc:482] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.471442 4903 raft_consensus.cc:3037] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.471662 4169 ts_tablet_manager.cc:1939] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:19.472187 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 319) took 2813 ms. Trace:
I20260811 18:57:19.472368 4169 rpcz_store.cc:276] 0811 18:57:16.659030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.659153 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:19.115205 (+2456052us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:19.115214 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.243539 (+128325us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.248217 (+ 4678us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.472138 (+223921us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2455645}
I20260811 18:57:19.473304 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 329) took 2633 ms. Trace:
I20260811 18:57:19.473697 4169 rpcz_store.cc:276] 0811 18:57:16.839609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.839680 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:19.472982 (+2633302us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:19.472997 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.473204 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2632860}
I20260811 18:57:19.474754 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 334) took 2539 ms. Trace:
I20260811 18:57:19.475224 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_06e3c71319514b82b5e73f430a685e0f [id=730987527e1749bebaa10e860079be4c] alter complete (version 2)
I20260811 18:57:19.476452 3550 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 470efca550444d9cb55bb2fe2d951b1c
I20260811 18:57:19.477401 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:19.478202 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 deleted and replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7
I20260811 18:57:19.479521 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f8eb041f3d704762868a048faec0e0fc (table default.Table_45c439603b1140e998b5d9199026da49 [id=32914b6ff485486f843da54878e2d49b]) successfully deleted
W20260811 18:57:19.480036 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 19243ab3d6744aaa87a133f1462cd2fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19243ab3d6744aaa87a133f1462cd2fa already in progress: creating tablet
I20260811 18:57:19.493937 4162 tablet_bootstrap.cc:492] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.494413 4162 ts_tablet_manager.cc:1397] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.178s user 0.020s sys 0.015s
I20260811 18:57:19.484264 4729 raft_consensus.cc:504] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.508244 4169 rpcz_store.cc:276] 0811 18:57:16.935221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.935298 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:19.474468 (+2539170us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:19.474482 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.474653 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2531308}
I20260811 18:57:19.509425 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 336) took 2567 ms. Trace:
I20260811 18:57:19.509788 4169 rpcz_store.cc:276] 0811 18:57:16.941869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.941952 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:19.509068 (+2567116us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:19.509086 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.509312 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2565489}
I20260811 18:57:19.511121 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 335) took 2574 ms. Trace:
W20260811 18:57:19.511044 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 48d32bb4eca047e4ba786c04b8bebf8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48d32bb4eca047e4ba786c04b8bebf8b already in progress: creating tablet
I20260811 18:57:19.511550 4169 rpcz_store.cc:276] 0811 18:57:16.936480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.936548 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:19.510767 (+2574219us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:19.510780 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.511012 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2566972}
I20260811 18:57:19.485929 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.459578 4163 raft_consensus.cc:348] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.512303 4163 raft_consensus.cc:374] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.512589 4163 raft_consensus.cc:729] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.513159 4163 consensus_queue.cc:260] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.515690 4162 raft_consensus.cc:348] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.521513 4729 leader_election.cc:288] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.525517 4903 raft_consensus.cc:504] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.527405 4163 ts_tablet_manager.cc:1428] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.069s user 0.001s sys 0.002s
I20260811 18:57:19.526052 4162 raft_consensus.cc:374] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.528223 4163 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.528825 4162 raft_consensus.cc:729] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.529230 4162 consensus_queue.cc:260] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.532315 4163 tablet_bootstrap.cc:654] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.533702 4903 leader_election.cc:288] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.533957 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:19.534739 3555 catalog_manager.cc:6565] ReplaceTablet: tablet c2247194db704f9abd54a8a0df1ca566 deleted and replaced by tablet 2dce6143309948d49e229dc9890d904f
I20260811 18:57:19.537528 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:19.540190 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8d1e244addf5462499fcadaa3332cba1
I20260811 18:57:19.542543 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a122fe2820f4d23b12edb970a94e958" } new_table_name: "default.Table_e496287ddd104a6db81655bea6a19802" modify_external_catalogs: true
I20260811 18:57:19.544608 4836 raft_consensus.cc:504] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.546686 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7455a95b7b2a4c5db623ad62d8332150"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.547569 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7455a95b7b2a4c5db623ad62d8332150 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.548547 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4981573f7a434b5da77a06d251ea684c" } modify_external_catalogs: true
I20260811 18:57:19.548900 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4981573f7a434b5da77a06d251ea684c" } modify_external_catalogs: true
I20260811 18:57:19.550877 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_914d7e775ebf4b26beed57cb21140166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.551563 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_914d7e775ebf4b26beed57cb21140166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.558517 4836 leader_election.cc:288] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.559983 4162 ts_tablet_manager.cc:1428] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.052s user 0.000s sys 0.004s
I20260811 18:57:19.560369 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.562067 4162 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.566423 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cdd3a653fc1b405cbfd89c5d98728e45" } new_table_name: "default.Table_2103ce2744174eec9f603b49604123e4" modify_external_catalogs: true
I20260811 18:57:19.558851 4914 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.566197 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a795d5395cc64155b84057a53c701064"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.571964 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:19.572444 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 91719.2us
W20260811 18:57:19.581918 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a795d5395cc64155b84057a53c701064 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.583820 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.584988 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01f622bd12ce4b2bb39355d87ed79fb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.586423 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d1eb56bed2494b7f85229d2c92bcb60c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.587146 3678 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.305s (SENT)
W20260811 18:57:19.587424 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1eb56bed2494b7f85229d2c92bcb60c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.568089 4914 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.594321 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.596972 4162 tablet_bootstrap.cc:654] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:19.538646 3824 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.175s (SENT)
I20260811 18:57:19.598173 3824 leader_election.cc:302] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:19.598628 3824 raft_consensus.cc:2670] T e7b3a14979e84b1e8ec6ed3f49b93b00 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:19.599534 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 60931.1us
W20260811 18:57:19.600010 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cbf6ce167fb348be8d61372791f0cbca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbf6ce167fb348be8d61372791f0cbca already in progress: creating tablet
I20260811 18:57:19.540310 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.541530 4878 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.600710 4878 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.601632 4878 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:19.602834 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.607769 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.562498 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 337) took 2593 ms. Trace:
I20260811 18:57:19.575578 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.610737 3888 raft_consensus.cc:2454] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.576767 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331396c9962f48ed9ea6dc66f1660902" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.611231 3891 raft_consensus.cc:3037] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.597434 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.587903 4915 raft_consensus.cc:482] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 57e8f62fa63a4fee97f268f5b7d3ae24)
I20260811 18:57:19.577649 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e215759d48c24d8cb99bbc3fc9009a66" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.578526 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0190cbd859274fcfa09c6fab899c4843" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.614718 3890 raft_consensus.cc:3037] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.614740 3892 raft_consensus.cc:3037] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.607285 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:19.609313 3678 leader_election.cc:334] T 7b227bc7305f4b2dae28f04e7001553b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.942s (SENT)
I20260811 18:57:19.619962 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.620122 3892 raft_consensus.cc:2454] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.620273 3888 raft_consensus.cc:2454] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.620916 4169 rpcz_store.cc:276] 0811 18:57:16.968908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.968996 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:19.562080 (+2593084us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:19.562106 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.562299 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2592128}
I20260811 18:57:19.621662 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 338) took 2647 ms. Trace:
I20260811 18:57:19.621949 4915 raft_consensus.cc:504] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.621760 3824 leader_election.cc:341] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:19.623168 4169 rpcz_store.cc:276] 0811 18:57:16.974651 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:16.974729 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:19.621374 (+2646645us) ts_tablet_manager.cc:1145] Deleting tablet 151df87806694f46be5013592c68c01a
0811 18:57:19.621388 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.621606 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2646309}
I20260811 18:57:19.624775 4169 tablet_replica.cc:330] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:19.624688 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 151df87806694f46be5013592c68c01a because the tablet was not found. No further retry: Not found: Tablet not found: 151df87806694f46be5013592c68c01a
I20260811 18:57:19.622741 3891 raft_consensus.cc:2454] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.624050 3890 raft_consensus.cc:2454] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.625967 4914 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:19.626497 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 57937.7us
I20260811 18:57:19.626718 4915 leader_election.cc:288] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.629411 4169 raft_consensus.cc:2229] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.633222 3676 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.634042 4915 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.634831 4915 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.636205 4915 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.636822 3676 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.643863 3676 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.637261 4915 raft_consensus.cc:2781] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.645310 4915 raft_consensus.cc:482] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.645843 4915 raft_consensus.cc:3037] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.644312 4914 raft_consensus.cc:2781] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.650761 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:19.651719 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a33a6c24205421f9fc488087672c6e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.652693 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a33a6c24205421f9fc488087672c6e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.653954 4915 raft_consensus.cc:504] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.655613 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.658840 4903 raft_consensus.cc:2781] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.659192 4903 raft_consensus.cc:482] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.659565 4903 raft_consensus.cc:3037] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.664837 4903 raft_consensus.cc:504] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.669786 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:19.672410 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a98bd35bd444195fd32fe6c67a04f consensus state has no leader
I20260811 18:57:19.672995 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_86432f719d4b48c7ae9f1e0fd8095fcc" } new_table_name: "default.Table_4da51fec1a044237ab13da18366dde64" modify_external_catalogs: true
I20260811 18:57:19.666443 4914 raft_consensus.cc:686] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.674610 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e18242f762d8407db0e2f8716890feb4
I20260811 18:57:19.675117 4914 consensus_queue.cc:237] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.675882 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47718da43cfe46efab776245606d964f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.667537 3676 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:19.676620 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47718da43cfe46efab776245606d964f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.676700 4836 raft_consensus.cc:2781] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.677631 4836 raft_consensus.cc:686] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.678031 4836 consensus_queue.cc:237] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:19.680449 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
I20260811 18:57:19.680474 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.680702 4169 raft_consensus.cc:2258] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.681066 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cb88cf12cee8407389570138b87a10d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:19.682034 4169 ts_tablet_manager.cc:1905] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:19.682930 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb88cf12cee8407389570138b87a10d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.684223 3676 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.684388 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet f75dc515d7f3473b8e43f20255a70e3e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.684861 4729 raft_consensus.cc:2781] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.684942 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5409ed17441242119ff946163da07a21" } new_table_name: "default.Table_70493900fc9346389c1d4b71c611b962" modify_external_catalogs: true
I20260811 18:57:19.686474 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ef1abae384a9404089b0f6600e935a5f" } new_table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" modify_external_catalogs: true
I20260811 18:57:19.669961 4167 ts_tablet_manager.cc:1918] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:19.696687 4167 log.cc:1196] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9f8c0444ec2242299482736a73551f80
I20260811 18:57:19.697539 4167 ts_tablet_manager.cc:1939] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:19.702546 3676 leader_election.cc:341] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 33979de5b48c4674a46c8da3b7151753
I20260811 18:57:19.703871 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.673521 4162 tablet_bootstrap.cc:492] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:19.704820 4162 ts_tablet_manager.cc:1397] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.143s user 0.007s sys 0.000s
I20260811 18:57:19.705180 3889 raft_consensus.cc:2454] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.708039 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6446af9e35994d3c856c29a9c3ff32d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.708844 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6446af9e35994d3c856c29a9c3ff32d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.711239 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4" } modify_external_catalogs: true
I20260811 18:57:19.711526 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0b53eaa03dd34a22a14fa65da3866fe4" } modify_external_catalogs: true
I20260811 18:57:19.714838 4162 raft_consensus.cc:348] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.715349 4162 raft_consensus.cc:374] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.715541 4162 raft_consensus.cc:729] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.716110 4162 consensus_queue.cc:260] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.717103 4162 ts_tablet_manager.cc:1428] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:19.717713 4162 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:19.705598 4914 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.706714 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.718998 4914 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.712808 4729 raft_consensus.cc:686] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.721273 4729 consensus_queue.cc:237] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.724287 4903 leader_election.cc:288] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.744290 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f25ff8a9e3a944269e9799995c73ebe0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.744899 3890 raft_consensus.cc:3037] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.749243 4163 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:19.749655 4163 ts_tablet_manager.cc:1397] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.222s user 0.004s sys 0.007s
I20260811 18:57:19.751076 3890 raft_consensus.cc:2454] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:19.752616 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3263da75f0d84528b7c8cc55dcaeaa46 (table default.Table_d7a06761b8a145519b7bb645e7a73657 [id=f51f38f3582f44db99007a265b665fd4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3263da75f0d84528b7c8cc55dcaeaa46 consensus state has no leader
I20260811 18:57:19.752539 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 326) took 2654 ms. Trace:
I20260811 18:57:19.753230 4167 rpcz_store.cc:276] 0811 18:57:17.098121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.104245 (+ 6124us) service_pool.cc:227] Handling call
0811 18:57:19.361397 (+2257152us) ts_tablet_manager.cc:1145] Deleting tablet 9f8c0444ec2242299482736a73551f80
0811 18:57:19.361412 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.521070 (+159658us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.669677 (+148607us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.752446 (+ 82769us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2245336}
I20260811 18:57:19.755801 4167 tablet_replica.cc:330] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:19.756244 4167 raft_consensus.cc:2229] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:19.756672 4167 pending_rounds.cc:62] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:19.756855 4167 pending_rounds.cc:68] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400106077646848 op_type: NO_OP noop_request { }
I20260811 18:57:19.757200 4167 raft_consensus.cc:2866] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:19.757402 4167 raft_consensus.cc:2258] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:19.759245 4915 leader_election.cc:288] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.759898 3676 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.760980 3676 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.761829 4836 raft_consensus.cc:2781] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.762027 4836 raft_consensus.cc:482] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.762405 4836 raft_consensus.cc:3037] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:19.764357 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cf6c8d542ae2489ba466ffadd518db12 consensus state has no leader
I20260811 18:57:19.767334 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9f8c0444ec2242299482736a73551f80 (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:19.770557 4903 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.771020 4924 raft_consensus.cc:2781] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.773448 4914 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.751400 4163 raft_consensus.cc:348] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.775101 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866ed614511649459b22a9fce55220b2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.777592 4903 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.779230 4903 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.779779 4915 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.781411 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406" } modify_external_catalogs: true
I20260811 18:57:19.781888 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406" } modify_external_catalogs: true
I20260811 18:57:19.782049 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.781975 4915 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.782879 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914c8b90fadf48e88b75d13897462d84" } modify_external_catalogs: true
I20260811 18:57:19.783274 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914c8b90fadf48e88b75d13897462d84" } modify_external_catalogs: true
I20260811 18:57:19.784250 4169 ts_tablet_manager.cc:1918] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260811 18:57:19.784475 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.784873 3889 raft_consensus.cc:2454] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:19.789198 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4dc2df4c6464d71bb13fe25e0046758 consensus state has no leader
I20260811 18:57:19.790629 3891 raft_consensus.cc:2454] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.790940 3890 raft_consensus.cc:3037] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.789768 4167 ts_tablet_manager.cc:1905] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:19.793809 4915 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.796249 4836 raft_consensus.cc:504] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.797149 3676 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.798014 3890 raft_consensus.cc:2454] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.798804 4163 raft_consensus.cc:374] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.798949 4163 raft_consensus.cc:729] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.798956 4915 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.799216 4915 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.799333 4163 consensus_queue.cc:260] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.801625 4927 raft_consensus.cc:2781] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.801893 4927 raft_consensus.cc:482] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.803372 4927 raft_consensus.cc:3037] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.804996 4915 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:19.807595 4163 ts_tablet_manager.cc:1428] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.058s user 0.004s sys 0.000s
I20260811 18:57:19.811080 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.811575 3889 raft_consensus.cc:2454] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:19.811970 3824 leader_election.cc:334] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:19.812294 3824 leader_election.cc:302] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:19.812563 3676 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.812870 4878 raft_consensus.cc:2726] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:19.816298 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 515d5b5afbd74ec5ab90a503aef2f4fb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.820183 3676 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.823151 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a732c700ff79432aaf1f0d1c1e90aabe" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.823531 3890 raft_consensus.cc:3037] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.800580 4924 raft_consensus.cc:686] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.827090 4903 raft_consensus.cc:2781] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.827379 4903 raft_consensus.cc:686] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:19.828814 4915 raft_consensus.cc:2781] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.829780 3551 catalog_manager.cc:5495] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:19.830900 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
W20260811 18:57:19.834566 3824 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.081s (SENT)
I20260811 18:57:19.834789 3676 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.835711 4924 consensus_queue.cc:237] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.837337 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a (DEFAULT_TABLE table=default.Table_f6a3985ef18b44138437cd5cd54f34ed [id=2bc826b3f7464185a308efa3382cd966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.838500 4915 raft_consensus.cc:482] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.839640 4915 raft_consensus.cc:3037] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.839963 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f3060822021431584a494e6941be756 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:19.840438 4836 leader_election.cc:288] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.844079 3860 tablet_service.cc:1467] Processing CreateTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 (DEFAULT_TABLE table=default.Table_74385eb07bdb4cb39d7783a47ae5781c [id=def664eaf5774792acf9f58193e6d6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.844758 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ebc32eb5d5443eaaeba245b566f98a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.847132 4162 tablet_bootstrap.cc:654] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.851536 3721 tablet_service.cc:1467] Processing CreateTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 (DEFAULT_TABLE table=default.Table_74385eb07bdb4cb39d7783a47ae5781c [id=def664eaf5774792acf9f58193e6d6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.852342 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2816cc2dfe45f0815ce50a5a929324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.852946 4914 raft_consensus.cc:2781] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.853303 4169 log.cc:1196] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7b227bc7305f4b2dae28f04e7001553b
I20260811 18:57:19.854267 4169 ts_tablet_manager.cc:1939] T 7b227bc7305f4b2dae28f04e7001553b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:19.840687 4903 consensus_queue.cc:237] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:19.860870 4914 raft_consensus.cc:482] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.861123 4914 raft_consensus.cc:3037] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:19.863966 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 11137ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:19.869401 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a (DEFAULT_TABLE table=default.Table_f6a3985ef18b44138437cd5cd54f34ed [id=2bc826b3f7464185a308efa3382cd966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.870455 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ebc32eb5d5443eaaeba245b566f98a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.859836 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:19.875159 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 85140a3800f34a23b716703fd4e73d17 (DEFAULT_TABLE table=default.Table_c1ceb39258a049f08a1f48c40df65d8b [id=ba51142694a0491f857bcb698e29d93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.875866 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85140a3800f34a23b716703fd4e73d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.878183 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 14e34732bda44670ae9da8facc405221 (DEFAULT_TABLE table=default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 [id=c2c29eaa8e054b14a1d3ae3d75be99d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.878902 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e34732bda44670ae9da8facc405221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.887791 4729 consensus_queue.cc:1035] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260811 18:57:19.895393 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bc6cce2c19746219b8b0fe54b13dcd6
I20260811 18:57:19.896327 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7a0e0d139c924f1dafe42da0bedada72
I20260811 18:57:19.897238 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8efe53501a143ed90881e375a449e81" } modify_external_catalogs: true
I20260811 18:57:19.897526 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8efe53501a143ed90881e375a449e81" } modify_external_catalogs: true
I20260811 18:57:19.899271 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 (DEFAULT_TABLE table=default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 [id=da25cefa0734439090e25c2e1f491657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.899456 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
I20260811 18:57:19.901289 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 14e34732bda44670ae9da8facc405221 (DEFAULT_TABLE table=default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09 [id=c2c29eaa8e054b14a1d3ae3d75be99d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.894255 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_90566ec48eaf42c8be4c06681c3c0268"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.902714 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90566ec48eaf42c8be4c06681c3c0268 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.903568 3889 raft_consensus.cc:2454] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:19.905611 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.906468 3891 raft_consensus.cc:1261] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:19.834100 4927 raft_consensus.cc:504] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:19.906087 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 (DEFAULT_TABLE table=default.Table_3ddaf0bc5d83468e8693ed3b6b5fa406 [id=da25cefa0734439090e25c2e1f491657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.916281 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f3670f8908d54ebb82f698cdd207e48f (DEFAULT_TABLE table=default.Table_e12ffa777d684216bfc19d52f2dad4be [id=7071461f50fc49958d87c5575b94335e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.920039 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" } modify_external_catalogs: true
I20260811 18:57:19.920234 3870 tablet_service.cc:1467] Processing CreateTablet for tablet f3670f8908d54ebb82f698cdd207e48f (DEFAULT_TABLE table=default.Table_e12ffa777d684216bfc19d52f2dad4be [id=7071461f50fc49958d87c5575b94335e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.920383 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c391f66ce98e48f3bf4a736969eca1af" } modify_external_catalogs: true
I20260811 18:57:19.920871 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc (DEFAULT_TABLE table=default.Table_278e0a9db8364b7cb00a0920de4b0561 [id=ff9e8d54b42c44d294858300dd87d043]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.921518 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 91818d0cbe144a67a458a6648968bf5b (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.919355 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.923074 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.926767 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:19.926700 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 85140a3800f34a23b716703fd4e73d17 (DEFAULT_TABLE table=default.Table_c1ceb39258a049f08a1f48c40df65d8b [id=ba51142694a0491f857bcb698e29d93a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.927189 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.927326 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3670f8908d54ebb82f698cdd207e48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.927697 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0e0d139c924f1dafe42da0bedada72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:19.927934 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.928037 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 (DEFAULT_TABLE table=default.Table_c8efe53501a143ed90881e375a449e81 [id=978fc92b4923484d9a4fdc96f4d4dba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.928385 3889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987b9ac6859444e3a86c56af786cc84e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:19.928676 3889 raft_consensus.cc:3037] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.934561 3889 raft_consensus.cc:2454] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.935155 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc (DEFAULT_TABLE table=default.Table_278e0a9db8364b7cb00a0920de4b0561 [id=ff9e8d54b42c44d294858300dd87d043]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.935708 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289e607f10e14a2983bc6ecc93ce9bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.936116 4163 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:19.937564 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 91818d0cbe144a67a458a6648968bf5b (DEFAULT_TABLE table=default.Table_97097723c5264ab781135d328c9be026 [id=f51e49fa4aed42fea2f5153042cecbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.938200 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91818d0cbe144a67a458a6648968bf5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.944509 3890 raft_consensus.cc:2454] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:19.949488 4927 leader_election.cc:288] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:19.952984 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 (DEFAULT_TABLE table=default.Table_c8efe53501a143ed90881e375a449e81 [id=978fc92b4923484d9a4fdc96f4d4dba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.953276 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 (DEFAULT_TABLE table=default.Table_ab9634fb958f4d44840fa6f1709ce326 [id=5350668f89b84b7899da4209b6bc2822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.953999 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7645a242bc4e4845ad49bfbe20b252f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.956238 4163 tablet_bootstrap.cc:654] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:19.956189 3676 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.957029 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 1811e5f62a714f8388460dfcb1717e81 (DEFAULT_TABLE table=default.Table_655b56a4862144f1afd52a620c3715d3 [id=a5bce8098d7f4150944fc21295d519c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.957329 3676 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.958240 4729 raft_consensus.cc:2781] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.958457 4729 raft_consensus.cc:482] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.959983 4729 raft_consensus.cc:3037] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.960614 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cee8eb42df9b40cd85eb35169133787a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.961551 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cee8eb42df9b40cd85eb35169133787a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.962126 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f20eabb4925482bbf71fe270dde24af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.962807 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f20eabb4925482bbf71fe270dde24af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.963413 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a85e1485efcd445a972fe25c37f2bb9e" } new_table_name: "default.Table_e5be012fe5c34e9bb842907f4b154b8e" modify_external_catalogs: true
I20260811 18:57:19.964161 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a594cb48b586458783a36cc76df1332c" } new_table_name: "default.Table_1b2efb13eaf1468c92c8030667191b84" modify_external_catalogs: true
I20260811 18:57:19.964587 4836 consensus_queue.cc:1035] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260811 18:57:19.964828 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b78f30561e284a9596437b5fc75df24f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:19.965619 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b78f30561e284a9596437b5fc75df24f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:19.968729 4924 raft_consensus.cc:2781] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.969450 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 1811e5f62a714f8388460dfcb1717e81 (DEFAULT_TABLE table=default.Table_655b56a4862144f1afd52a620c3715d3 [id=a5bce8098d7f4150944fc21295d519c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.976454 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1811e5f62a714f8388460dfcb1717e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.972774 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 (DEFAULT_TABLE table=default.Table_ab9634fb958f4d44840fa6f1709ce326 [id=5350668f89b84b7899da4209b6bc2822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:19.981921 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3468b39bd73c4272a84dad37fcfd6f27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:19.986560 3678 leader_election.cc:334] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.151s (SENT)
I20260811 18:57:19.986936 3678 leader_election.cc:302] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:19.988406 3678 raft_consensus.cc:2670] T 08b8dcf7a56043efbbd40bb888616492 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:19.991863 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2816cc2dfe45f0815ce50a5a929324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:19.997192 3676 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:19.998935 4927 raft_consensus.cc:2781] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.999486 4927 raft_consensus.cc:686] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.000350 4927 consensus_queue.cc:237] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.012413 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.015030 4903 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.015476 4903 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.024799 4924 raft_consensus.cc:686] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.025878 4914 raft_consensus.cc:504] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.026407 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e34732bda44670ae9da8facc405221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.028587 4924 consensus_queue.cc:237] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.030692 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289e607f10e14a2983bc6ecc93ce9bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.034591 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7645a242bc4e4845ad49bfbe20b252f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.038743 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3670f8908d54ebb82f698cdd207e48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.042791 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91818d0cbe144a67a458a6648968bf5b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.042865 3678 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.444s (SENT)
I20260811 18:57:20.057799 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1811e5f62a714f8388460dfcb1717e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.060886 4162 tablet_bootstrap.cc:492] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:20.061311 4162 ts_tablet_manager.cc:1397] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.344s user 0.004s sys 0.004s
I20260811 18:57:20.062606 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85140a3800f34a23b716703fd4e73d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.065333 4915 raft_consensus.cc:504] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.066781 4915 leader_election.cc:288] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.070956 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3468b39bd73c4272a84dad37fcfd6f27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.074944 3824 leader_election.cc:334] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.576s (SENT)
I20260811 18:57:20.075242 3824 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:20.075600 3824 raft_consensus.cc:2670] T 8af3e74889b64c408c9ed9c54916f890 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.075521 4162 raft_consensus.cc:348] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.076241 4162 raft_consensus.cc:374] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.076468 4162 raft_consensus.cc:729] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.076781 4162 consensus_queue.cc:260] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.076949 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48782.1us
I20260811 18:57:20.078960 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7259cf84f5c4c89a1b4510f743c2bc5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.079298 3888 raft_consensus.cc:3037] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.083863 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 339) took 3005 ms. Trace:
I20260811 18:57:20.084180 4169 rpcz_store.cc:276] 0811 18:57:17.017120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.018895 (+ 1775us) service_pool.cc:227] Handling call
0811 18:57:19.624548 (+2605653us) ts_tablet_manager.cc:1145] Deleting tablet 7b227bc7305f4b2dae28f04e7001553b
0811 18:57:19.624576 (+ 28us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.731026 (+106450us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:19.784094 (+ 53068us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.022319 (+238225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2605271}
I20260811 18:57:20.085243 4914 leader_election.cc:288] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.085769 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7b227bc7305f4b2dae28f04e7001553b (table default.Table_3bd2f4a0463f483aa76c5d66c9f83d53 [id=76fa1a8be6ed4f28932466dba0a991db]) successfully deleted
I20260811 18:57:20.086565 4729 raft_consensus.cc:504] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.086958 4169 tablet_replica.cc:330] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:20.089527 4903 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.091303 4729 leader_election.cc:288] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.099225 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d564ad80064912a8ce8e31c1b6c4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.099597 3891 raft_consensus.cc:3037] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.108527 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:20.108886 4878 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.113304 3893 raft_consensus.cc:2454] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:20.113191 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405773849fb94800be61ad8a7a6055a4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.114501 3888 raft_consensus.cc:2454] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.136670 3892 raft_consensus.cc:3037] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.142077 4878 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.144562 4167 ts_tablet_manager.cc:1918] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.148981 4167 log.cc:1196] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/33979de5b48c4674a46c8da3b7151753
I20260811 18:57:20.148182 3676 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.151269 3676 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.151980 4924 raft_consensus.cc:2781] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.153724 4878 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:20.154420 4924 raft_consensus.cc:686] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.154886 4924 consensus_queue.cc:237] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.151418 4927 raft_consensus.cc:2781] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:20.155859 4927 raft_consensus.cc:482] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:20.156173 4927 raft_consensus.cc:3037] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.157547 3892 raft_consensus.cc:2454] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.146071 4162 ts_tablet_manager.cc:1428] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.084s user 0.005s sys 0.001s
I20260811 18:57:20.160383 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.159560 4162 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.160624 3676 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.164703 4729 raft_consensus.cc:2781] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.165271 4729 raft_consensus.cc:686] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.165951 4729 consensus_queue.cc:237] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.168418 3824 leader_election.cc:341] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.174144 4167 ts_tablet_manager.cc:1939] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.175421 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 327) took 3070 ms. Trace:
W20260811 18:57:20.181833 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9166ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:20.192169 4167 rpcz_store.cc:276] 0811 18:57:17.104642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.105872 (+ 1230us) service_pool.cc:227] Handling call
0811 18:57:19.755633 (+2649761us) ts_tablet_manager.cc:1145] Deleting tablet 33979de5b48c4674a46c8da3b7151753
0811 18:57:19.755647 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:19.890165 (+134518us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.144381 (+254216us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.175318 (+ 30937us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2639274}
I20260811 18:57:20.201788 4167 tablet_replica.cc:330] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.202432 4167 raft_consensus.cc:2229] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.202723 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 33979de5b48c4674a46c8da3b7151753 (table default.Table_fd619a3970764d0fb914fcc6f59ea7a5 [id=bc9956004bf143178c1002192f259ac3]) successfully deleted
I20260811 18:57:20.203116 4167 raft_consensus.cc:2258] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.204119 4167 ts_tablet_manager.cc:1905] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.205477 4162 tablet_bootstrap.cc:654] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:20.209110 4163 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:20.218243 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 85609.2us
I20260811 18:57:20.220569 4163 ts_tablet_manager.cc:1397] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.350s user 0.001s sys 0.007s
I20260811 18:57:20.226723 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6f3060822021431584a494e6941be756
I20260811 18:57:20.227550 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 6f3060822021431584a494e6941be756 deleted and replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76
I20260811 18:57:20.229676 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7645a242bc4e4845ad49bfbe20b252f2
I20260811 18:57:20.235864 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.237460 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_680160aaac0a4c8da66634dc41d6abb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.239955 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.240965 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.243378 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_680160aaac0a4c8da66634dc41d6abb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.245456 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.246836 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:20.248018 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4d5db1f0452e40eea01177ba93223341"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.248889 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d5db1f0452e40eea01177ba93223341 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.256352 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.257973 4878 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.260167 4878 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.261051 3891 raft_consensus.cc:2454] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.261754 3676 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.262279 4941 raft_consensus.cc:2781] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.263180 4941 raft_consensus.cc:686] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.263630 4941 consensus_queue.cc:237] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.265170 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.266479 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.271255 4162 tablet_bootstrap.cc:492] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:20.271845 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.272560 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 405773849fb94800be61ad8a7a6055a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 0aee6beb2e70421083841520733087f8) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:20.273818 3824 leader_election.cc:341] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.191524 4927 raft_consensus.cc:504] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:20.282073 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 62440.2us
W20260811 18:57:20.282650 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:20.285823 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e7259cf84f5c4c89a1b4510f743c2bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.286440 4162 ts_tablet_manager.cc:1397] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.112s user 0.005s sys 0.006s
I20260811 18:57:20.286728 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.300436 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.309434 4878 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:20.310525 4729 consensus_queue.cc:1035] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:20.311959 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 7645a242bc4e4845ad49bfbe20b252f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.312477 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c8443df21949ff9bdc63fa6aa34108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.313081 4167 ts_tablet_manager.cc:1918] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.313244 4167 log.cc:1196] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6ce8db230c7e41b6a4792273a0f07e6e
I20260811 18:57:20.317286 4927 leader_election.cc:288] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.322067 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d995d72f8142afaeb57a2c29910fa3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.322417 3892 raft_consensus.cc:3037] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.329556 4169 raft_consensus.cc:2229] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.332145 4169 raft_consensus.cc:2258] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.335960 4162 raft_consensus.cc:348] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.337700 4169 ts_tablet_manager.cc:1905] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.338121 4927 consensus_queue.cc:1035] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:20.342468 4162 raft_consensus.cc:374] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.342700 4162 raft_consensus.cc:729] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.343151 4162 consensus_queue.cc:260] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.344092 4941 raft_consensus.cc:482] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.344455 4941 raft_consensus.cc:504] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.344703 4162 ts_tablet_manager.cc:1428] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.058s user 0.000s sys 0.004s
I20260811 18:57:20.345255 4162 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.345513 4941 leader_election.cc:288] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:20.351154 3676 leader_election.cc:341] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: f8eb041f3d704762868a048faec0e0fc
W20260811 18:57:20.357061 3678 leader_election.cc:334] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.269s (SENT)
I20260811 18:57:20.357326 3678 leader_election.cc:302] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:20.357651 3678 raft_consensus.cc:2670] T ae03bc2e2b4b4d0d9e26cc4c2fbe5d97 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.367069 4163 raft_consensus.cc:348] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.368963 4163 raft_consensus.cc:374] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.369997 4163 raft_consensus.cc:729] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.371222 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ebb36a4f14a0491692aec7b72295b80c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.371953 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebb36a4f14a0491692aec7b72295b80c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.373869 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7abca85e5a6d4b55a38c0f9e50f02f76 has no consensus state
W20260811 18:57:20.373986 3678 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.075s (SENT)
I20260811 18:57:20.374564 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b2c3c3b4a2948bbba5b4b556faf1015" } new_table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" modify_external_catalogs: true
W20260811 18:57:20.376236 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4c7f500a5444bae9a015a5e903c0622 consensus state has no leader
I20260811 18:57:20.373376 3892 raft_consensus.cc:2454] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.376936 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d00075c1e094c1cb6a03b6760496fb5" } new_table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" modify_external_catalogs: true
I20260811 18:57:20.379182 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47718da43cfe46efab776245606d964f" } new_table_name: "default.Table_709cd55123504b35877d469f96354d28" modify_external_catalogs: true
I20260811 18:57:20.379660 3676 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.381254 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2e58ff1b0c44ee8b71812ae6399b4da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.381336 4927 raft_consensus.cc:2781] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:20.381929 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2e58ff1b0c44ee8b71812ae6399b4da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.381976 4927 raft_consensus.cc:686] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.382639 4927 consensus_queue.cc:237] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.383795 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4da51fec1a044237ab13da18366dde64" } modify_external_catalogs: true
I20260811 18:57:20.384102 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4da51fec1a044237ab13da18366dde64" } modify_external_catalogs: true
I20260811 18:57:20.386700 4169 ts_tablet_manager.cc:1918] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:20.386868 4169 log.cc:1196] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:20.387367 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d" } modify_external_catalogs: true
I20260811 18:57:20.387491 4169 ts_tablet_manager.cc:1939] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:20.387660 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e85b84e5b0e45bf80321dfb55c1704d" } modify_external_catalogs: true
I20260811 18:57:20.387903 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 340) took 3149 ms. Trace:
I20260811 18:57:20.388062 4169 rpcz_store.cc:276] 0811 18:57:17.238141 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.238401 (+ 260us) service_pool.cc:227] Handling call
0811 18:57:20.084840 (+2846439us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:20.084860 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.329978 (+245118us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x615000128f24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:20.382559 (+ 52581us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.386533 (+ 3974us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.387850 (+ 1317us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":556880128,"tablet-delete.queue_time_us":2841653}
I20260811 18:57:20.391129 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" } modify_external_catalogs: true
I20260811 18:57:20.391381 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e0c0cbe2c2f846c2b85cda84670f448c" } modify_external_catalogs: true
I20260811 18:57:20.425534 4167 ts_tablet_manager.cc:1939] T 6ce8db230c7e41b6a4792273a0f07e6e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.397949 4163 consensus_queue.cc:260] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.427018 4163 ts_tablet_manager.cc:1428] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.206s user 0.004s sys 0.000s
I20260811 18:57:20.427052 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 328) took 3320 ms. Trace:
I20260811 18:57:20.428323 4163 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:20.430466 4162 tablet_bootstrap.cc:654] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:20.376696 4950 log.cc:924] Time spent T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260811 18:57:20.423003 3824 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.427s (SENT)
I20260811 18:57:20.434881 4163 tablet_bootstrap.cc:654] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:20.437631 4169 tablet_replica.cc:330] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:20.437652 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f1a55a42d1e64f04ad7f433614a3a1aa (table default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]) successfully deleted
I20260811 18:57:20.443317 4958 tablet.cc:1722] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:20.448776 3824 leader_election.cc:334] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.702s (SENT)
W20260811 18:57:20.475102 3678 leader_election.cc:334] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.313s (SENT)
I20260811 18:57:20.475492 3678 leader_election.cc:302] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:20.475955 3678 raft_consensus.cc:2670] T d106e0e1f4b845429b10f4e100f69a8b P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:20.477116 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 54115.2us
W20260811 18:57:20.477942 3678 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.620s (SENT)
I20260811 18:57:20.478176 3678 leader_election.cc:302] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:20.478909 4957 tablet.cc:1722] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:20.485512 3824 leader_election.cc:302] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:20.485941 3824 raft_consensus.cc:2670] T a81334e7da3741a9b24f8978312181b9 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.492033 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet d2d7d986900244ddbc05dbe73a708dcf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.492684 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet a732c700ff79432aaf1f0d1c1e90aabe from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.492704 4167 rpcz_store.cc:276] 0811 18:57:17.106095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.107484 (+ 1389us) service_pool.cc:227] Handling call
0811 18:57:20.198843 (+3091359us) ts_tablet_manager.cc:1145] Deleting tablet 6ce8db230c7e41b6a4792273a0f07e6e
0811 18:57:20.198863 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.239321 (+ 40458us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.307547 (+ 68226us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.426933 (+119386us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3072199}
W20260811 18:57:20.493273 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45fde25bcb40493aafb083e161aa6932 consensus state has no leader
I20260811 18:57:20.493646 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 331396c9962f48ed9ea6dc66f1660902 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.494376 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 3744b3d608754de68c4c57b072d1dc15 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:20.495173 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3652d29236a4b12b7abd44659c70311"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.496178 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3652d29236a4b12b7abd44659c70311 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.497915 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28af2d2c61254b0ca399dddb05e9ee70 has no consensus state
I20260811 18:57:20.498584 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_97097723c5264ab781135d328c9be026" } modify_external_catalogs: true
I20260811 18:57:20.499006 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_97097723c5264ab781135d328c9be026" } modify_external_catalogs: true
I20260811 18:57:20.500366 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de899ce498b24cd5b270adf6fbb81b92" } modify_external_catalogs: true
I20260811 18:57:20.500758 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de899ce498b24cd5b270adf6fbb81b92" } modify_external_catalogs: true
W20260811 18:57:20.501367 3678 raft_consensus.cc:2670] T 7f640c60e81b4a369458b1497a269c9c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.501888 4169 raft_consensus.cc:2229] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.502213 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6ce8db230c7e41b6a4792273a0f07e6e (table default.Table_7fa5c52d66624877a73a99af0be2381a [id=e15d32a4751c46acbb6f77b2eb896b9f]) successfully deleted
W20260811 18:57:20.502213 3676 consensus_peers.cc:487] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:20.502462 4169 pending_rounds.cc:62] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:20.502694 4169 pending_rounds.cc:68] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400122051297280 op_type: NO_OP noop_request { }
I20260811 18:57:20.502913 4169 raft_consensus.cc:2866] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:20.503270 4169 raft_consensus.cc:2258] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.504714 4167 tablet_replica.cc:330] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.505710 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6381a951a966473c80670b0dd1aff062
I20260811 18:57:20.505987 4962 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:20.507336 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 6381a951a966473c80670b0dd1aff062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.513226 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6381a951a966473c80670b0dd1aff062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.513777 4167 raft_consensus.cc:2229] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.515974 4167 raft_consensus.cc:2258] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:20.523599 3678 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.417s (SENT)
I20260811 18:57:20.524971 4169 ts_tablet_manager.cc:1905] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.525054 4167 ts_tablet_manager.cc:1905] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.528277 4927 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.536227 4163 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:20.536747 4163 ts_tablet_manager.cc:1397] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.109s user 0.001s sys 0.009s
I20260811 18:57:20.529476 4927 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.544386 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:20.545347 3888 raft_consensus.cc:2454] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:20.547474 4163 raft_consensus.cc:348] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.548135 3676 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.548710 4163 raft_consensus.cc:374] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.548606 4957 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:20.548834 4914 raft_consensus.cc:2781] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:20.549064 4914 raft_consensus.cc:482] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:20.549139 4163 raft_consensus.cc:729] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.549336 4914 raft_consensus.cc:3037] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.552515 4163 consensus_queue.cc:260] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.554239 4914 raft_consensus.cc:504] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.556377 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c1a7d44717415ca76a2d0b7e94394e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:20.563063 3888 raft_consensus.cc:3037] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:20.571105 3888 raft_consensus.cc:2454] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:20.578853 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c7fb552f75e54888873b50a45cb0a475 (DEFAULT_TABLE table=default.Table_6446af9e35994d3c856c29a9c3ff32d2 [id=b2dab2b70454407ab25aefe2843ada47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.579417 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7fb552f75e54888873b50a45cb0a475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.584072 3861 tablet_service.cc:1467] Processing CreateTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 (DEFAULT_TABLE table=default.Table_cb88cf12cee8407389570138b87a10d2 [id=fe56a8d6cdca49f1ba5a900cf37399b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.584092 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0b59cafb154416952aecaa418e861f (DEFAULT_TABLE table=default.Table_5a33a6c24205421f9fc488087672c6e0 [id=9f664fe34e8348f0989ba7b3583e0e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.585026 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0b59cafb154416952aecaa418e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.585214 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 (DEFAULT_TABLE table=default.Table_709cd55123504b35877d469f96354d28 [id=26e34b672a354ee0a67f8eae9b099af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.585263 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]). Expected version 5 got 4
I20260811 18:57:20.585834 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28af2d2c61254b0ca399dddb05e9ee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.588680 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeec0c5672b4b3190d7d7cf341cb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.589061 4914 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.590418 4927 leader_election.cc:288] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:20.593840 3676 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:20.594205 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.597070 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 (DEFAULT_TABLE table=default.Table_709cd55123504b35877d469f96354d28 [id=26e34b672a354ee0a67f8eae9b099af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.597698 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28af2d2c61254b0ca399dddb05e9ee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.598659 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2345d5c2154329b601762c8085bc07 (DEFAULT_TABLE table=default.Table_d1eb56bed2494b7f85229d2c92bcb60c [id=5ebb2f681b264b2caa3fb73f68af73bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.599192 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2345d5c2154329b601762c8085bc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.602411 3718 tablet_service.cc:1467] Processing CreateTablet for tablet c7fb552f75e54888873b50a45cb0a475 (DEFAULT_TABLE table=default.Table_6446af9e35994d3c856c29a9c3ff32d2 [id=b2dab2b70454407ab25aefe2843ada47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.602833 4952 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:20.603168 4952 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.606024 4952 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:20.603014 4914 raft_consensus.cc:2781] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:20.607966 4914 raft_consensus.cc:686] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:20.608403 4914 consensus_queue.cc:237] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:20.603319 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0b59cafb154416952aecaa418e861f (DEFAULT_TABLE table=default.Table_5a33a6c24205421f9fc488087672c6e0 [id=9f664fe34e8348f0989ba7b3583e0e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.605131 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7fb552f75e54888873b50a45cb0a475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.619659 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7bc942d59fd34beda3b8fb9e03e84cb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.620662 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.605871 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 7e2345d5c2154329b601762c8085bc07 (DEFAULT_TABLE table=default.Table_d1eb56bed2494b7f85229d2c92bcb60c [id=5ebb2f681b264b2caa3fb73f68af73bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.623093 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e2345d5c2154329b601762c8085bc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.624413 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_886df95cce4342f998d27c8661df890e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.625250 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_886df95cce4342f998d27c8661df890e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.606575 3710 tablet_service.cc:1467] Processing CreateTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 (DEFAULT_TABLE table=default.Table_cb88cf12cee8407389570138b87a10d2 [id=fe56a8d6cdca49f1ba5a900cf37399b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.607510 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:20.627035 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeec0c5672b4b3190d7d7cf341cb118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.627049 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_459873f5dbe2412c80b73ca35dce90ed" } new_table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" modify_external_catalogs: true
I20260811 18:57:20.628959 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea042ebbfb444a2086b87857b6775140" } new_table_name: "default.Table_315c3cbf08354dae86745d91bbc2cd11" modify_external_catalogs: true
I20260811 18:57:20.630039 4836 consensus_queue.cc:1035] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:20.630811 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0b59cafb154416952aecaa418e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.641204 4162 tablet_bootstrap.cc:492] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:20.641506 3824 leader_election.cc:341] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:20.642688 3871 tablet_service.cc:1467] Processing CreateTablet for tablet a5bbd3274980470b9fe71457b49bdd62 (DEFAULT_TABLE table=default.Table_ea042ebbfb444a2086b87857b6775140 [id=bfc7c5f6b2f041a1a215254c038a1fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.643297 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bbd3274980470b9fe71457b49bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.645392 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e407d34d3c984ecb8c8ffde304f23ce6 (DEFAULT_TABLE table=default.Table_771bec99eb584420a45d22e9b038229a [id=a7de43e4f94740fbb4fd1ba5aa5009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.645509 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe733ace0064e2d886cecbdd6decc76 (DEFAULT_TABLE table=default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 [id=ca9587388aae4f4e9eedc7ff9a137476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.646109 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e407d34d3c984ecb8c8ffde304f23ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.646517 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe733ace0064e2d886cecbdd6decc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.646999 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.647696 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d24aff9fd34f3eb05fdf942cc7efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.680415 4957 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260811 18:57:20.682152 4162 ts_tablet_manager.cc:1397] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.300s user 0.008s sys 0.005s
I20260811 18:57:20.683203 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 22e1a33348e34e35bab786f8966c408b (DEFAULT_TABLE table=default.Table_01f622bd12ce4b2bb39355d87ed79fb2 [id=bf02bf181b36444382ab04e4bc399e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.688876 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce6143309948d49e229dc9890d904f (DEFAULT_TABLE table=default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.691831 4163 ts_tablet_manager.cc:1428] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.131s user 0.002s sys 0.002s
I20260811 18:57:20.694051 4958 tablet.cc:1722] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260811 18:57:20.690210 3856 tablet_service.cc:1467] Processing CreateTablet for tablet aeb83eaf12444d6fbd52af568389cb3c (DEFAULT_TABLE table=default.Table_7455a95b7b2a4c5db623ad62d8332150 [id=93a003db2a41483b8675373b7ab6772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.696552 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:20.697743 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fc78208e540445719638da3757163df5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.698740 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 91818d0cbe144a67a458a6648968bf5b
I20260811 18:57:20.699883 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_30ac5178633e44aeafd707e48d7c4cba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.700769 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30ac5178633e44aeafd707e48d7c4cba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.703029 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:20.704568 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab9ea53bcc98448384f4fd3d30d2528f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.705556 3824 leader_election.cc:334] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.942s (SENT)
W20260811 18:57:20.705729 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9ea53bcc98448384f4fd3d30d2528f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.715516 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc78208e540445719638da3757163df5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.718088 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:20.718953 3547 catalog_manager.cc:6565] ReplaceTablet: tablet a732c700ff79432aaf1f0d1c1e90aabe deleted and replaced by tablet fb878de2c0d94408aa6e98447bd7e958
I20260811 18:57:20.719734 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f713f72dfeb48658564074b098a97d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.720551 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f713f72dfeb48658564074b098a97d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.723012 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:20.723583 3554 catalog_manager.cc:5495] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.724140 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:20.724911 3554 catalog_manager.cc:5495] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.725718 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:20.726914 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 059b325df1db4fde9d4b26b559121316
I20260811 18:57:20.727562 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e1a33348e34e35bab786f8966c408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.728018 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9dd303f12b584056a04e385cc80faffe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.728938 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9dd303f12b584056a04e385cc80faffe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.730360 4163 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:20.731374 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 91818d0cbe144a67a458a6648968bf5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.733875 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce6143309948d49e229dc9890d904f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.734876 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.735668 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.736668 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe with delete_type TABLET_DATA_DELETED (replaced by tablet fb878de2c0d94408aa6e98447bd7e958) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.737164 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:20.740718 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb83eaf12444d6fbd52af568389cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.749873 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d8aea8dc31914052832bdc83d638461f consensus state has no leader
I20260811 18:57:20.750846 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_badc11f1b0554e629ddb3f8a20e151e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.751703 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_badc11f1b0554e629ddb3f8a20e151e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:20.753412 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5bbd3274980470b9fe71457b49bdd62 consensus state has no leader
I20260811 18:57:20.753978 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_89b7704fd6264ef8bd2289a13f9a488d" } new_table_name: "default.Table_8e4cf372a70649839b2083d6740f8c99" modify_external_catalogs: true
I20260811 18:57:20.757264 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafb4240a8584ec184a99a633b46eeb1" } new_table_name: "default.Table_235289a804494101acba3a4f14192af3" modify_external_catalogs: true
W20260811 18:57:20.759374 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d73a98bd35bd444195fd32fe6c67a04f consensus state has no leader
I20260811 18:57:20.783845 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe733ace0064e2d886cecbdd6decc76 (DEFAULT_TABLE table=default.Table_df38d857d4cd4a1ab964cd1b5ed1bc88 [id=ca9587388aae4f4e9eedc7ff9a137476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.784653 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe733ace0064e2d886cecbdd6decc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.782013 3718 tablet_service.cc:1467] Processing CreateTablet for tablet a5bbd3274980470b9fe71457b49bdd62 (DEFAULT_TABLE table=default.Table_ea042ebbfb444a2086b87857b6775140 [id=bfc7c5f6b2f041a1a215254c038a1fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.790803 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e407d34d3c984ecb8c8ffde304f23ce6 (DEFAULT_TABLE table=default.Table_771bec99eb584420a45d22e9b038229a [id=a7de43e4f94740fbb4fd1ba5aa5009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.794720 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.794898 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 5a75a480f38047d880b0c1fa51403786 (DEFAULT_TABLE table=default.Table_914d7e775ebf4b26beed57cb21140166 [id=b39c0641dd714d8a89839cadc2c567eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.795272 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d24aff9fd34f3eb05fdf942cc7efc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.796753 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a75a480f38047d880b0c1fa51403786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.799855 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bbd3274980470b9fe71457b49bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.802028 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5fef93c0a9b546199a3217a8b8bd67dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.803077 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5fef93c0a9b546199a3217a8b8bd67dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.804709 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 22e1a33348e34e35bab786f8966c408b (DEFAULT_TABLE table=default.Table_01f622bd12ce4b2bb39355d87ed79fb2 [id=bf02bf181b36444382ab04e4bc399e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.812987 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e407d34d3c984ecb8c8ffde304f23ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.819764 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 2dce6143309948d49e229dc9890d904f (DEFAULT_TABLE table=default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.820708 3712 tablet_service.cc:1467] Processing CreateTablet for tablet aeb83eaf12444d6fbd52af568389cb3c (DEFAULT_TABLE table=default.Table_7455a95b7b2a4c5db623ad62d8332150 [id=93a003db2a41483b8675373b7ab6772c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.820489 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 (DEFAULT_TABLE table=default.Table_a795d5395cc64155b84057a53c701064 [id=0e2d7e83bb53429db83d32be7ea5b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.822530 4162 raft_consensus.cc:348] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.827093 4162 raft_consensus.cc:374] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:20.824658 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.826514 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e1a33348e34e35bab786f8966c408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.829411 4162 raft_consensus.cc:729] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:20.829849 4162 consensus_queue.cc:260] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:20.832702 4162 ts_tablet_manager.cc:1428] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.148s user 0.002s sys 0.002s
I20260811 18:57:20.835361 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 91818d0cbe144a67a458a6648968bf5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.836452 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d988ef7e9dd43158d107998ea4d82b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.838672 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeb83eaf12444d6fbd52af568389cb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.851121 4162 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:20.852525 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dce6143309948d49e229dc9890d904f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:20.854143 3824 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.087s (SENT)
I20260811 18:57:20.854298 4167 ts_tablet_manager.cc:1918] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:20.854399 3824 leader_election.cc:302] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:20.854665 4167 log.cc:1196] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/dde27d8cc3eb4d09879e717fc90305fd
W20260811 18:57:20.854770 3824 raft_consensus.cc:2670] T ed3346bd12d04c45a511308c427856e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:20.855448 4167 ts_tablet_manager.cc:1939] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:20.856211 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 345) took 3336 ms. Trace:
I20260811 18:57:20.856333 4167 rpcz_store.cc:276] 0811 18:57:17.519879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.519940 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:20.493260 (+2973320us) ts_tablet_manager.cc:1145] Deleting tablet dde27d8cc3eb4d09879e717fc90305fd
0811 18:57:20.493274 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.642394 (+149120us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.682506 (+ 40112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.856144 (+173638us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2957939}
I20260811 18:57:20.857016 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 344) took 3337 ms. Trace:
I20260811 18:57:20.857203 4167 rpcz_store.cc:276] 0811 18:57:17.519741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.520199 (+ 458us) service_pool.cc:227] Handling call
0811 18:57:20.856690 (+3336491us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:20.856703 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.856886 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3300834}
I20260811 18:57:20.857820 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 346) took 3321 ms. Trace:
I20260811 18:57:20.858012 4167 rpcz_store.cc:276] 0811 18:57:17.535946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.536125 (+ 179us) service_pool.cc:227] Handling call
0811 18:57:20.857546 (+3321421us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:20.857557 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.857686 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3300614}
I20260811 18:57:20.858496 4167 tablet_replica.cc:330] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:20.858829 4167 raft_consensus.cc:2229] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:20.859315 4167 raft_consensus.cc:2258] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.860267 4167 ts_tablet_manager.cc:1905] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:20.868438 3678 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.126s (SENT)
I20260811 18:57:20.869624 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.869881 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.873030 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet dde27d8cc3eb4d09879e717fc90305fd (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
W20260811 18:57:20.873442 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c already in progress: creating tablet
W20260811 18:57:20.875202 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b4de110bd3348f0b0dae21c9f2b6a19 already in progress: creating tablet
I20260811 18:57:20.877668 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 (DEFAULT_TABLE table=default.Table_a795d5395cc64155b84057a53c701064 [id=0e2d7e83bb53429db83d32be7ea5b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.896981 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_278e0a9db8364b7cb00a0920de4b0561" } modify_external_catalogs: true
I20260811 18:57:20.898115 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d988ef7e9dd43158d107998ea4d82b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.900918 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 5a75a480f38047d880b0c1fa51403786 (DEFAULT_TABLE table=default.Table_914d7e775ebf4b26beed57cb21140166 [id=b39c0641dd714d8a89839cadc2c567eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:20.902355 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a75a480f38047d880b0c1fa51403786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:20.909883 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_278e0a9db8364b7cb00a0920de4b0561" } modify_external_catalogs: true
I20260811 18:57:20.911743 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet a732c700ff79432aaf1f0d1c1e90aabe with delete_type TABLET_DATA_DELETED (replaced by tablet fb878de2c0d94408aa6e98447bd7e958) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:20.898779 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9fe9862d3266414295d433fd5b136638" } new_table_name: "default.Table_7f7c3f328d7e49db85f19e13aa462cd6" modify_external_catalogs: true
I20260811 18:57:20.918699 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" } modify_external_catalogs: true
I20260811 18:57:20.919121 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7d0c2a4e217c4217aa3fe24ec0738635" } modify_external_catalogs: true
I20260811 18:57:20.928136 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" } modify_external_catalogs: true
I20260811 18:57:20.928696 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3428bf646eff41ebb8394b062d83cbe0" } modify_external_catalogs: true
I20260811 18:57:20.933800 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9634fb958f4d44840fa6f1709ce326" } modify_external_catalogs: true
I20260811 18:57:20.934235 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9634fb958f4d44840fa6f1709ce326" } modify_external_catalogs: true
I20260811 18:57:20.900236 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_99de6d48691743d798f56442d85d5be8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:20.937593 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet f8eb041f3d704762868a048faec0e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:20.938279 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99de6d48691743d798f56442d85d5be8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.905267 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 62c365f79d4a4201ac323f16bddf3e62 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:20.906064 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb878de2c0d94408aa6e98447bd7e958 has no consensus state
W20260811 18:57:20.906491 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91c8101778b54239b834a4f11ef4a08a consensus state has no leader
I20260811 18:57:20.949719 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb88cf12cee8407389570138b87a10d2" } modify_external_catalogs: true
I20260811 18:57:20.950377 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cb88cf12cee8407389570138b87a10d2" } modify_external_catalogs: true
W20260811 18:57:20.960992 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb51e419dd24459caf3ba3adfbbd4517 consensus state has no leader
I20260811 18:57:20.964299 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3468b39bd73c4272a84dad37fcfd6f27
I20260811 18:57:20.968024 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf5749d4eebd42319df1b52eeba4c062
I20260811 18:57:20.968257 4914 consensus_queue.cc:1035] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260811 18:57:20.969586 4924 consensus_queue.cc:1035] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260811 18:57:20.970279 3678 leader_election.cc:334] T e7b3a14979e84b1e8ec6ed3f49b93b00 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.036s (SENT)
I20260811 18:57:20.975189 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8214722ffbdc49bf98d5aa539aebed52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.976153 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8214722ffbdc49bf98d5aa539aebed52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:20.982679 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6d87a15f366a4e118c84e294c3b9926e" } new_table_name: "default.Table_10f91c342797480e9545a00a8d6be4a9" modify_external_catalogs: true
I20260811 18:57:20.988174 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:20.988646 3546 catalog_manager.cc:5495] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:20.989506 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:20.991899 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be30e8194fd44874a1fdd996128eb4d4" } new_table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" modify_external_catalogs: true
I20260811 18:57:20.994082 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65d5ebf213ce401984be01bac1813640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:20.995407 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65d5ebf213ce401984be01bac1813640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.001557 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.002928 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5749d4eebd42319df1b52eeba4c062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.004294 4914 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.006844 4914 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.008036 4914 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:21.026654 3678 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.026s (SENT)
I20260811 18:57:21.026961 3678 leader_election.cc:302] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:21.030231 3678 raft_consensus.cc:2670] T 60274115fe164d85badcf2e72d640f54 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:21.038472 4162 tablet_bootstrap.cc:654] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.034170 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5749d4eebd42319df1b52eeba4c062 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
#6 0x7fc7c23bfbb1 in std::default_delete<kudu::tablet::AlterSchemaOpState>::operator()(kudu::tablet::AlterSchemaOpState*) const ../../../include/c++/8/bits/unique_ptr.h:81:2
I20260811 18:57:21.044723 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 3468b39bd73c4272a84dad37fcfd6f27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.062283 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.066112 3890 raft_consensus.cc:2454] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
#7 0x7fc7c237cf7b in std::unique_ptr<kudu::tablet::AlterSchemaOpState, std::default_delete<kudu::tablet::AlterSchemaOpState> >::~unique_ptr() ../../../include/c++/8/bits/unique_ptr.h:274:4
I20260811 18:57:21.073127 4169 ts_tablet_manager.cc:1918] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
#8 0x7fc7bfab3d04 in kudu::tablet::AlterSchemaOp::~AlterSchemaOp() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:126:7
I20260811 18:57:21.078022 4162 tablet_bootstrap.cc:492] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.078629 4162 ts_tablet_manager.cc:1397] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.245s user 0.006s sys 0.004s
#9 0x7fc7bfab3d42 in kudu::tablet::AlterSchemaOp::~AlterSchemaOp() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/alter_schema_op.h:126:7
I20260811 18:57:21.079121 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 42405aec21e3405486b6c658438b5fb2
I20260811 18:57:21.080523 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fdeec0c5672b4b3190d7d7cf341cb118
I20260811 18:57:21.080631 4162 raft_consensus.cc:348] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.081511 4162 raft_consensus.cc:374] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.081707 4162 raft_consensus.cc:729] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.082525 4162 consensus_queue.cc:260] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.084630 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.085191 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.094691 4162 ts_tablet_manager.cc:1428] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.016s user 0.001s sys 0.003s
I20260811 18:57:21.097700 4162 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.100497 4169 log.cc:1196] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d9421bc220774f66b90bf537a47f1e13
I20260811 18:57:21.101045 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d8f42fcd065448e5a6332ff5512645ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.101917 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8f42fcd065448e5a6332ff5512645ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.103792 4163 tablet_bootstrap.cc:654] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.105407 3676 consensus_peers.cc:487] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1a55a42d1e64f04ad7f433614a3a1aa. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:21.106443 4914 consensus_queue.cc:1035] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.107803 3676 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.108420 4914 raft_consensus.cc:2781] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.108758 4914 raft_consensus.cc:482] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.109074 4924 consensus_queue.cc:1035] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.109762 4914 raft_consensus.cc:3037] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.115005 4914 raft_consensus.cc:504] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.117379 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b9115c298dfa4ecb970e76c7de2cd8e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.117656 4914 leader_election.cc:288] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:21.120034 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.121932 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8c94c9b20d784df0811302a8381a7301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.123606 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeec0c5672b4b3190d7d7cf341cb118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.124043 4162 tablet_bootstrap.cc:654] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.125058 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c94c9b20d784df0811302a8381a7301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:21.126237 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 11432ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:21.127442 4167 ts_tablet_manager.cc:1918] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:21.130661 4167 log.cc:1196] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9c18120da2d245f495c52a670047760d
I20260811 18:57:21.132068 4167 ts_tablet_manager.cc:1939] T 9c18120da2d245f495c52a670047760d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.133118 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.135195 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 351) took 3391 ms. Trace:
I20260811 18:57:21.138564 4167 rpcz_store.cc:276] 0811 18:57:17.743813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.743880 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:20.858247 (+3114367us) ts_tablet_manager.cc:1145] Deleting tablet 9c18120da2d245f495c52a670047760d
0811 18:57:20.858257 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.100126 (+241869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.127197 (+ 27071us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.135134 (+ 7937us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3112188}
I20260811 18:57:21.141067 4167 tablet_replica.cc:330] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.141036 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9c18120da2d245f495c52a670047760d (table default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]) successfully deleted
I20260811 18:57:21.142673 4167 raft_consensus.cc:2229] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:21.142171 4162 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.143168 4162 ts_tablet_manager.cc:1397] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.001s
I20260811 18:57:21.143003 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36724a83bf9844b9a27a88ca36a9bf50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.144711 4914 consensus_queue.cc:1035] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.145417 4836 consensus_queue.cc:1035] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.145921 4162 raft_consensus.cc:348] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.146248 4162 raft_consensus.cc:374] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.146497 4162 raft_consensus.cc:729] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.146880 4162 consensus_queue.cc:260] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.147359 4167 pending_rounds.cc:62] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:21.148974 4167 pending_rounds.cc:68] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400125563219968 op_type: NO_OP noop_request { }
I20260811 18:57:21.150151 3890 raft_consensus.cc:3037] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.151580 4167 raft_consensus.cc:2866] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.151965 4167 raft_consensus.cc:2258] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.153084 4167 ts_tablet_manager.cc:1905] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:21.158435 3824 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.056s (SENT)
I20260811 18:57:21.151400 4162 ts_tablet_manager.cc:1428] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260811 18:57:21.161175 4162 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.162621 4169 ts_tablet_manager.cc:1939] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.163321 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 341) took 3924 ms. Trace:
I20260811 18:57:21.164518 4162 tablet_bootstrap.cc:654] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.173243 4169 rpcz_store.cc:276] 0811 18:57:17.238328 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.238639 (+ 311us) service_pool.cc:227] Handling call
0811 18:57:20.437490 (+3198851us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:20.437502 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:20.502193 (+ 64691us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x6150001b3b24. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1463a04d 00007f4d158de5ec 00007f4d1826d2e6 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c 00007f4d09148344 00007f4d0914b91b 00007f4d0914b798
0811 18:57:20.551452 (+ 49259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:20.875463 (+324011us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.163269 (+287806us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":146215168,"tablet-delete.queue_time_us":3194249}
I20260811 18:57:21.175544 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d9421bc220774f66b90bf537a47f1e13 (table default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]) successfully deleted
I20260811 18:57:21.183238 4836 consensus_queue.cc:1035] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.190992 4162 tablet_bootstrap.cc:492] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.191476 4162 ts_tablet_manager.cc:1397] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.002s
I20260811 18:57:21.193382 4162 raft_consensus.cc:348] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.198676 4162 raft_consensus.cc:374] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.199018 4162 raft_consensus.cc:729] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.200117 4162 consensus_queue.cc:260] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.206398 4162 ts_tablet_manager.cc:1428] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.002s
I20260811 18:57:21.208562 4162 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.202615 4167 ts_tablet_manager.cc:1918] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.209911 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 357) took 3692 ms. Trace:
I20260811 18:57:21.210239 4167 log.cc:1196] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:21.211216 4167 ts_tablet_manager.cc:1939] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.218712 4169 rpcz_store.cc:276] 0811 18:57:17.511075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.511155 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:21.176005 (+3664850us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:21.202360 (+ 26355us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.203298 (+ 938us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3664560}
I20260811 18:57:21.219303 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 358) took 3685 ms. Trace:
I20260811 18:57:21.218097 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 348) took 3478 ms. Trace:
I20260811 18:57:21.220002 4167 rpcz_store.cc:276] 0811 18:57:17.739260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.739354 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:21.140950 (+3401596us) ts_tablet_manager.cc:1145] Deleting tablet 987b9ac6859444e3a86c56af786cc84e
0811 18:57:21.140968 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.176327 (+ 35359us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.202312 (+ 25985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.218016 (+ 15704us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3394614}
W20260811 18:57:21.220434 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c6e0d1eeb63493bbea4cd78dd085a3c already in progress: creating tablet
I20260811 18:57:21.223701 4162 tablet_bootstrap.cc:654] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.228552 4924 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.224589 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 349) took 3484 ms. Trace:
I20260811 18:57:21.229238 4924 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.230417 4924 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.231448 4167 rpcz_store.cc:276] 0811 18:57:17.739664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.739746 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:21.224206 (+3484460us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:21.224223 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.224472 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3477669}
W20260811 18:57:21.232378 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 58c515c153df4317b3fb427983ee15e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58c515c153df4317b3fb427983ee15e0 already in progress: creating tablet
I20260811 18:57:21.232741 4924 consensus_queue.cc:1035] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.233652 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 347) took 3526 ms. Trace:
I20260811 18:57:21.230563 4169 rpcz_store.cc:276] 0811 18:57:17.534042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.534263 (+ 221us) service_pool.cc:227] Handling call
0811 18:57:21.219090 (+3684827us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:21.219114 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.219260 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3684499}
I20260811 18:57:21.234014 4167 rpcz_store.cc:276] 0811 18:57:17.707470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.707630 (+ 160us) service_pool.cc:227] Handling call
0811 18:57:21.232174 (+3524544us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:21.232194 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.233496 (+ 1302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3484594}
W20260811 18:57:21.234905 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b4de110bd3348f0b0dae21c9f2b6a19 already in progress: creating tablet
W20260811 18:57:21.235533 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 968fa8b8912b40a8bdba77db1d39c5c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 968fa8b8912b40a8bdba77db1d39c5c0 already in progress: creating tablet
I20260811 18:57:21.236797 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 350) took 3496 ms. Trace:
I20260811 18:57:21.238404 4167 rpcz_store.cc:276] 0811 18:57:17.740268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.740336 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:21.236416 (+3496080us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:21.236437 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.236656 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3487878}
I20260811 18:57:21.239181 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 359) took 3598 ms. Trace:
W20260811 18:57:21.239333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d2d7d986900244ddbc05dbe73a708dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2d7d986900244ddbc05dbe73a708dcf already in progress: creating tablet
I20260811 18:57:21.239516 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 363) took 3230 ms. Trace:
I20260811 18:57:21.239831 4167 rpcz_store.cc:276] 0811 18:57:18.009233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.009331 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:21.239301 (+3229970us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:21.239328 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.239476 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3226459}
W20260811 18:57:21.240599 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f75dc515d7f3473b8e43f20255a70e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f75dc515d7f3473b8e43f20255a70e3e already in progress: creating tablet
I20260811 18:57:21.245160 4169 rpcz_store.cc:276] 0811 18:57:17.640524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.640595 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:21.236765 (+3596170us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:21.236785 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.239087 (+ 2302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3595910}
I20260811 18:57:21.245920 4169 tablet_replica.cc:330] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:21.246169 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 968fa8b8912b40a8bdba77db1d39c5c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 968fa8b8912b40a8bdba77db1d39c5c0 already in progress: creating tablet
I20260811 18:57:21.246858 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 362) took 3237 ms. Trace:
I20260811 18:57:21.247264 4167 rpcz_store.cc:276] 0811 18:57:18.009104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.010031 (+ 927us) service_pool.cc:227] Handling call
0811 18:57:21.246531 (+3236500us) ts_tablet_manager.cc:1145] Deleting tablet ebf7720e31d84ae2ab592da1e7a3132a
0811 18:57:21.246551 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.246758 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3232618}
I20260811 18:57:21.248127 4167 tablet_replica.cc:330] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:21.248278 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ebf7720e31d84ae2ab592da1e7a3132a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf7720e31d84ae2ab592da1e7a3132a already in progress: creating tablet
I20260811 18:57:21.248641 4167 raft_consensus.cc:2229] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:21.249984 4167 pending_rounds.cc:62] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:21.250317 4167 pending_rounds.cc:68] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400116300894208 op_type: NO_OP noop_request { }
I20260811 18:57:21.251466 4167 raft_consensus.cc:2866] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.253616 4167 raft_consensus.cc:2258] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.254719 4167 ts_tablet_manager.cc:1905] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.253147 4162 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.255230 4162 ts_tablet_manager.cc:1397] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.006s
I20260811 18:57:21.257027 4162 raft_consensus.cc:348] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.257484 4162 raft_consensus.cc:374] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.257814 4162 raft_consensus.cc:729] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.258296 4162 consensus_queue.cc:260] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.261337 3890 raft_consensus.cc:2454] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.262867 4162 ts_tablet_manager.cc:1428] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:21.263285 4169 raft_consensus.cc:2229] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.261812 3892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.265333 3676 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.269132 4169 pending_rounds.cc:62] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:21.269433 4924 raft_consensus.cc:2781] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.269946 4924 raft_consensus.cc:686] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.270449 4924 consensus_queue.cc:237] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:21.274907 4169 pending_rounds.cc:68] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400125563219968 op_type: NO_OP noop_request { }
I20260811 18:57:21.275220 4169 raft_consensus.cc:2866] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:21.276104 4169 raft_consensus.cc:2258] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.277518 4169 ts_tablet_manager.cc:1905] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.282588 4162 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.282750 3892 raft_consensus.cc:2454] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.298874 4163 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.306216 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 289e607f10e14a2983bc6ecc93ce9bbc
I20260811 18:57:21.307497 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7ddb21035302480fbfaf4738e6229f2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.307749 3676 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:21.308351 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ddb21035302480fbfaf4738e6229f2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.309092 4729 raft_consensus.cc:2781] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.309240 4729 raft_consensus.cc:482] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.309284 4163 ts_tablet_manager.cc:1397] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.616s user 0.001s sys 0.015s
I20260811 18:57:21.309536 4729 raft_consensus.cc:3037] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.310883 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_632b7ba5285744ee8f4bdc969820dae6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.311884 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_632b7ba5285744ee8f4bdc969820dae6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.320417 4729 raft_consensus.cc:504] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.320773 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d81e93739f44d9e99b6ebc647e1d0e4 consensus state has no leader
I20260811 18:57:21.321645 4729 leader_election.cc:288] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.321684 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_22bccdb99d2b4a87be82af7e1e97bebd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.322654 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22bccdb99d2b4a87be82af7e1e97bebd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.324000 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3e16ca699564a61a49c4cb989ad865f" } new_table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" modify_external_catalogs: true
I20260811 18:57:21.326915 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:21.337139 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:21.338034 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fce143fe6d0849a0a3af8939df804986" } new_table_name: "default.Table_ddcc89fb831040ed9752150dc2ad4b51" modify_external_catalogs: true
I20260811 18:57:21.341702 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 289e607f10e14a2983bc6ecc93ce9bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.349556 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c37e3fb34b3f446d854006b05dbe5a94" } new_table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" modify_external_catalogs: true
I20260811 18:57:21.352083 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" } modify_external_catalogs: true
I20260811 18:57:21.352550 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2d0a89e435ec4d47a5a6133147e03564" } modify_external_catalogs: true
I20260811 18:57:21.354149 4169 ts_tablet_manager.cc:1918] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.354713 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c" } modify_external_catalogs: true
I20260811 18:57:21.355193 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74385eb07bdb4cb39d7783a47ae5781c" } modify_external_catalogs: true
W20260811 18:57:21.359277 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 50939.5us
W20260811 18:57:21.368247 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 76308.3us
I20260811 18:57:21.374262 4836 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.374755 4836 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.382902 4836 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.387166 4162 tablet_bootstrap.cc:654] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.387681 4169 log.cc:1196] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/987b9ac6859444e3a86c56af786cc84e
I20260811 18:57:21.390566 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494939a4eb5a45d6a123d174ad513bcb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.396344 3888 raft_consensus.cc:3037] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.393919 4163 raft_consensus.cc:348] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.404048 3888 raft_consensus.cc:2454] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.404874 4163 raft_consensus.cc:374] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.408882 4167 ts_tablet_manager.cc:1918] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:21.410498 4163 raft_consensus.cc:729] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.411132 4163 consensus_queue.cc:260] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.412856 4163 ts_tablet_manager.cc:1428] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.103s user 0.003s sys 0.001s
I20260811 18:57:21.413475 4163 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.414551 4167 log.cc:1196] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d3d6daff4b7540309ff58755758e9752
I20260811 18:57:21.416272 4167 ts_tablet_manager.cc:1939] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.418629 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 364) took 3380 ms. Trace:
I20260811 18:57:21.419832 4162 tablet_bootstrap.cc:492] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.420281 4162 ts_tablet_manager.cc:1397] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.138s user 0.005s sys 0.005s
I20260811 18:57:21.420799 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.421283 3888 raft_consensus.cc:2454] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.421864 4162 raft_consensus.cc:348] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.423517 4914 consensus_queue.cc:1035] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.425132 3676 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.426760 4836 raft_consensus.cc:2781] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.426986 4836 raft_consensus.cc:686] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.427445 4836 consensus_queue.cc:237] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.428535 4169 ts_tablet_manager.cc:1939] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.429355 4986 consensus_queue.cc:1035] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.459882 4162 raft_consensus.cc:374] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.460147 4162 raft_consensus.cc:729] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.460774 4162 consensus_queue.cc:260] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.462133 4162 ts_tablet_manager.cc:1428] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.042s user 0.002s sys 0.001s
I20260811 18:57:21.463073 4162 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.469717 4167 rpcz_store.cc:276] 0811 18:57:18.037862 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.037931 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:21.247966 (+3210035us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:21.247985 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.395870 (+147885us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.408602 (+ 12732us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.418570 (+ 9968us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3209700}
I20260811 18:57:21.470894 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d3d6daff4b7540309ff58755758e9752 (table default.Table_923c4369260d436fafb1aff17a2ec8d4 [id=d3bfa725726d4ed5b1c34b2b9bc02c6d]) successfully deleted
I20260811 18:57:21.471084 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 360) took 3829 ms. Trace:
I20260811 18:57:21.471465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 365) took 3331 ms. Trace:
I20260811 18:57:21.471378 4169 rpcz_store.cc:276] 0811 18:57:17.641643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.642049 (+ 406us) service_pool.cc:227] Handling call
0811 18:57:21.245775 (+3603726us) ts_tablet_manager.cc:1145] Deleting tablet 987b9ac6859444e3a86c56af786cc84e
0811 18:57:21.245793 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.290934 (+ 45141us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.353894 (+ 62960us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.470993 (+117099us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3603176}
I20260811 18:57:21.471747 4167 rpcz_store.cc:276] 0811 18:57:18.139835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.139907 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:21.471039 (+3331132us) ts_tablet_manager.cc:1145] Deleting tablet 38e24f8d4522466d81605ea37fa7064a
0811 18:57:21.471061 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.471312 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3329440}
I20260811 18:57:21.472288 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 361) took 3830 ms. Trace:
I20260811 18:57:21.472556 4167 tablet_replica.cc:330] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.472515 4169 rpcz_store.cc:276] 0811 18:57:17.641984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.642158 (+ 174us) service_pool.cc:227] Handling call
0811 18:57:21.471998 (+3829840us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:21.472015 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.472180 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3829076}
I20260811 18:57:21.473026 4167 raft_consensus.cc:2229] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:21.473079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 58c515c153df4317b3fb427983ee15e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58c515c153df4317b3fb427983ee15e0 already in progress: creating tablet
I20260811 18:57:21.473331 4167 raft_consensus.cc:2258] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.474546 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 362) took 3814 ms. Trace:
I20260811 18:57:21.474809 4169 rpcz_store.cc:276] 0811 18:57:17.659953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.660016 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:21.474291 (+3814275us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:21.474315 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.474456 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3814006}
I20260811 18:57:21.479290 4163 tablet_bootstrap.cc:654] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.479775 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.480997 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f" } modify_external_catalogs: true
I20260811 18:57:21.481349 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d56bc2fd31fd443b93f4c38bf94b6e6f" } modify_external_catalogs: true
I20260811 18:57:21.482801 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_25065dcc52b34b50986b88e1376ee9bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.483944 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25065dcc52b34b50986b88e1376ee9bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.485503 3552 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27] alter complete (version 5)
I20260811 18:57:21.486356 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.487323 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.490762 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cd4eb198addc42f2b6d860ef805c589f" } modify_external_catalogs: true
I20260811 18:57:21.491025 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cd4eb198addc42f2b6d860ef805c589f" } modify_external_catalogs: true
I20260811 18:57:21.491554 4952 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.491768 4952 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.492532 4952 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:21.492805 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet ffa8a4a1c0cb4834be501bdd7d468472 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.494071 4162 tablet_bootstrap.cc:654] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.496507 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d2d7d986900244ddbc05dbe73a708dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2d7d986900244ddbc05dbe73a708dcf already in progress: creating tablet
W20260811 18:57:21.500661 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 452ee450778e4d33adfa7e2745b5d970 consensus state has no leader
I20260811 18:57:21.515954 4169 tablet_replica.cc:330] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:21.516520 4169 raft_consensus.cc:2229] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.517153 4169 raft_consensus.cc:2258] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.518244 4169 ts_tablet_manager.cc:1905] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.525032 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:21.525576 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d6daff4b7540309ff58755758e9752 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.525928 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 36724a83bf9844b9a27a88ca36a9bf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:21.526620 3824 leader_election.cc:341] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:21.527143 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 99553.8us
I20260811 18:57:21.527835 3676 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.528352 4986 raft_consensus.cc:2781] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.528503 4986 raft_consensus.cc:482] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.528614 4986 raft_consensus.cc:3037] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:21.534219 3678 leader_election.cc:334] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.623s (SENT)
I20260811 18:57:21.534487 3678 leader_election.cc:302] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:21.534775 3678 raft_consensus.cc:2670] T 2852f9e5f67c4025836018af553388a5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:21.536235 4986 raft_consensus.cc:504] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.537825 4986 leader_election.cc:288] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.540216 3891 raft_consensus.cc:1261] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:57:21.544750 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:21.548055 4167 ts_tablet_manager.cc:1905] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:21.548481 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0cccebfc54484a9b9abffcce4e459fb5 consensus state has no leader
I20260811 18:57:21.549079 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" } modify_external_catalogs: true
I20260811 18:57:21.549412 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8397de033b74ce9accaa4085f59d7e6" } modify_external_catalogs: true
I20260811 18:57:21.550055 4163 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.550724 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d8aea8dc31914052832bdc83d638461f
I20260811 18:57:21.552030 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_66121f6795f240ea88c01d7c0ae02e00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.552932 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66121f6795f240ea88c01d7c0ae02e00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.554037 4163 ts_tablet_manager.cc:1397] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.137s user 0.001s sys 0.005s
I20260811 18:57:21.554855 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf2816cc2dfe45f0815ce50a5a929324
I20260811 18:57:21.555996 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b78f30561e284a9596437b5fc75df24f" } new_table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" modify_external_catalogs: true
W20260811 18:57:21.557346 3824 leader_election.cc:334] T 7f640c60e81b4a369458b1497a269c9c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.877s (SENT)
I20260811 18:57:21.559432 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.561827 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb51e419dd24459caf3ba3adfbbd4517" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.562063 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet d8aea8dc31914052832bdc83d638461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.562307 3891 raft_consensus.cc:3037] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.565174 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d8aea8dc31914052832bdc83d638461f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.565688 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2816cc2dfe45f0815ce50a5a929324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.569653 3891 raft_consensus.cc:2454] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.572431 4995 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.572849 4995 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.575425 4995 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.581430 4924 consensus_queue.cc:1035] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:21.582284 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a01b26055fd433693bcb44191617f54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.583411 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a01b26055fd433693bcb44191617f54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.585955 4162 tablet_bootstrap.cc:492] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.585915 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ffa8a4a1c0cb4834be501bdd7d468472
I20260811 18:57:21.586314 4162 ts_tablet_manager.cc:1397] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.124s user 0.005s sys 0.003s
I20260811 18:57:21.587654 4952 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.587936 4952 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.588091 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd120e0ab3b34418960cefc5eb459cdb" } new_table_name: "default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28" modify_external_catalogs: true
I20260811 18:57:21.589052 4952 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:21.587663 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:21.590479 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:21.591570 3824 leader_election.cc:341] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:21.594041 3676 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.595405 4994 raft_consensus.cc:2781] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.595728 4994 raft_consensus.cc:686] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.597535 4994 consensus_queue.cc:237] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.605067 3678 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.751s (SENT)
I20260811 18:57:21.611268 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:21.620218 3824 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.023s (SENT)
I20260811 18:57:21.620633 3824 leader_election.cc:302] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:21.621223 4952 raft_consensus.cc:2726] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:21.621631 3676 leader_election.cc:341] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:21.598373 4162 raft_consensus.cc:348] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.631472 4162 raft_consensus.cc:374] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.633016 4163 raft_consensus.cc:348] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.633811 4163 raft_consensus.cc:374] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.634128 4163 raft_consensus.cc:729] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.636683 4162 raft_consensus.cc:729] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:21.647855 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2605007cc2454599868f44c717deb2d2 has no consensus state
I20260811 18:57:21.648764 3548 catalog_manager.cc:6565] ReplaceTablet: tablet ffa8a4a1c0cb4834be501bdd7d468472 deleted and replaced by tablet 07a78361bbc44fcf86985f9f20c5665b
I20260811 18:57:21.648178 4162 consensus_queue.cc:260] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.637319 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:21.655323 4163 consensus_queue.cc:260] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.657229 4162 ts_tablet_manager.cc:1428] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.062s user 0.004s sys 0.000s
I20260811 18:57:21.657680 4162 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:21.661321 4914 raft_consensus.cc:482] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.661675 4914 raft_consensus.cc:504] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.661732 5008 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:21.665838 4914 leader_election.cc:288] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.666980 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2a926bfdbff42feaa7cfed27e0c04a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.667271 4163 ts_tablet_manager.cc:1428] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.113s user 0.000s sys 0.004s
W20260811 18:57:21.667783 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.670488 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_50769590dc5544e0a551369ba2c9768c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.671187 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50769590dc5544e0a551369ba2c9768c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.671324 4163 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.672287 5010 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:21.677374 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_533afa7d5e834d5cb281ddc9619575f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:21.678085 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_533afa7d5e834d5cb281ddc9619575f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:21.681936 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb82243777a9494b96959ced0c24de5a consensus state has no leader
I20260811 18:57:21.683516 4167 ts_tablet_manager.cc:1918] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:21.684011 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "125757b8c1a6448b841c63339f392339" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:21.684531 3890 raft_consensus.cc:2454] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:21.687963 4167 log.cc:1196] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a30bb384136f4a169a5751655437d5a3
I20260811 18:57:21.688827 4163 tablet_bootstrap.cc:654] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.688891 4167 ts_tablet_manager.cc:1939] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:21.689889 3676 leader_election.cc:302] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.690459 4914 raft_consensus.cc:2781] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:21.690783 4914 raft_consensus.cc:482] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:21.691061 4914 raft_consensus.cc:3037] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.695801 4914 raft_consensus.cc:504] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.697700 4914 leader_election.cc:288] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:21.703321 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd82f86a4579497fb4e50639ea1bf581"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.704066 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 366) took 3529 ms. Trace:
I20260811 18:57:21.704330 4167 rpcz_store.cc:276] 0811 18:57:18.160437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.163999 (+ 3562us) service_pool.cc:227] Handling call
0811 18:57:21.472407 (+3308408us) ts_tablet_manager.cc:1145] Deleting tablet a30bb384136f4a169a5751655437d5a3
0811 18:57:21.472424 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.601304 (+128880us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.648085 (+ 46781us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.689791 (+ 41706us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307633}
W20260811 18:57:21.704670 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd82f86a4579497fb4e50639ea1bf581 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.705143 4167 tablet_replica.cc:330] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:21.705592 4167 raft_consensus.cc:2229] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.706034 4167 raft_consensus.cc:2258] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.706816 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a30bb384136f4a169a5751655437d5a3 (table default.Table_e159333f7174424eaf413e0a07c149e9 [id=a1560b822851411892f110b6c2e7b05f]) successfully deleted
I20260811 18:57:21.707576 4167 ts_tablet_manager.cc:1905] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.713145 4162 tablet_bootstrap.cc:654] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:21.719604 3824 leader_election.cc:334] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.111s (SENT)
W20260811 18:57:21.724287 3824 leader_election.cc:334] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.119s (SENT)
I20260811 18:57:21.734745 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "125757b8c1a6448b841c63339f392339" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:21.735396 3890 raft_consensus.cc:3037] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:21.741257 3890 raft_consensus.cc:2454] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:21.742286 3676 leader_election.cc:302] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:21.743057 4914 raft_consensus.cc:2781] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:21.743369 4914 raft_consensus.cc:686] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:21.744843 4914 consensus_queue.cc:237] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.752562 4163 tablet_bootstrap.cc:492] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:21.754065 4163 ts_tablet_manager.cc:1397] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.021s
I20260811 18:57:21.755488 4163 raft_consensus.cc:348] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.755862 4163 raft_consensus.cc:374] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.756105 4163 raft_consensus.cc:729] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.756491 4163 consensus_queue.cc:260] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.759487 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_ddcc89fb831040ed9752150dc2ad4b51 [id=bc887ab2039c4ff0a9c3ff318bf243e4], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:21.760928 4163 ts_tablet_manager.cc:1428] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:21.761386 4163 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:21.766891 4163 tablet_bootstrap.cc:654] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:21.773136 4162 tablet_bootstrap.cc:492] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:21.773507 4162 ts_tablet_manager.cc:1397] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.116s user 0.006s sys 0.003s
I20260811 18:57:21.774955 4162 raft_consensus.cc:348] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.775048 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.775404 4162 raft_consensus.cc:374] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:21.775550 4162 raft_consensus.cc:729] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:21.776014 4162 consensus_queue.cc:260] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.776844 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.784148 3872 tablet_service.cc:1467] Processing CreateTablet for tablet cdb92902c2584751af77d3117f52cb4e (DEFAULT_TABLE table=default.Table_d3652d29236a4b12b7abd44659c70311 [id=868b4bfe43bb47e3ae5a5b5308ff85bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.772903 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_853572d26c9642b9b8ad616f574301dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:21.789793 3713 tablet_service.cc:1467] Processing CreateTablet for tablet cdb92902c2584751af77d3117f52cb4e (DEFAULT_TABLE table=default.Table_d3652d29236a4b12b7abd44659c70311 [id=868b4bfe43bb47e3ae5a5b5308ff85bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.790591 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb92902c2584751af77d3117f52cb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.790820 3861 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6378d1219433cb724148f1ae74b25 (DEFAULT_TABLE table=default.Table_cee8eb42df9b40cd85eb35169133787a [id=b8fedf26521040b1bb58aa7edb6373c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.790279 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 (DEFAULT_TABLE table=default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.794507 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 (DEFAULT_TABLE table=default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.792443 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:21.795451 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302cf69ea7c14fdaac72f6a9a9aff113. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.797793 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_853572d26c9642b9b8ad616f574301dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:21.798573 3706 tablet_service.cc:1467] Processing CreateTablet for tablet f5910ff87f9e4a7e8fbf877adf8c7d56 (DEFAULT_TABLE table=default.Table_4f20eabb4925482bbf71fe270dde24af [id=f7310226124846a29f01f43aff0bafcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.799196 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5910ff87f9e4a7e8fbf877adf8c7d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.800349 3856 tablet_service.cc:1467] Processing CreateTablet for tablet f5910ff87f9e4a7e8fbf877adf8c7d56 (DEFAULT_TABLE table=default.Table_4f20eabb4925482bbf71fe270dde24af [id=f7310226124846a29f01f43aff0bafcb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:21.800846 3678 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.165s (SENT)
I20260811 18:57:21.801281 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2605007cc2454599868f44c717deb2d2 (DEFAULT_TABLE table=default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4 [id=1323eccbf4b04ee8b1a819101b8ad9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.801888 3865 tablet_service.cc:1467] Processing CreateTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 (DEFAULT_TABLE table=default.Table_886df95cce4342f998d27c8661df890e [id=abbf777ec3574c3e9fe6c01c4073fd57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.803830 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_25065dcc52b34b50986b88e1376ee9bc" } new_table_name: "default.Table_ed61380b12e548ce843c5056342085b9" modify_external_catalogs: true
I20260811 18:57:21.804957 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 2605007cc2454599868f44c717deb2d2 (DEFAULT_TABLE table=default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4 [id=1323eccbf4b04ee8b1a819101b8ad9c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.805523 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2605007cc2454599868f44c717deb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.806639 3715 tablet_service.cc:1467] Processing CreateTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 (DEFAULT_TABLE table=default.Table_886df95cce4342f998d27c8661df890e [id=abbf777ec3574c3e9fe6c01c4073fd57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.806691 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 90576c068ce4468a9c572307dcb79b10 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.807765 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa95131ce50943e3b0c6b5bdd5a30939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.812186 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 90576c068ce4468a9c572307dcb79b10 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.812858 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90576c068ce4468a9c572307dcb79b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.815486 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 (DEFAULT_TABLE table=default.Table_680160aaac0a4c8da66634dc41d6abb9 [id=8e026fcb9e9d456f8b9326a3b4712e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.816439 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb30e64e86040098dcee5fc2e9b7266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.816972 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 (DEFAULT_TABLE table=default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 [id=f9a016e11f694e16b5b75c9a742f6f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.817546 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f389d2b7ff354ac4bab5d68b9ff201c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.819988 3862 tablet_service.cc:1467] Processing CreateTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 (DEFAULT_TABLE table=default.Table_7bc942d59fd34beda3b8fb9e03e84cb2 [id=f9a016e11f694e16b5b75c9a742f6f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.815585 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6378d1219433cb724148f1ae74b25 (DEFAULT_TABLE table=default.Table_cee8eb42df9b40cd85eb35169133787a [id=b8fedf26521040b1bb58aa7edb6373c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.819381 3863 tablet_service.cc:1467] Processing CreateTablet for tablet fb878de2c0d94408aa6e98447bd7e958 (DEFAULT_TABLE table=default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.821161 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa95131ce50943e3b0c6b5bdd5a30939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.826776 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6378d1219433cb724148f1ae74b25. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.816556 3678 leader_election.cc:334] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.648s (SENT)
I20260811 18:57:21.831161 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fb878de2c0d94408aa6e98447bd7e958 (DEFAULT_TABLE table=default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.832880 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb878de2c0d94408aa6e98447bd7e958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.833227 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb878de2c0d94408aa6e98447bd7e958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.833791 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6378d1219433cb724148f1ae74b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.834745 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 (DEFAULT_TABLE table=default.Table_680160aaac0a4c8da66634dc41d6abb9 [id=8e026fcb9e9d456f8b9326a3b4712e99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.836369 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 7216c095cc03424bbbe97a52a9600ebb (DEFAULT_TABLE table=default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 [id=cf2ed3c747254c8981e9e04fa0a40d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.837041 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7216c095cc03424bbbe97a52a9600ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.838125 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 6432abd4fc5c4f9ca95971ca4795e1cf from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:21.838498 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 7216c095cc03424bbbe97a52a9600ebb (DEFAULT_TABLE table=default.Table_d7fac7298cb540bc89b4fc9cb2651eb6 [id=cf2ed3c747254c8981e9e04fa0a40d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.839071 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eb30e64e86040098dcee5fc2e9b7266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.840094 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5910ff87f9e4a7e8fbf877adf8c7d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.840483 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f389d2b7ff354ac4bab5d68b9ff201c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.841400 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 55c1a7d44717415ca76a2d0b7e94394e (table default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed]). Expected version 1 got 0
I20260811 18:57:21.842692 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 866ed614511649459b22a9fce55220b2 (table default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd]). Expected version 2 got 0
I20260811 18:57:21.843377 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:21.847028 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 (DEFAULT_TABLE table=default.Table_4d5db1f0452e40eea01177ba93223341 [id=82ab42a0b1e448a39ec7d6a27771daa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.848591 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc1822d4b94081ad86a6177acb3afe (DEFAULT_TABLE table=default.Table_90566ec48eaf42c8be4c06681c3c0268 [id=fc2d04fe3bad4dbd9dccd38355861a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.849061 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2605007cc2454599868f44c717deb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.849776 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302cf69ea7c14fdaac72f6a9a9aff113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.841178 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7216c095cc03424bbbe97a52a9600ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.851284 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cc1822d4b94081ad86a6177acb3afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.843858 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 (DEFAULT_TABLE table=default.Table_4d5db1f0452e40eea01177ba93223341 [id=82ab42a0b1e448a39ec7d6a27771daa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.857719 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb92902c2584751af77d3117f52cb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.862573 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90576c068ce4468a9c572307dcb79b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.862941 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b922f2a71544e8a18533aa0a1ccb52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.868781 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b922f2a71544e8a18533aa0a1ccb52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.871519 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc1822d4b94081ad86a6177acb3afe (DEFAULT_TABLE table=default.Table_90566ec48eaf42c8be4c06681c3c0268 [id=fc2d04fe3bad4dbd9dccd38355861a79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.872157 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21cc1822d4b94081ad86a6177acb3afe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:21.874583 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.115s (SENT) [suppressed 27 similar messages]
I20260811 18:57:21.879936 4169 ts_tablet_manager.cc:1918] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:21.880400 4169 log.cc:1196] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9c18120da2d245f495c52a670047760d
I20260811 18:57:21.882962 4169 ts_tablet_manager.cc:1939] T 9c18120da2d245f495c52a670047760d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:21.883328 3857 tablet_service.cc:1467] Processing CreateTablet for tablet f06e5b6a30764caa978ba0022ca1d6bc (DEFAULT_TABLE table=default.Table_c2e58ff1b0c44ee8b71812ae6399b4da [id=d81e384f7a3e426ca75fe55e69ef3051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.885023 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e5b6a30764caa978ba0022ca1d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.904174 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 363) took 4227 ms. Trace:
I20260811 18:57:21.904479 4169 rpcz_store.cc:276] 0811 18:57:17.664828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.664905 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:21.515767 (+3850862us) ts_tablet_manager.cc:1145] Deleting tablet 9c18120da2d245f495c52a670047760d
0811 18:57:21.515780 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.636601 (+120821us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.815003 (+178402us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:21.892221 (+ 77218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3850539}
I20260811 18:57:21.906541 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 366) took 4018 ms. Trace:
I20260811 18:57:21.906708 4169 rpcz_store.cc:276] 0811 18:57:17.887279 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.888560 (+ 1281us) service_pool.cc:227] Handling call
0811 18:57:21.905259 (+4016699us) ts_tablet_manager.cc:1145] Deleting tablet b1ce1042ae974f4dbac964bd7ea16a94
0811 18:57:21.905282 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.905405 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4016367}
I20260811 18:57:21.907960 4169 tablet_replica.cc:330] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:21.908445 4169 raft_consensus.cc:2229] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:21.933262 3706 tablet_service.cc:1467] Processing CreateTablet for tablet f06e5b6a30764caa978ba0022ca1d6bc (DEFAULT_TABLE table=default.Table_c2e58ff1b0c44ee8b71812ae6399b4da [id=d81e384f7a3e426ca75fe55e69ef3051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.947364 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e5b6a30764caa978ba0022ca1d6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.944550 4169 raft_consensus.cc:2258] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:21.948333 4990 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.954382 4914 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:21.958628 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7abca85e5a6d4b55a38c0f9e50f02f76 (DEFAULT_TABLE table=default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:21.961395 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abca85e5a6d4b55a38c0f9e50f02f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:21.961757 4169 ts_tablet_manager.cc:1905] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:21.953092 4990 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:21.844755 3547 catalog_manager.cc:5495] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:21.969667 4162 ts_tablet_manager.cc:1428] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.186s user 0.005s sys 0.000s
I20260811 18:57:21.966984 4914 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:21.971735 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 57896.7us
I20260811 18:57:21.974625 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9c18120da2d245f495c52a670047760d (table default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3 [id=221d836949c341a09235288dc5802137]) successfully deleted
W20260811 18:57:21.978080 3539 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 69655.4us
W20260811 18:57:21.978951 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b1ce1042ae974f4dbac964bd7ea16a94 because the tablet was not found. No further retry: Not found: Tablet not found: b1ce1042ae974f4dbac964bd7ea16a94
I20260811 18:57:21.996753 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_22bccdb99d2b4a87be82af7e1e97bebd" } new_table_name: "default.Table_c05597a50f6e4aea9c286eae0cf9a54d" modify_external_catalogs: true
I20260811 18:57:21.997507 4163 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.000730 4914 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.002606 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:22.018141 3678 leader_election.cc:334] T 807cee88c18340dbb23c51cad54ab56d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.684s (SENT)
I20260811 18:57:22.019021 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7abca85e5a6d4b55a38c0f9e50f02f76 (DEFAULT_TABLE table=default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.019659 3890 raft_consensus.cc:2454] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.031337 5010 tablet.cc:1722] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:22.033046 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.033448 3893 raft_consensus.cc:2454] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.033749 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abca85e5a6d4b55a38c0f9e50f02f76. 1 dirs total, 0 dirs full, 0 dirs failed
#10 0x7fc7bfa84211 in std::default_delete<kudu::tablet::Op>::operator()(kudu::tablet::Op*) const ../../../include/c++/8/bits/unique_ptr.h:81:2
W20260811 18:57:22.041298 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b40da8cb7324da9804e6fdafd8e8266 has no consensus state
I20260811 18:57:22.042116 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
I20260811 18:57:22.043120 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:22.045764 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e23d79bc476d496eaa713465bcfb1abf" } new_table_name: "default.Table_04ac8209b46549e3bb3f12971ab01b73" modify_external_catalogs: true
I20260811 18:57:22.047528 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" } modify_external_catalogs: true
#11 0x7fc7bfa76c5b in std::unique_ptr<kudu::tablet::Op, std::default_delete<kudu::tablet::Op> >::~unique_ptr() ../../../include/c++/8/bits/unique_ptr.h:274:4
I20260811 18:57:22.047878 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1a65cf33a6e4735bb456fd4b28c0ee4" } modify_external_catalogs: true
#12 0x7fc7bfa93862 in kudu::tablet::OpDriver::~OpDriver() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/ops/op_driver.h:304:16
I20260811 18:57:22.048859 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4927a23edf074354bdd4cf94e4b74eff" } modify_external_catalogs: true
I20260811 18:57:22.049135 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4927a23edf074354bdd4cf94e4b74eff" } modify_external_catalogs: true
I20260811 18:57:22.051409 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.052181 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dbfb0ccfdef4c018c7eca1d0452375a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#13 0x7fc7bfa9382a in kudu::RefCountedThreadSafe<kudu::tablet::OpDriver, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver> >::DeleteInternal(kudu::tablet::OpDriver const*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:153:44
#14 0x7fc7bfa93800 in kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver>::Destruct(kudu::tablet::OpDriver const*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:116:5
I20260811 18:57:22.062770 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.062865 4990 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.063443 3676 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.064208 3676 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.065017 4994 raft_consensus.cc:2781] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
#15 0x7fc7bfa937ce in kudu::RefCountedThreadSafe<kudu::tablet::OpDriver, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::OpDriver> >::Release() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:144:7
I20260811 18:57:22.066746 5008 tablet.cc:1722] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
#16 0x7fc7bfa76d37 in scoped_refptr<kudu::tablet::OpDriver>::~scoped_refptr() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/ref_counted.h:266:13
#17 0x7fc7bfa60add in kudu::tablet::TabletReplica::SubmitAlterSchema(std::unique_ptr<kudu::tablet::AlterSchemaOpState, std::default_delete<kudu::tablet::AlterSchemaOpState> >, kudu::MonoTime) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_replica.cc:627:1
W20260811 18:57:22.074508 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9342ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
#18 0x7fc7c232926d in kudu::tserver::TabletServiceAdminImpl::AlterSchema(kudu::tserver::AlterSchemaRequestPB const*, kudu::tserver::AlterSchemaResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1213:23
I20260811 18:57:22.077044 4163 ts_tablet_manager.cc:1397] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.237s user 0.088s sys 0.064s
I20260811 18:57:22.087790 4994 raft_consensus.cc:482] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.088097 4994 raft_consensus.cc:3037] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.088878 4162 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:22.091390 4990 raft_consensus.cc:2781] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.091754 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.093466 4990 raft_consensus.cc:482] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.093729 4990 raft_consensus.cc:3037] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.097509 4169 ts_tablet_manager.cc:1918] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.097738 4169 log.cc:1196] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:22.098037 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.098577 4169 ts_tablet_manager.cc:1939] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.098996 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 367) took 4211 ms. Trace:
I20260811 18:57:22.099159 4169 rpcz_store.cc:276] 0811 18:57:17.887441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:17.888919 (+ 1478us) service_pool.cc:227] Handling call
0811 18:57:21.907462 (+4018543us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:21.907476 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.024012 (+116536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.097322 (+ 73310us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.098957 (+ 1635us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4018272}
I20260811 18:57:22.079913 4163 raft_consensus.cc:348] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:22.102495 4163 raft_consensus.cc:374] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.102626 4163 raft_consensus.cc:729] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.102988 4163 consensus_queue.cc:260] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:22.104043 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.104880 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_971d1dda6c9c4e65a732e83f6528e2de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.106405 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7991dabe263b44f6b7b7d1d43b96ee44
I20260811 18:57:22.107429 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a01b26055fd433693bcb44191617f54" } new_table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" modify_external_catalogs: true
W20260811 18:57:22.108373 3824 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.845s (SENT)
I20260811 18:57:22.108681 3824 leader_election.cc:302] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:22.106206 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.109345 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f25ff8a9e3a944269e9799995c73ebe0 (table default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]) successfully deleted
I20260811 18:57:22.111052 4994 raft_consensus.cc:504] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.112793 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828993fb34bb4c62bce2a9322d7c992a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.113045 3893 raft_consensus.cc:3037] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.117755 3893 raft_consensus.cc:2454] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
W20260811 18:57:22.118992 3824 raft_consensus.cc:2670] T 8426ccbe530446fcbaa162a03cbb1892 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:22.121793 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.122615 4163 ts_tablet_manager.cc:1428] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:22.110244 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:22.123281 4163 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:22.123620 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 6432abd4fc5c4f9ca95971ca4795e1cf deleted and replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
I20260811 18:57:22.124006 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_10f91c342797480e9545a00a8d6be4a9" } new_table_name: "default.Table_26f305827594444882a9b3b84edba403" modify_external_catalogs: true
W20260811 18:57:22.114881 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae has no consensus state
I20260811 18:57:22.125389 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 377) took 4138 ms. Trace:
I20260811 18:57:22.125631 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ace00158b744bd8a97dc1c47fc6049c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.125612 4169 rpcz_store.cc:276] 0811 18:57:17.987057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.001297 (+ 14240us) service_pool.cc:227] Handling call
0811 18:57:22.125070 (+4123773us) ts_tablet_manager.cc:1145] Deleting tablet ebf7720e31d84ae2ab592da1e7a3132a
0811 18:57:22.125082 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.125310 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4118276}
I20260811 18:57:22.126322 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 378) took 4119 ms. Trace:
I20260811 18:57:22.126629 4169 rpcz_store.cc:276] 0811 18:57:18.006401 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.006476 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:22.126067 (+4119591us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:22.126079 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.126204 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4119110}
I20260811 18:57:22.127424 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 379) took 3988 ms. Trace:
I20260811 18:57:22.127642 4169 rpcz_store.cc:276] 0811 18:57:18.139084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.139174 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:22.127211 (+3988037us) ts_tablet_manager.cc:1145] Deleting tablet 38e24f8d4522466d81605ea37fa7064a
0811 18:57:22.127222 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.127355 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3987692}
W20260811 18:57:22.128401 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ebf7720e31d84ae2ab592da1e7a3132a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf7720e31d84ae2ab592da1e7a3132a already in progress: creating tablet
I20260811 18:57:22.128575 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.128782 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f75dc515d7f3473b8e43f20255a70e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f75dc515d7f3473b8e43f20255a70e3e already in progress: creating tablet
I20260811 18:57:22.129158 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 380) took 3905 ms. Trace:
I20260811 18:57:22.116581 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.129386 4169 rpcz_store.cc:276] 0811 18:57:18.223715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.223930 (+ 215us) service_pool.cc:227] Handling call
0811 18:57:22.128831 (+3904901us) ts_tablet_manager.cc:1145] Deleting tablet 4a64df7c295c4d9baded1b63a9980581
0811 18:57:22.128841 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.129033 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3841411}
W20260811 18:57:22.130069 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_056c759cfe3543f7a43de5b2fa3a4160 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.130599 3676 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.131206 4995 raft_consensus.cc:2781] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:22.117180 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 330801031cc74eb2a5bc2caf53a18ead consensus state has no leader
I20260811 18:57:22.132092 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.133272 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8554df7eb5f4fbbbe828834c1d45f25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.119803 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]). Expected version 1 got 0
I20260811 18:57:22.134684 4995 raft_consensus.cc:686] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.135143 4995 consensus_queue.cc:237] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.122771 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9ec4bba841dd446aa3db751358ce7f32
I20260811 18:57:22.136783 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e863da40303e45bda8d9464f8ec9b8a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.137385 4836 raft_consensus.cc:482] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:22.137553 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e863da40303e45bda8d9464f8ec9b8a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.140038 4836 raft_consensus.cc:504] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.141139 4836 leader_election.cc:288] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.142189 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a08e6dbf04e3fa58689ee0cd45db3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.142756 3890 raft_consensus.cc:2454] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.144872 4994 leader_election.cc:288] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.145278 3676 leader_election.cc:302] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:22.145298 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4a64df7c295c4d9baded1b63a9980581 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a64df7c295c4d9baded1b63a9980581 already in progress: creating tablet
I20260811 18:57:22.145752 4163 tablet_bootstrap.cc:654] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.148195 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.150566 4994 raft_consensus.cc:2781] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.150919 4994 raft_consensus.cc:482] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.152609 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.153760 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed948e1e9ee14cffae82bcd5517709f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.156536 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 59108us
I20260811 18:57:22.157863 4994 raft_consensus.cc:3037] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
#19 0x7fc7bc95c786 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_5::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:241:13
I20260811 18:57:22.160414 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 381) took 3868 ms. Trace:
I20260811 18:57:22.160796 4169 rpcz_store.cc:276] 0811 18:57:18.261588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.263172 (+ 1584us) service_pool.cc:227] Handling call
0811 18:57:22.129923 (+3866751us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:22.129932 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.130204 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3812639}
I20260811 18:57:22.165436 4990 raft_consensus.cc:504] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:22.167222 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet eb8f5b87062349c4b19ec99fa0175f24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb8f5b87062349c4b19ec99fa0175f24 already in progress: creating tablet
#20 0x7fc7bc95c4b5 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_5>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:22.168682 4990 leader_election.cc:288] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.169492 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef47211aa8864d13a55551da47804c19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.170069 3890 raft_consensus.cc:3037] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.176676 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.177359 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.178000 4169 tablet_replica.cc:330] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:22.178133 3678 leader_election.cc:334] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.176s (SENT)
I20260811 18:57:22.178529 3678 leader_election.cc:302] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:22.178594 4169 raft_consensus.cc:2229] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.179004 4169 raft_consensus.cc:2258] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:22.179520 3678 raft_consensus.cc:2670] T 8af3e74889b64c408c9ed9c54916f890 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:22.180414 4169 ts_tablet_manager.cc:1905] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.181392 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.178648 3890 raft_consensus.cc:2454] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.183259 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.183745 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 828993fb34bb4c62bce2a9322d7c992a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.183602 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7991dabe263b44f6b7b7d1d43b96ee44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.187703 3676 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.188522 4914 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.188766 4914 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.193113 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:22.193653 3890 raft_consensus.cc:2454] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:22.199685 4163 tablet_bootstrap.cc:492] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.199971 4163 ts_tablet_manager.cc:1397] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.000s sys 0.006s
I20260811 18:57:22.200810 3676 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.201577 4163 raft_consensus.cc:348] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.202054 4163 raft_consensus.cc:374] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.206391 4163 raft_consensus.cc:729] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.206852 4163 consensus_queue.cc:260] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.212363 4914 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.215111 4163 ts_tablet_manager.cc:1428] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:22.218609 4986 raft_consensus.cc:2781] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:22.218966 4986 raft_consensus.cc:482] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:22.222010 4986 raft_consensus.cc:3037] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.223285 4990 raft_consensus.cc:2781] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.226781 4163 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:22.229395 4986 raft_consensus.cc:504] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.234673 4990 raft_consensus.cc:686] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.236732 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3021c51525846e199f279f988248af0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.237437 3890 raft_consensus.cc:3037] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:22.246861 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3030c31bc2b24ee296f29b5d5aa402ff has no consensus state
I20260811 18:57:22.247357 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet e14f8717bcd94d6d88295fd8be130ca5 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:22.248013 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:22.248334 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 3263da75f0d84528b7c8cc55dcaeaa46 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.249191 4986 leader_election.cc:288] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.249385 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1204e0b30877462982c245f439f0bae4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.250314 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1204e0b30877462982c245f439f0bae4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.250651 4163 tablet_bootstrap.cc:654] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.251312 4167 ts_tablet_manager.cc:1918] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:22.251565 4167 log.cc:1196] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cf285cdf5235451bb1531cb294017101
I20260811 18:57:22.256906 4169 ts_tablet_manager.cc:1918] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.257273 4169 log.cc:1196] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:22.258383 4167 ts_tablet_manager.cc:1939] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:22.259196 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 367) took 4072 ms. Trace:
I20260811 18:57:22.259550 4169 ts_tablet_manager.cc:1939] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.259717 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:22.260839 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 0ace00158b744bd8a97dc1c47fc6049c deleted and replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b
I20260811 18:57:22.260640 4167 rpcz_store.cc:276] 0811 18:57:18.186313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.186591 (+ 278us) service_pool.cc:227] Handling call
0811 18:57:21.704950 (+3518359us) ts_tablet_manager.cc:1145] Deleting tablet cf285cdf5235451bb1531cb294017101
0811 18:57:21.704970 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:21.761665 (+ 56695us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.176240 (+414575us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.259128 (+ 82888us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3517986}
I20260811 18:57:22.261289 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 368) took 4035 ms. Trace:
I20260811 18:57:22.261536 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a93638c401d94d3d8dad242dd4c9b60a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:22.263134 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cf285cdf5235451bb1531cb294017101 (table default.Table_d35faa013d8c45e29cddf10c7c62e27d [id=90bfe7bd9d95476f9fd0c647b05e4125]) successfully deleted
I20260811 18:57:22.263170 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c with delete_type TABLET_DATA_DELETED (replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.264717 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a93638c401d94d3d8dad242dd4c9b60a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.264914 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 388) took 3726 ms. Trace:
I20260811 18:57:22.265096 4169 rpcz_store.cc:276] 0811 18:57:18.538208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539790 (+ 1582us) service_pool.cc:227] Handling call
0811 18:57:22.177855 (+3638065us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:22.177881 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.217514 (+ 39633us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.256675 (+ 39161us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.264845 (+ 8170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3632993}
I20260811 18:57:22.271499 4994 raft_consensus.cc:504] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.271963 4169 tablet_replica.cc:330] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:22.273036 4990 consensus_queue.cc:237] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.275806 4169 raft_consensus.cc:2229] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.276712 4169 raft_consensus.cc:2258] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.278296 4169 ts_tablet_manager.cc:1905] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.283082 3890 raft_consensus.cc:2454] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.284286 3676 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:22.284778 3678 leader_election.cc:334] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.750s (SENT)
I20260811 18:57:22.285557 4986 raft_consensus.cc:2781] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.286001 4986 raft_consensus.cc:686] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.286554 4986 consensus_queue.cc:237] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.290823 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619" } modify_external_catalogs: true
I20260811 18:57:22.291208 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a303b89a5ef54a7ea92f2bef6298d619" } modify_external_catalogs: true
I20260811 18:57:22.298524 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a08e6dbf04e3fa58689ee0cd45db3" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:22.298902 3893 raft_consensus.cc:3037] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:22.299592 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db899a033c3144559d1d5c7bf1d92fbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.300729 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db899a033c3144559d1d5c7bf1d92fbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.301765 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 374c94a12e374d0e82180421c80d92d0 (DEFAULT_TABLE table=default.Table_50769590dc5544e0a551369ba2c9768c [id=c8432eabc0be4256880d9eb6a5afde61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.302614 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 374c94a12e374d0e82180421c80d92d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.303505 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f713f72dfeb48658564074b098a97d9" } new_table_name: "default.Table_5727f2e5abe54ecaaeec9a52f1bd78da" modify_external_catalogs: true
I20260811 18:57:22.304345 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ace00158b744bd8a97dc1c47fc6049c with delete_type TABLET_DATA_DELETED (replaced by tablet bef0f6d56cda40249f12b25a8eb3ab6b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.304601 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 374c94a12e374d0e82180421c80d92d0 (DEFAULT_TABLE table=default.Table_50769590dc5544e0a551369ba2c9768c [id=c8432eabc0be4256880d9eb6a5afde61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.305176 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 374c94a12e374d0e82180421c80d92d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.308602 4162 tablet_bootstrap.cc:654] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.304575 4167 rpcz_store.cc:276] 0811 18:57:18.225745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.225819 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:22.261082 (+4035263us) ts_tablet_manager.cc:1145] Deleting tablet 4a64df7c295c4d9baded1b63a9980581
0811 18:57:22.261095 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.261249 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4034940}
I20260811 18:57:22.310750 3857 tablet_service.cc:1467] Processing CreateTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 (DEFAULT_TABLE table=default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 [id=77b12e379e234e6f8285ebe4c3c46aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.311409 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff (DEFAULT_TABLE table=default.Table_45b8aa8da0974fe59719c2d87bb95773 [id=98a8126304dd4be4a2575ef1e86cbeed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.311945 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea579c33f8b14e9fa7fa389ce7f45931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.312584 4994 leader_election.cc:288] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:22.313601 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3030c31bc2b24ee296f29b5d5aa402ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.315258 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8f42fcd065448e5a6332ff5512645ed" } modify_external_catalogs: true
I20260811 18:57:22.315954 3864 tablet_service.cc:1467] Processing CreateTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae (DEFAULT_TABLE table=default.Table_c05597a50f6e4aea9c286eae0cf9a54d [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.316813 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.317701 3870 tablet_service.cc:1467] Processing CreateTablet for tablet c77882ff8d974057a61f037f1943f58f (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.317226 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4a64df7c295c4d9baded1b63a9980581 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a64df7c295c4d9baded1b63a9980581 already in progress: creating tablet
I20260811 18:57:22.318450 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8f42fcd065448e5a6332ff5512645ed" } modify_external_catalogs: true
I20260811 18:57:22.318794 3712 tablet_service.cc:1467] Processing CreateTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae (DEFAULT_TABLE table=default.Table_c05597a50f6e4aea9c286eae0cf9a54d [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.319064 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff (DEFAULT_TABLE table=default.Table_45b8aa8da0974fe59719c2d87bb95773 [id=98a8126304dd4be4a2575ef1e86cbeed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.319393 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.321201 3723 tablet_service.cc:1467] Processing CreateTablet for tablet c77882ff8d974057a61f037f1943f58f (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.321859 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77882ff8d974057a61f037f1943f58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.322546 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3030c31bc2b24ee296f29b5d5aa402ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.323518 3722 tablet_service.cc:1467] Processing CreateTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 (DEFAULT_TABLE table=default.Table_c2a926bfdbff42feaa7cfed27e0c04a0 [id=77b12e379e234e6f8285ebe4c3c46aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.324065 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea579c33f8b14e9fa7fa389ce7f45931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.325320 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c77882ff8d974057a61f037f1943f58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.331296 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2605007cc2454599868f44c717deb2d2
I20260811 18:57:22.332255 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 515d5b5afbd74ec5ab90a503aef2f4fb from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:22.332175 3555 catalog_manager.cc:5495] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.332913 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:22.333515 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_3c807c78cf1249abbf878e819573553d [id=1f86b4e3748c4cb7b34b2a24524dd1ed] alter complete (version 1)
I20260811 18:57:22.334542 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2605007cc2454599868f44c717deb2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.336131 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:22.336944 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 3263da75f0d84528b7c8cc55dcaeaa46 deleted and replaced by tablet 4dd203b7eb8643f493b4642c0396e258
I20260811 18:57:22.337809 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e14f8717bcd94d6d88295fd8be130ca5
I20260811 18:57:22.338024 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e51a08e6dbf04e3fa58689ee0cd45db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.338475 3546 catalog_manager.cc:6565] ReplaceTablet: tablet e14f8717bcd94d6d88295fd8be130ca5 deleted and replaced by tablet 649cb98f553a418b911e307fdaf49917
I20260811 18:57:22.338953 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.339382 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.340051 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.340368 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 2605007cc2454599868f44c717deb2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.342041 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.342504 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 369) took 4048 ms. Trace:
I20260811 18:57:22.342869 4167 rpcz_store.cc:276] 0811 18:57:18.264244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.264479 (+ 235us) service_pool.cc:227] Handling call
0811 18:57:22.310055 (+4045576us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:22.310069 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.313028 (+ 2959us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4045252}
I20260811 18:57:22.343571 4167 tablet_replica.cc:330] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:22.354327 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260811 18:57:22.359824 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7d747f03ad234805a4ea4ef60840d426 consensus state has no leader
I20260811 18:57:22.360096 3858 tablet_service.cc:1467] Processing CreateTablet for tablet c2b283002d0e4316844e482468f97c31 (DEFAULT_TABLE table=default.Table_5fef93c0a9b546199a3217a8b8bd67dd [id=65c745bb0fdf4b85bf0e3451c8a180b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.360225 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.360785 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2b283002d0e4316844e482468f97c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.361845 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a847581412494e34b2c76e605ff11772
W20260811 18:57:22.364388 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eb8f5b87062349c4b19ec99fa0175f24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb8f5b87062349c4b19ec99fa0175f24 already in progress: creating tablet
I20260811 18:57:22.366430 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 7d747f03ad234805a4ea4ef60840d426 (DEFAULT_TABLE table=default.Table_4f713f72dfeb48658564074b098a97d9 [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.371897 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e3a2005266804c6c94f0325437caa659"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.372718 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3a2005266804c6c94f0325437caa659 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.374382 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d747f03ad234805a4ea4ef60840d426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.375576 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_815d74faaf044b2f9000790283e989c9" } modify_external_catalogs: true
I20260811 18:57:22.375885 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_815d74faaf044b2f9000790283e989c9" } modify_external_catalogs: true
I20260811 18:57:22.378055 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.379345 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.379755 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7d747f03ad234805a4ea4ef60840d426 (DEFAULT_TABLE table=default.Table_4f713f72dfeb48658564074b098a97d9 [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.380529 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d747f03ad234805a4ea4ef60840d426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.383035 4167 raft_consensus.cc:2229] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.383478 4167 raft_consensus.cc:2258] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.383654 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 97997b0b3a5d46b487a9a33dcc30f9cd (DEFAULT_TABLE table=default.Table_ab9ea53bcc98448384f4fd3d30d2528f [id=e5b7ad7f9abb4ff190a1d4f32e468362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.384392 4167 ts_tablet_manager.cc:1905] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.385047 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97997b0b3a5d46b487a9a33dcc30f9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.386123 3893 raft_consensus.cc:2454] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:22.386837 3676 leader_election.cc:302] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:22.387691 4990 raft_consensus.cc:2781] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:22.389963 4994 consensus_queue.cc:1035] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:22.390314 3676 consensus_peers.cc:487] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f25ff8a9e3a944269e9799995c73ebe0. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.392596 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" } modify_external_catalogs: true
I20260811 18:57:22.392939 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d6c4d770dc949e78c1707ddbab71740" } modify_external_catalogs: true
I20260811 18:57:22.393704 4990 raft_consensus.cc:686] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:22.394066 4990 consensus_queue.cc:237] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.396134 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.397171 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e12ffa777d684216bfc19d52f2dad4be" } new_table_name: "default.Table_28fea07a2f434dc1997642a34e6b6e2b" modify_external_catalogs: true
I20260811 18:57:22.399113 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3294569c49f84a2291010174baee10af" } new_table_name: "default.Table_8076b90bff8d49c08328583af352ae7c" modify_external_catalogs: true
I20260811 18:57:22.401755 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c6e9da8dfb4a4c6c9275c092fae7d910
W20260811 18:57:22.409982 3824 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.012s (SENT)
W20260811 18:57:22.410643 3824 leader_election.cc:334] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 5.471s (SENT)
I20260811 18:57:22.412942 3824 leader_election.cc:302] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:22.413913 4952 raft_consensus.cc:2726] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:22.419685 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:22.427639 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.428550 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.429682 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1f60a286ac024c45bcb845227c5445c9 (DEFAULT_TABLE table=default.Table_7ddb21035302480fbfaf4738e6229f2b [id=466cb5f3f05048f3b2857ba8bf3426bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.430866 4162 tablet_bootstrap.cc:492] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:22.433416 4162 ts_tablet_manager.cc:1397] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.463s user 0.002s sys 0.010s
I20260811 18:57:22.435292 4162 raft_consensus.cc:348] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.435788 4162 raft_consensus.cc:374] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.435977 4162 raft_consensus.cc:729] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.436372 4162 consensus_queue.cc:260] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.437480 4162 ts_tablet_manager.cc:1428] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:22.440332 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f60a286ac024c45bcb845227c5445c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:22.442273 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
I20260811 18:57:22.443634 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 (DEFAULT_TABLE table=default.Table_badc11f1b0554e629ddb3f8a20e151e2 [id=e0494fbbb30942c2aa3fa5f264a3f93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.444283 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4af28c051e8243eaa73d4a5ed1abcea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.446015 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 25266bc7788f49f8af8eb9a948e61c8d (DEFAULT_TABLE table=default.Table_30ac5178633e44aeafd707e48d7c4cba [id=31b3d877b5da424eb8f4b3fd7086cda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.447064 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 838a9f3025a1466296844c218cedf4fa (DEFAULT_TABLE table=default.Table_8c94c9b20d784df0811302a8381a7301 [id=0052bf55538f46c5b17831eaa1cfad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.447579 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.448071 3705 tablet_service.cc:1467] Processing CreateTablet for tablet ea4ad482bde44d6bab5ec809054aaebd (DEFAULT_TABLE table=default.Table_fc78208e540445719638da3757163df5 [id=37f03604188246b2aedfa959d6346824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.448783 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4ad482bde44d6bab5ec809054aaebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.449703 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838a9f3025a1466296844c218cedf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.450552 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25266bc7788f49f8af8eb9a948e61c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.452754 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 97997b0b3a5d46b487a9a33dcc30f9cd (DEFAULT_TABLE table=default.Table_ab9ea53bcc98448384f4fd3d30d2528f [id=e5b7ad7f9abb4ff190a1d4f32e468362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.453414 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97997b0b3a5d46b487a9a33dcc30f9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.454082 3709 tablet_service.cc:1467] Processing CreateTablet for tablet c2b283002d0e4316844e482468f97c31 (DEFAULT_TABLE table=default.Table_5fef93c0a9b546199a3217a8b8bd67dd [id=65c745bb0fdf4b85bf0e3451c8a180b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.454672 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 494939a4eb5a45d6a123d174ad513bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.457214 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_af28562bfbf94af98e28afef24fda608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.458179 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af28562bfbf94af98e28afef24fda608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.461603 4163 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:22.454926 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2b283002d0e4316844e482468f97c31. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:22.471258 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:22.471983 4162 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:22.472586 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7a06761b8a145519b7bb645e7a73657" } new_table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" modify_external_catalogs: true
I20260811 18:57:22.474908 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_52e514412ff342e3b34ba958e863d4ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.475736 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52e514412ff342e3b34ba958e863d4ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.493422 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.494884 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 07a78361bbc44fcf86985f9f20c5665b (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.472379 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 (DEFAULT_TABLE table=default.Table_badc11f1b0554e629ddb3f8a20e151e2 [id=e0494fbbb30942c2aa3fa5f264a3f93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.474946 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1f60a286ac024c45bcb845227c5445c9 (DEFAULT_TABLE table=default.Table_7ddb21035302480fbfaf4738e6229f2b [id=466cb5f3f05048f3b2857ba8bf3426bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.475319 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 25266bc7788f49f8af8eb9a948e61c8d (DEFAULT_TABLE table=default.Table_30ac5178633e44aeafd707e48d7c4cba [id=31b3d877b5da424eb8f4b3fd7086cda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.484699 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 838a9f3025a1466296844c218cedf4fa (DEFAULT_TABLE table=default.Table_8c94c9b20d784df0811302a8381a7301 [id=0052bf55538f46c5b17831eaa1cfad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.485082 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 494939a4eb5a45d6a123d174ad513bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.486285 3865 tablet_service.cc:1467] Processing CreateTablet for tablet ea4ad482bde44d6bab5ec809054aaebd (DEFAULT_TABLE table=default.Table_fc78208e540445719638da3757163df5 [id=37f03604188246b2aedfa959d6346824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.501452 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4af28c051e8243eaa73d4a5ed1abcea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.502318 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.508996 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a78361bbc44fcf86985f9f20c5665b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.511420 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25266bc7788f49f8af8eb9a948e61c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.513926 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838a9f3025a1466296844c218cedf4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.514739 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f60a286ac024c45bcb845227c5445c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.513242 4986 consensus_queue.cc:1035] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260811 18:57:22.525353 3719 tablet_service.cc:1467] Processing CreateTablet for tablet d593aea288e14bb1a4c54360b260cffa (DEFAULT_TABLE table=default.Table_fd82f86a4579497fb4e50639ea1bf581 [id=20f5d41745b94d84a5796ce17ca05376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.526091 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d593aea288e14bb1a4c54360b260cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.526198 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 (DEFAULT_TABLE table=default.Table_ed61380b12e548ce843c5056342085b9 [id=b17ff3c80703482a9bd612ab9dacdd8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.529747 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f2f31331974f42c3a22b3ac21b690f82 (DEFAULT_TABLE table=default.Table_533afa7d5e834d5cb281ddc9619575f8 [id=c8a25582e429459eb7442d0517c93ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.531018 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f31331974f42c3a22b3ac21b690f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.534508 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e3a2005266804c6c94f0325437caa659" } new_table_name: "default.Table_9de9363efd764bd4ac323f86fd70658d" modify_external_catalogs: true
I20260811 18:57:22.534732 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4ad482bde44d6bab5ec809054aaebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.537076 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b7d4eb2831c407a989326cf56a7d8f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.538455 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b7d4eb2831c407a989326cf56a7d8f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.540904 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d18d57b020e046b8a069552ea63ae961"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.541797 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d18d57b020e046b8a069552ea63ae961 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.543820 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4fe6e07876144884a155b97fcbf9ed8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.544704 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fe6e07876144884a155b97fcbf9ed8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.548084 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:22.549309 3555 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0] alter complete (version 1)
I20260811 18:57:22.554701 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 45d6d76629d440129e4509fad38dde43 (DEFAULT_TABLE table=default.Table_99de6d48691743d798f56442d85d5be8 [id=0d95920bcace41f7873200170610c9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.557157 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 07a78361bbc44fcf86985f9f20c5665b (DEFAULT_TABLE table=default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.557142 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.558444 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_204c26eb8e554700a777796f2df5e7b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.559316 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_204c26eb8e554700a777796f2df5e7b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.562517 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b40da8cb7324da9804e6fdafd8e8266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.567406 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111),
I20260811 18:57:22.567716 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:22.569244 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8" } modify_external_catalogs: true
I20260811 18:57:22.569568 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3d490ceda4c745668e9deb77c4a07fc8" } modify_external_catalogs: true
I20260811 18:57:22.571672 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_655b56a4862144f1afd52a620c3715d3" } new_table_name: "default.Table_609ebcc1b6e4405782a26228865b9460" modify_external_catalogs: true
I20260811 18:57:22.573261 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ab47bc9432074c05b97dce613fcbcdae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.574002 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab47bc9432074c05b97dce613fcbcdae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.581557 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4dd203b7eb8643f493b4642c0396e258 has no consensus state
W20260811 18:57:22.582399 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ca801c79fc499ab50cfe76cdb03b00 has no consensus state
I20260811 18:57:22.584630 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_90566ec48eaf42c8be4c06681c3c0268" } modify_external_catalogs: true
I20260811 18:57:22.585047 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_90566ec48eaf42c8be4c06681c3c0268" } modify_external_catalogs: true
I20260811 18:57:22.587095 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.588030 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:22.591949 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1811e5f62a714f8388460dfcb1717e81 consensus state has no leader
I20260811 18:57:22.593472 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b20872d80c9545e5822bfc82d496ce97
I20260811 18:57:22.610100 4163 ts_tablet_manager.cc:1397] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.286s user 0.011s sys 0.004s
I20260811 18:57:22.617297 3860 tablet_service.cc:1467] Processing CreateTablet for tablet e32f59bdd0a54fcb9c76d7d9132d966c (DEFAULT_TABLE table=default.Table_9dd303f12b584056a04e385cc80faffe [id=25ed995924b2435daa8d514770e952f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.566406 3857 tablet_service.cc:1467] Processing CreateTablet for tablet f2f31331974f42c3a22b3ac21b690f82 (DEFAULT_TABLE table=default.Table_533afa7d5e834d5cb281ddc9619575f8 [id=c8a25582e429459eb7442d0517c93ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.618714 3555 catalog_manager.cc:5495] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.619377 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d6d76629d440129e4509fad38dde43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.620136 3555 catalog_manager.cc:5495] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:22.561483 3724 tablet_service.cc:1467] Processing CreateTablet for tablet e32f59bdd0a54fcb9c76d7d9132d966c (DEFAULT_TABLE table=default.Table_9dd303f12b584056a04e385cc80faffe [id=25ed995924b2435daa8d514770e952f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.568526 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f (DEFAULT_TABLE table=default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 [id=f1e29c96194e4ca0bc9b14c0f8811a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.569643 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 9948c33ed58b4160932be82ef9615ba5 (DEFAULT_TABLE table=default.Table_8214722ffbdc49bf98d5aa539aebed52 [id=bc5efeb04bb64cad932828d746ef6a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.591912 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.596185 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 125757b8c1a6448b841c63339f392339 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ac9bdc606be40558dfa33e1e7a8ebc7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.558146 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 (DEFAULT_TABLE table=default.Table_ed61380b12e548ce843c5056342085b9 [id=b17ff3c80703482a9bd612ab9dacdd8a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.630043 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 10988ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:22.631278 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e32f59bdd0a54fcb9c76d7d9132d966c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.639175 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:22.655861 3714 tablet_service.cc:1467] Processing CreateTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 (DEFAULT_TABLE table=default.Table_d8f42fcd065448e5a6332ff5512645ed [id=213a95b02ba64f3895f789564b1bc5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.671063 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 45d6d76629d440129e4509fad38dde43 (DEFAULT_TABLE table=default.Table_99de6d48691743d798f56442d85d5be8 [id=0d95920bcace41f7873200170610c9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.661737 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.676569 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d593aea288e14bb1a4c54360b260cffa (DEFAULT_TABLE table=default.Table_fd82f86a4579497fb4e50639ea1bf581 [id=20f5d41745b94d84a5796ce17ca05376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.684582 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a78361bbc44fcf86985f9f20c5665b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.688513 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d593aea288e14bb1a4c54360b260cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.694777 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f (DEFAULT_TABLE table=default.Table_b9115c298dfa4ecb970e76c7de2cd8e9 [id=f1e29c96194e4ca0bc9b14c0f8811a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.693259 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 9948c33ed58b4160932be82ef9615ba5 (DEFAULT_TABLE table=default.Table_8214722ffbdc49bf98d5aa539aebed52 [id=bc5efeb04bb64cad932828d746ef6a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.708067 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6009c9f6bbb0493ebe125120a1a47f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.720587 4167 ts_tablet_manager.cc:1918] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:22.725498 4162 tablet_bootstrap.cc:654] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:22.726486 4994 consensus_queue.cc:1035] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260811 18:57:22.728178 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b40e44a2f4940af891eddf6c3b6293c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.738647 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e9da8dfb4a4c6c9275c092fae7d910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.703323 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.755769 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 84e152c74f33463e83843b92c0783803 (DEFAULT_TABLE table=default.Table_632b7ba5285744ee8f4bdc969820dae6 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:22.772378 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 405773849fb94800be61ad8a7a6055a4 consensus state has no leader
I20260811 18:57:22.785250 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9948c33ed58b4160932be82ef9615ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.795826 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e152c74f33463e83843b92c0783803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.796855 4167 log.cc:1196] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5ae200f3814e431a8288c30fe94bffcf
I20260811 18:57:22.803898 4169 ts_tablet_manager.cc:1918] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:22.804296 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6009c9f6bbb0493ebe125120a1a47f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.806321 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.812279 4163 raft_consensus.cc:348] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.812422 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" } modify_external_catalogs: true
I20260811 18:57:22.812808 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b" } modify_external_catalogs: true
I20260811 18:57:22.815680 4169 log.cc:1196] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:22.819530 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.848317 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91d1c65c57764085a678b08ec4e1938b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.849229 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91d1c65c57764085a678b08ec4e1938b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.852560 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f31331974f42c3a22b3ac21b690f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.853116 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b4ff060dff9a4bf19670e304147315ac" } modify_external_catalogs: true
I20260811 18:57:22.853349 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b4ff060dff9a4bf19670e304147315ac" } modify_external_catalogs: true
I20260811 18:57:22.855033 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4fe6e07876144884a155b97fcbf9ed8d" } new_table_name: "default.Table_6914252b8edc40e79af29a5145201b08" modify_external_catalogs: true
I20260811 18:57:22.860067 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2108d07d209a411da92158a80c5d08ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:22.861200 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2108d07d209a411da92158a80c5d08ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:22.862978 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_886df95cce4342f998d27c8661df890e" } new_table_name: "default.Table_0a55918c905f4a4386f6529271fb60dc" modify_external_catalogs: true
I20260811 18:57:22.869374 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b40da8cb7324da9804e6fdafd8e8266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.874363 4994 consensus_queue.cc:1035] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260811 18:57:22.885382 4952 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.891321 4163 raft_consensus.cc:374] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:22.891268 4952 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.891850 4163 raft_consensus.cc:729] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:22.892204 4952 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:22.893306 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 94943cfbe41a47b3a36c81957a3adaeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:08 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:22.893784 3676 consensus_peers.cc:487] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.900017 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45d6d76629d440129e4509fad38dde43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.904083 4167 ts_tablet_manager.cc:1939] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:22.916716 3865 tablet_service.cc:1467] Processing CreateTablet for tablet c6e9da8dfb4a4c6c9275c092fae7d910 (DEFAULT_TABLE table=default.Table_d8f42fcd065448e5a6332ff5512645ed [id=213a95b02ba64f3895f789564b1bc5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.941574 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e32f59bdd0a54fcb9c76d7d9132d966c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.943265 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:22.953635 4986 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:22.954298 4169 ts_tablet_manager.cc:1939] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:22.955518 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 389) took 4417 ms. Trace:
I20260811 18:57:22.955705 4169 rpcz_store.cc:276] 0811 18:57:18.538387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.540022 (+ 1635us) service_pool.cc:227] Handling call
0811 18:57:22.269812 (+3729790us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:22.269829 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.288932 (+ 19103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.709610 (+420678us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.955468 (+245858us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3717478}
I20260811 18:57:22.956228 4169 tablet_replica.cc:330] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:22.956553 4169 raft_consensus.cc:2229] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:22.956853 4169 raft_consensus.cc:2258] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:22.957789 4169 ts_tablet_manager.cc:1905] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:22.957581 4163 consensus_queue.cc:260] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.958631 4163 ts_tablet_manager.cc:1428] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.348s user 0.005s sys 0.001s
W20260811 18:57:22.969825 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260811 18:57:22.936164 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:22.972826 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e9da8dfb4a4c6c9275c092fae7d910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:22.985975 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 370) took 4644 ms. Trace:
W20260811 18:57:22.987315 3676 consensus_peers.cc:487] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e215759d48c24d8cb99bbc3fc9009a66. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:22.993732 4986 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:22.997413 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 84e152c74f33463e83843b92c0783803 (DEFAULT_TABLE table=default.Table_632b7ba5285744ee8f4bdc969820dae6 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.008214 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b40e44a2f4940af891eddf6c3b6293c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.011576 4167 rpcz_store.cc:276] 0811 18:57:18.264406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.264849 (+ 443us) service_pool.cc:227] Handling call
0811 18:57:22.343352 (+4078503us) ts_tablet_manager.cc:1145] Deleting tablet 5ae200f3814e431a8288c30fe94bffcf
0811 18:57:22.343370 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:22.470630 (+127260us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.668476 (+197846us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:22.909320 (+240844us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30158,"tablet-delete.queue_time_us":4078253}
I20260811 18:57:23.016940 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.019747 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08b93745a5f44ce8bc61f090e881f952" } modify_external_catalogs: true
I20260811 18:57:23.020084 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_08b93745a5f44ce8bc61f090e881f952" } modify_external_catalogs: true
I20260811 18:57:23.023991 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_598311ac8416435286cfa4a6301380bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.025719 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc16d2310b294ccb8c18945b693c6fce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.026684 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4b6ec419ed7747c3b24dc11992518e88
W20260811 18:57:23.027453 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 335fc15ea75d483990e6ae5a5872682f has no consensus state
I20260811 18:57:23.028103 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d9b7a2cfa85347bf88241286c4bf0083" } new_table_name: "default.Table_d7ed0b81c09744e7af02ac80fb40508c" modify_external_catalogs: true
W20260811 18:57:23.030397 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa95131ce50943e3b0c6b5bdd5a30939 consensus state has no leader
I20260811 18:57:23.031169 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_df08bd98f2ef430eaf0e10876b9e265e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.032850 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1bf717c45ad424daa97f656cabd9db8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.037408 3890 raft_consensus.cc:2454] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:23.038048 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84e152c74f33463e83843b92c0783803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.022276 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 21cc1822d4b94081ad86a6177acb3afe
W20260811 18:57:23.039948 3824 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.146s (SENT)
I20260811 18:57:23.041342 5037 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.043227 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d79d187445ee43d6864f03f905400405"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.044379 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d79d187445ee43d6864f03f905400405 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.047528 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_269ccd3c54914a34be3c79acea4eeed3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.048338 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_269ccd3c54914a34be3c79acea4eeed3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.049721 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1bf717c45ad424daa97f656cabd9db8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.050290 4163 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:23.050812 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc16d2310b294ccb8c18945b693c6fce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.041718 5037 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.054595 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28fea07a2f434dc1997642a34e6b6e2b" } new_table_name: "default.Table_db07259b8d8744d49c2c9fc89d4c5506" modify_external_catalogs: true
I20260811 18:57:23.055495 5037 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.056529 3824 leader_election.cc:302] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.056701 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df08bd98f2ef430eaf0e10876b9e265e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.057032 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 56770.8us
W20260811 18:57:23.061901 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_598311ac8416435286cfa4a6301380bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.065114 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9948c33ed58b4160932be82ef9615ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.070232 4169 ts_tablet_manager.cc:1918] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:23.077530 3824 leader_election.cc:334] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.438s (SENT)
I20260811 18:57:23.077855 3824 leader_election.cc:302] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:23.079185 5037 raft_consensus.cc:2726] T 3c70d9db064e45af91fd9901d85e211f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:23.081707 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 70983.9us
I20260811 18:57:23.084097 4167 tablet_replica.cc:330] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.084391 4167 raft_consensus.cc:2229] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.084607 4167 raft_consensus.cc:2258] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.085219 4167 ts_tablet_manager.cc:1905] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.105540 4169 log.cc:1196] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/33d995d72f8142afaeb57a2c29910fa3
W20260811 18:57:23.107264 3824 raft_consensus.cc:2670] T 5ae200f3814e431a8288c30fe94bffcf P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:23.112247 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 54878.8us
W20260811 18:57:23.112710 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT) [suppressed 35 similar messages]
W20260811 18:57:23.113122 3824 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.002s (SENT)
I20260811 18:57:23.120004 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc1822d4b94081ad86a6177acb3afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.123031 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.142915 4163 tablet_bootstrap.cc:654] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:23.146768 4986 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.147230 3676 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.144701 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:23.150411 4169 ts_tablet_manager.cc:1939] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:23.153740 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 386) took 4615 ms. Trace:
W20260811 18:57:23.154058 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 9122ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:23.153995 4169 rpcz_store.cc:276] 0811 18:57:18.537795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539719 (+ 1924us) service_pool.cc:227] Handling call
0811 18:57:22.956124 (+4416405us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:22.956151 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.056324 (+100173us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.070077 (+ 13753us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.153377 (+ 83300us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4401603}
I20260811 18:57:23.154878 3744 raft_consensus.cc:2454] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:23.155999 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 33d995d72f8142afaeb57a2c29910fa3 (table default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]) successfully deleted
I20260811 18:57:23.158985 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:23.162864 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 113111us
I20260811 18:57:23.163964 4169 tablet_replica.cc:330] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:23.164353 4169 raft_consensus.cc:2229] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.165259 4169 raft_consensus.cc:2258] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.166153 4169 ts_tablet_manager.cc:1905] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.168143 3824 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.170444 5037 raft_consensus.cc:2781] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.170657 5037 raft_consensus.cc:482] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.172164 4952 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.172493 4952 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.174948 4952 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:23.175220 5037 raft_consensus.cc:3037] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.180444 4986 raft_consensus.cc:2781] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.184600 4986 raft_consensus.cc:482] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.184628 4162 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.188472 4986 raft_consensus.cc:3037] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.190075 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:23.190613 4162 ts_tablet_manager.cc:1397] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.747s user 0.021s sys 0.018s
W20260811 18:57:23.202183 3824 leader_election.cc:341] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:23.208544 3824 leader_election.cc:341] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:23.217131 4986 raft_consensus.cc:504] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.218995 4162 raft_consensus.cc:348] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.221333 5037 raft_consensus.cc:504] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.223425 4162 raft_consensus.cc:374] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.223733 4162 raft_consensus.cc:729] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:23.227653 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1800e9e2b064a6fb546b9d33cefa68b consensus state has no leader
I20260811 18:57:23.233817 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4efc43ae63254c67aac88d7bd9b6c6a2" } new_table_name: "default.Table_2e5ca40ed9be45ee85dd2681b4523778" modify_external_catalogs: true
I20260811 18:57:23.237129 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:23.237834 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a64df7c295c4d9baded1b63a9980581 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.241137 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd82f86a4579497fb4e50639ea1bf581" } modify_external_catalogs: true
I20260811 18:57:23.241684 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd82f86a4579497fb4e50639ea1bf581" } modify_external_catalogs: true
W20260811 18:57:23.244542 3678 leader_election.cc:334] T ed3346bd12d04c45a511308c427856e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.999s (SENT)
W20260811 18:57:23.245492 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
W20260811 18:57:23.245843 3678 leader_election.cc:334] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.429s (SENT)
W20260811 18:57:23.246255 3678 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.014s (SENT)
W20260811 18:57:23.246640 3678 leader_election.cc:334] T 508f0b39cd654aa991a2928a4b067762 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.015s (SENT)
I20260811 18:57:23.246922 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3744b3d608754de68c4c57b072d1dc15" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:23.249568 3890 raft_consensus.cc:3037] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.253182 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc1822d4b94081ad86a6177acb3afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.253631 4986 leader_election.cc:288] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.254207 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.254524 4162 consensus_queue.cc:260] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.234658 5041 consensus_queue.cc:1035] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260811 18:57:23.263789 3890 raft_consensus.cc:2454] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:23.271030 5041 consensus_queue.cc:1035] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260811 18:57:23.271468 4162 ts_tablet_manager.cc:1428] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.070s user 0.000s sys 0.005s
I20260811 18:57:23.272850 4162 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:23.272796 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94943cfbe41a47b3a36c81957a3adaeb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:23.273782 3744 raft_consensus.cc:3037] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.285765 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_313ce0e9dd1042e2827c0eebceaf3729"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.288879 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2103ce2744174eec9f603b49604123e4" } new_table_name: "default.Table_c8f7ec32b3654845877bb1de84a57f5f" modify_external_catalogs: true
I20260811 18:57:23.291292 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7075f060dbd942cca40c448af9eb5825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.292259 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2b67d7e59584a2bb877f79bdafed3a1" } new_table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" modify_external_catalogs: true
I20260811 18:57:23.294554 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.294313 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_de17e2e7bbbe429c9dea56a8b734f6b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.303570 5037 leader_election.cc:288] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:23.317373 5043 log.cc:924] Time spent T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260811 18:57:23.322865 4163 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:23.323261 4163 ts_tablet_manager.cc:1397] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.364s user 0.005s sys 0.008s
I20260811 18:57:23.325166 4163 raft_consensus.cc:348] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.325539 4163 raft_consensus.cc:374] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.325846 4163 raft_consensus.cc:729] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:23.326871 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.328583 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_313ce0e9dd1042e2827c0eebceaf3729 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.334511 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7075f060dbd942cca40c448af9eb5825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.339001 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:23.340958 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c7a814de93aa4d49a3ab5c7d2b167976"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.342034 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7a814de93aa4d49a3ab5c7d2b167976 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.344544 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5ae200f3814e431a8288c30fe94bffcf (table default.Table_a1cacf3713764528a169a58c4f2812b6 [id=c679a9f78be340db9d0c5703fda08f09]) successfully deleted
I20260811 18:57:23.353336 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7ddb21035302480fbfaf4738e6229f2b" } modify_external_catalogs: true
I20260811 18:57:23.353755 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7ddb21035302480fbfaf4738e6229f2b" } modify_external_catalogs: true
I20260811 18:57:23.366887 4167 ts_tablet_manager.cc:1918] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:23.367165 4167 log.cc:1196] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a300a7aca2814da6b999eb4d865fea66
I20260811 18:57:23.326205 4163 consensus_queue.cc:260] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.368067 4167 ts_tablet_manager.cc:1939] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:23.368677 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 371) took 5022 ms. Trace:
I20260811 18:57:23.368830 4167 rpcz_store.cc:276] 0811 18:57:18.346360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.348121 (+ 1761us) service_pool.cc:227] Handling call
0811 18:57:23.012530 (+4664409us) ts_tablet_manager.cc:1145] Deleting tablet a300a7aca2814da6b999eb4d865fea66
0811 18:57:23.012588 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.229722 (+217134us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.366576 (+136854us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.368632 (+ 2056us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4657527}
I20260811 18:57:23.370015 4167 tablet_replica.cc:330] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.370657 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a300a7aca2814da6b999eb4d865fea66 (table default.Table_1ded7e1a5cba4ca1a19ab9a21c3f87db [id=7818472e23a14febb1990a619b485823]) successfully deleted
I20260811 18:57:23.370754 4167 raft_consensus.cc:2229] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:23.371121 4167 raft_consensus.cc:2258] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.372146 4167 ts_tablet_manager.cc:1905] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.375403 4169 ts_tablet_manager.cc:1918] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:23.376653 4169 log.cc:1196] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/331396c9962f48ed9ea6dc66f1660902
I20260811 18:57:23.376888 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de9acdb33a2438eb4378a5eac844c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.379005 4162 tablet_bootstrap.cc:654] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:23.376359 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.379309 4163 ts_tablet_manager.cc:1428] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.052s user 0.004s sys 0.000s
I20260811 18:57:23.386860 4163 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:23.392939 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4059e5907b1e488ab9e1d04e563f96cd consensus state has no leader
I20260811 18:57:23.395211 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
I20260811 18:57:23.396421 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" } modify_external_catalogs: true
I20260811 18:57:23.396742 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e67c791ed9cd4da88ae7ce53ba362301" } modify_external_catalogs: true
I20260811 18:57:23.417495 3744 raft_consensus.cc:2454] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:23.419411 3824 leader_election.cc:334] T 60274115fe164d85badcf2e72d640f54 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.441s (SENT)
I20260811 18:57:23.423815 3824 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.424887 5037 raft_consensus.cc:2781] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.427224 5037 raft_consensus.cc:686] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:23.427737 5037 consensus_queue.cc:237] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:23.428874 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
W20260811 18:57:23.433243 3676 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 178272us
W20260811 18:57:23.433844 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae7da0868b114052ba421801b76fb33c consensus state has no leader
I20260811 18:57:23.435194 3676 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.435971 5041 raft_consensus.cc:2781] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.436158 5041 raft_consensus.cc:686] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:23.436239 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_016d0800749f4083ad87c2a640e058c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.436589 5041 consensus_queue.cc:237] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:23.437247 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_016d0800749f4083ad87c2a640e058c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.444339 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet b3f0cdcee681460084662e01152ad9dd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.445220 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9693b748bd0490cb416e0cb36a62966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.445621 4162 tablet_bootstrap.cc:492] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.445935 4162 ts_tablet_manager.cc:1397] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.174s user 0.009s sys 0.002s
W20260811 18:57:23.446022 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9693b748bd0490cb416e0cb36a62966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.447520 4162 raft_consensus.cc:348] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.447944 4162 raft_consensus.cc:374] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.448235 4162 raft_consensus.cc:729] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.449206 4162 consensus_queue.cc:260] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.450394 4162 ts_tablet_manager.cc:1428] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 18:57:23.451758 4162 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:23.454017 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_03949bb266e742cbbe858496f37177f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:23.455173 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_269ccd3c54914a34be3c79acea4eeed3" } new_table_name: "default.Table_4b41b319741e458abbf697fc780af7da" modify_external_catalogs: true
I20260811 18:57:23.456358 4162 tablet_bootstrap.cc:654] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:23.461269 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03949bb266e742cbbe858496f37177f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.498620 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3a85d24a6d8d4b038094e6e7607ddbd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.499656 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.502228 4169 ts_tablet_manager.cc:1939] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:23.502526 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_61f28b085f594e1eaf7d7eb8905a7ba2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.503489 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.505718 4163 tablet_bootstrap.cc:654] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:23.523239 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d11b889e9d044a049d0586382c263951 has no consensus state
I20260811 18:57:23.523269 4990 consensus_queue.cc:1035] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 18:57:23.523985 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_084495aab60944cda9e4a3f796132fa2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.524883 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_084495aab60944cda9e4a3f796132fa2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.527148 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c8f7ec32b3654845877bb1de84a57f5f" } new_table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" modify_external_catalogs: true
I20260811 18:57:23.536916 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:23.520198 5048 consensus_queue.cc:1035] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260811 18:57:23.539427 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 387) took 4978 ms. Trace:
I20260811 18:57:23.540673 4169 rpcz_store.cc:276] 0811 18:57:18.537974 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539755 (+ 1781us) service_pool.cc:227] Handling call
0811 18:57:23.154434 (+4614679us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:23.154449 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.332674 (+178225us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.359210 (+ 26536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.516112 (+156902us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4599887}
I20260811 18:57:23.531931 5052 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.546095 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 385) took 5008 ms. Trace:
I20260811 18:57:23.546159 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 331396c9962f48ed9ea6dc66f1660902 (table default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]) successfully deleted
I20260811 18:57:23.547070 4169 rpcz_store.cc:276] 0811 18:57:18.537581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.539682 (+ 2101us) service_pool.cc:227] Handling call
0811 18:57:23.545782 (+5006100us) ts_tablet_manager.cc:1145] Deleting tablet 8426ccbe530446fcbaa162a03cbb1892
0811 18:57:23.545796 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.545927 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4972232}
I20260811 18:57:23.548347 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 397) took 4900 ms. Trace:
I20260811 18:57:23.548466 5052 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:23.549167 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8426ccbe530446fcbaa162a03cbb1892 because the tablet was not found. No further retry: Not found: Tablet not found: 8426ccbe530446fcbaa162a03cbb1892
I20260811 18:57:23.550566 5052 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.554203 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 335fc15ea75d483990e6ae5a5872682f (DEFAULT_TABLE table=default.Table_6914252b8edc40e79af29a5145201b08 [id=c3840f888f554af48ca56484f725a3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.554728 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 335fc15ea75d483990e6ae5a5872682f (DEFAULT_TABLE table=default.Table_6914252b8edc40e79af29a5145201b08 [id=c3840f888f554af48ca56484f725a3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.555049 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fc15ea75d483990e6ae5a5872682f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.555433 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335fc15ea75d483990e6ae5a5872682f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.562233 5059 tablet.cc:1722] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:23.562867 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1" } modify_external_catalogs: true
I20260811 18:57:23.563230 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdfd955ae68f4bf09dea8635a00bc3b1" } modify_external_catalogs: true
I20260811 18:57:23.567646 4167 ts_tablet_manager.cc:1918] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:23.566569 4169 rpcz_store.cc:276] 0811 18:57:18.648065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.648160 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:23.548049 (+4899889us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:23.548062 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.548224 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4892991}
I20260811 18:57:23.568053 4167 log.cc:1196] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f1a55a42d1e64f04ad7f433614a3a1aa
I20260811 18:57:23.569067 4167 ts_tablet_manager.cc:1939] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:23.569561 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 62c365f79d4a4201ac323f16bddf3e62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62c365f79d4a4201ac323f16bddf3e62 already in progress: creating tablet
I20260811 18:57:23.569941 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 382) took 5027 ms. Trace:
I20260811 18:57:23.570075 3864 tablet_service.cc:1467] Processing CreateTablet for tablet dbe6f14450cd43129a2387d9de67260a (DEFAULT_TABLE table=default.Table_8b7d4eb2831c407a989326cf56a7d8f9 [id=08ff7cf1de89473ba831e1ce57958694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.571033 4167 rpcz_store.cc:276] 0811 18:57:18.541912 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.541995 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:23.369851 (+4827856us) ts_tablet_manager.cc:1145] Deleting tablet f1a55a42d1e64f04ad7f433614a3a1aa
0811 18:57:23.369867 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.507967 (+138100us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.567483 (+ 59516us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.569767 (+ 2284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4827152}
I20260811 18:57:23.571532 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbe6f14450cd43129a2387d9de67260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.573992 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 (DEFAULT_TABLE table=default.Table_52e514412ff342e3b34ba958e863d4ba [id=270f54fa01e044bfbb480b9776cd0b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.574571 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429eb5824f3f4b2b8da6e17ba7c69b92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.578135 3824 leader_election.cc:334] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.050s (SENT)
I20260811 18:57:23.571467 3714 tablet_service.cc:1467] Processing CreateTablet for tablet dbe6f14450cd43129a2387d9de67260a (DEFAULT_TABLE table=default.Table_8b7d4eb2831c407a989326cf56a7d8f9 [id=08ff7cf1de89473ba831e1ce57958694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.578469 3824 leader_election.cc:302] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.578805 3824 raft_consensus.cc:2670] T a300a7aca2814da6b999eb4d865fea66 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.579130 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbe6f14450cd43129a2387d9de67260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.580646 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 (DEFAULT_TABLE table=default.Table_52e514412ff342e3b34ba958e863d4ba [id=270f54fa01e044bfbb480b9776cd0b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.581393 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429eb5824f3f4b2b8da6e17ba7c69b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.582773 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f1a55a42d1e64f04ad7f433614a3a1aa (table default.Table_542b2f198c2a42abb286bbdcf948584b [id=a9971bcae1874a11a84225daf9b74313]) successfully deleted
I20260811 18:57:23.595153 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.595552 3890 raft_consensus.cc:2454] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:23.599478 3824 leader_election.cc:334] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.480s (SENT)
W20260811 18:57:23.610904 3824 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.232s (SENT)
I20260811 18:57:23.611382 3824 leader_election.cc:302] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:23.611925 3824 raft_consensus.cc:2670] T b1ce1042ae974f4dbac964bd7ea16a94 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.614530 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 6d28c486e408451a990ae3905ef3c2ff (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.615445 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d28c486e408451a990ae3905ef3c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.618561 3676 consensus_peers.cc:487] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33d995d72f8142afaeb57a2c29910fa3. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:23.629997 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 398) took 4965 ms. Trace:
I20260811 18:57:23.631318 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28b2e1b607a34ab2badc82e3c333a386" } new_table_name: "default.Table_c4384a8ba27a4a7696a6051325ddede2" modify_external_catalogs: true
I20260811 18:57:23.631281 4169 rpcz_store.cc:276] 0811 18:57:18.664239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.664303 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:23.570981 (+4906678us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:23.570993 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.629871 (+ 58878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4897385}
I20260811 18:57:23.632068 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 399) took 4927 ms. Trace:
I20260811 18:57:23.632109 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6d28c486e408451a990ae3905ef3c2ff (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.632364 4169 rpcz_store.cc:276] 0811 18:57:18.704762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.739715 (+ 34953us) service_pool.cc:227] Handling call
0811 18:57:23.631770 (+4892055us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:23.631787 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.631941 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4880127}
I20260811 18:57:23.632961 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d28c486e408451a990ae3905ef3c2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.633021 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 400) took 4860 ms. Trace:
I20260811 18:57:23.633328 4169 rpcz_store.cc:276] 0811 18:57:18.772467 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.798089 (+ 25622us) service_pool.cc:227] Handling call
0811 18:57:23.632665 (+4834576us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:23.632678 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.632912 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4832278}
I20260811 18:57:23.633827 4169 tablet_replica.cc:330] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:23.634245 4169 raft_consensus.cc:2229] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:23.637929 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e6a28da9c6fa458d9daa95fdbd30920c" } new_table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" modify_external_catalogs: true
I20260811 18:57:23.638082 3676 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.640931 4986 raft_consensus.cc:2781] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:23.642753 4986 raft_consensus.cc:482] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:23.642988 4986 raft_consensus.cc:3037] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:23.644671 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c99ceca57cb245f4a3a6cb4b36f1306d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c99ceca57cb245f4a3a6cb4b36f1306d already in progress: creating tablet
W20260811 18:57:23.645361 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f7d4c26da86b49b8a37f2289f4ef6614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d4c26da86b49b8a37f2289f4ef6614 already in progress: creating tablet
W20260811 18:57:23.645726 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d00d943b778741b4aeec076ba46a9c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d00d943b778741b4aeec076ba46a9c40 already in progress: creating tablet
W20260811 18:57:23.646726 3676 consensus_peers.cc:487] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 331396c9962f48ed9ea6dc66f1660902. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:23.661069 3824 leader_election.cc:334] T 80cf2375b4c64dbfae4e20003545da60 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.406s (SENT)
I20260811 18:57:23.662849 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 91d19b48908740128914fa5c231dfc04 (DEFAULT_TABLE table=default.Table_e863da40303e45bda8d9464f8ec9b8a0 [id=25706cf458f849498982901790caaaa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.663579 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91d19b48908740128914fa5c231dfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.670519 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 50af4e9b20a74ed983b26281c3c6eda1 (DEFAULT_TABLE table=default.Table_1204e0b30877462982c245f439f0bae4 [id=a260a40d963940198ab9f177c512f356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.671224 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50af4e9b20a74ed983b26281c3c6eda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.671748 5048 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.674314 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 50af4e9b20a74ed983b26281c3c6eda1 (DEFAULT_TABLE table=default.Table_1204e0b30877462982c245f439f0bae4 [id=a260a40d963940198ab9f177c512f356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.676842 5048 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.677634 4162 tablet_bootstrap.cc:492] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:23.677742 5048 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:23.682986 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 384) took 4956 ms. Trace:
I20260811 18:57:23.684655 4167 rpcz_store.cc:276] 0811 18:57:18.617796 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.624796 (+ 7000us) service_pool.cc:227] Handling call
0811 18:57:23.573454 (+4948658us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:23.573474 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.573702 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4927387}
I20260811 18:57:23.684710 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 91d19b48908740128914fa5c231dfc04 (DEFAULT_TABLE table=default.Table_e863da40303e45bda8d9464f8ec9b8a0 [id=25706cf458f849498982901790caaaa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.685114 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 71a7eda2248c4f708bdc772ea3598dec (DEFAULT_TABLE table=default.Table_056c759cfe3543f7a43de5b2fa3a4160 [id=673bce4a4bcf4d05b1e83791d99fb0f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.685566 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50af4e9b20a74ed983b26281c3c6eda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.685886 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91d19b48908740128914fa5c231dfc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.686272 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71a7eda2248c4f708bdc772ea3598dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.687433 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 385) took 5056 ms. Trace:
I20260811 18:57:23.687819 4167 rpcz_store.cc:276] 0811 18:57:18.631256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.644580 (+ 13324us) service_pool.cc:227] Handling call
0811 18:57:23.685638 (+5041058us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:23.685670 (+ 32us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.687233 (+ 1563us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5025357}
I20260811 18:57:23.688580 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 386) took 4984 ms. Trace:
I20260811 18:57:23.689049 4167 rpcz_store.cc:276] 0811 18:57:18.704084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.704164 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:23.688305 (+4984141us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:23.688317 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.688453 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4983802}
W20260811 18:57:23.694669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 62c365f79d4a4201ac323f16bddf3e62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62c365f79d4a4201ac323f16bddf3e62 already in progress: creating tablet
W20260811 18:57:23.695111 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c99ceca57cb245f4a3a6cb4b36f1306d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c99ceca57cb245f4a3a6cb4b36f1306d already in progress: creating tablet
W20260811 18:57:23.699846 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b203df4124364737839bf4c1ae072988 consensus state has no leader
I20260811 18:57:23.700554 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_00f2f8bec82f44f3a42420818d7dc6a0" } new_table_name: "default.Table_1074819ead5e452d8a479310453e4b9a" modify_external_catalogs: true
I20260811 18:57:23.703137 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a2ab815e5ecf491ab1ca7f1730ba148c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.703981 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2ab815e5ecf491ab1ca7f1730ba148c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.706084 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1f60a286ac024c45bcb845227c5445c9
I20260811 18:57:23.706691 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:23.707408 3554 catalog_manager.cc:6565] ReplaceTablet: tablet b3f0cdcee681460084662e01152ad9dd deleted and replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0
I20260811 18:57:23.708002 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.708703 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea1ee62f594e4ee798829d582bc2b53f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.710330 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d593aea288e14bb1a4c54360b260cffa
I20260811 18:57:23.711462 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_425203e0bd2944109d8f9ba2f0a7ba6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.712318 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_425203e0bd2944109d8f9ba2f0a7ba6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.713773 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:23.714617 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_69be92ae6ea74daf931e8ccd58a8dd4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.715312 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69be92ae6ea74daf931e8ccd58a8dd4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.716990 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5d495354c0994fb4a74c376078e41a26
I20260811 18:57:23.720752 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_061ae2e2037348cab0a2499a98d273a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.721613 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_061ae2e2037348cab0a2499a98d273a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.724061 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.725394 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.698419 4162 ts_tablet_manager.cc:1397] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.227s user 0.003s sys 0.015s
I20260811 18:57:23.699677 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 78d442dd6e29433ab006e1559c13d06c (DEFAULT_TABLE table=default.Table_a93638c401d94d3d8dad242dd4c9b60a [id=50af5dec6ee3414989f38b8d91db91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.696504 5061 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:23.725548 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.726435 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.728374 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:23.731389 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e8859199c3440ebb2b9664b65e0fb71" } new_table_name: "default.Table_4e412149fc55445bb720ffed59a05618" modify_external_catalogs: true
W20260811 18:57:23.735451 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f7d4c26da86b49b8a37f2289f4ef6614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f7d4c26da86b49b8a37f2289f4ef6614 already in progress: creating tablet
I20260811 18:57:23.736374 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d593aea288e14bb1a4c54360b260cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.740118 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f60a286ac024c45bcb845227c5445c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.743191 4986 raft_consensus.cc:504] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.744951 4986 leader_election.cc:288] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:23.720104 4169 raft_consensus.cc:2258] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.723758 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:23.741055 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 71a7eda2248c4f708bdc772ea3598dec (DEFAULT_TABLE table=default.Table_056c759cfe3543f7a43de5b2fa3a4160 [id=673bce4a4bcf4d05b1e83791d99fb0f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.751004 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71a7eda2248c4f708bdc772ea3598dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.745148 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet d593aea288e14bb1a4c54360b260cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.747781 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.747165 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 78d442dd6e29433ab006e1559c13d06c (DEFAULT_TABLE table=default.Table_a93638c401d94d3d8dad242dd4c9b60a [id=50af5dec6ee3414989f38b8d91db91f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.754087 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:23.757107 3676 leader_election.cc:341] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:23.752365 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.762567 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.744746 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.745469 5062 tablet.cc:1722] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:23.764351 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 649cb98f553a418b911e307fdaf49917 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.765625 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f60a286ac024c45bcb845227c5445c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.766263 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d442dd6e29433ab006e1559c13d06c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.766635 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edd393d9c4d24aab8a170e1329400656 consensus state has no leader
I20260811 18:57:23.767511 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fd9fce095e4a769dbb38c7903359cc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:23.768764 3887 raft_consensus.cc:3037] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.768776 3708 tablet_service.cc:1467] Processing CreateTablet for tablet ab03996c5c624e5eb17267b124e27f45 (DEFAULT_TABLE table=default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 [id=c89c9365aee249419be03a7bd8b091fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.769943 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb51e419dd24459caf3ba3adfbbd4517 (table default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8]). Expected version 1 got 0
W20260811 18:57:23.770802 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e0747ef88574c68ae074732cb7095b9 consensus state has no leader
I20260811 18:57:23.771531 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.772430 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47326d9f3fac4cb7b76516843cf7bfd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.774112 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae7da0868b114052ba421801b76fb33c (table default.Table_32b9a3e653554115b17152dc38aa0439 [id=c417185e41c94dad8cc5e5f5937adb2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae7da0868b114052ba421801b76fb33c consensus state has no leader
I20260811 18:57:23.778985 4162 raft_consensus.cc:348] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.775082 5059 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38286 (request call id 542) took 1091 ms. Trace:
I20260811 18:57:23.782310 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d442dd6e29433ab006e1559c13d06c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.786782 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fd9fce095e4a769dbb38c7903359cc with delete_type TABLET_DATA_DELETED (replaced by tablet 1b73a97b7abf4bd58c792426be6e938d) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.787170 3710 tablet_service.cc:1467] Processing CreateTablet for tablet ae02c9b0b066474894546829aed304e3 (DEFAULT_TABLE table=default.Table_204c26eb8e554700a777796f2df5e7b1 [id=137cef1d67054dc6be05fcef9b22c375]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:23.796768 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 16570ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
I20260811 18:57:23.797506 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.798071 4169 ts_tablet_manager.cc:1905] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.799137 4167 tablet_replica.cc:330] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:23.800895 4162 raft_consensus.cc:374] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.801090 4162 raft_consensus.cc:729] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.801484 3709 tablet_service.cc:1467] Processing CreateTablet for tablet b16cb4c1796147fbb10109f52071505e (DEFAULT_TABLE table=default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d [id=35e57410bd7f4f1eba55e9abc899cc86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.801891 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649cb98f553a418b911e307fdaf49917. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.804129 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f287040b879642448367abe2b1c73cd9 consensus state has no leader
I20260811 18:57:23.803347 5059 rpcz_store.cc:276] 0811 18:57:22.635397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.635475 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:23.727069 (+1091594us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:22.672761 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:22.672924 (+ 163us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:22.672943 (+ 19us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:22.672953 (+ 10us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:22.673013 (+ 60us) alter_schema_op.cc:126] START. Timestamp: 1786474642672929
0811 18:57:22.673018 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:22.673414 (+ 396us) log.cc:842] Serialized 173 byte log entry
0811 18:57:23.475256 (+801842us) op_driver.cc:444] REPLICATION: finished
0811 18:57:23.561982 (+ 86726us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:23.563498 (+ 1516us) log.cc:842] Serialized 5 byte log entry
0811 18:57:23.720018 (+156520us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:23.726546 (+ 6528us) log.cc:842] Serialized 14 byte log entry
0811 18:57:23.727011 (+ 465us) alter_schema_op.cc:71] Released schema lock
0811 18:57:23.727021 (+ 10us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":86386,"prepare.queue_time_us":6946,"prepare.run_cpu_time_us":5142,"prepare.run_wall_time_us":82023,"raft.queue_time_us":129,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":346,"replication_time_us":802192,"thread_start_us":664721,"threads_started":3,"wal-append.queue_time_us":116536}]]}
I20260811 18:57:23.810170 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16cb4c1796147fbb10109f52071505e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.811296 4163 tablet_bootstrap.cc:492] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:23.814308 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02c9b0b066474894546829aed304e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.821590 4163 ts_tablet_manager.cc:1397] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.431s user 0.004s sys 0.007s
I20260811 18:57:23.821939 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 649cb98f553a418b911e307fdaf49917 (DEFAULT_TABLE table=default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.825492 3863 tablet_service.cc:1467] Processing CreateTablet for tablet ab03996c5c624e5eb17267b124e27f45 (DEFAULT_TABLE table=default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69 [id=c89c9365aee249419be03a7bd8b091fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.825862 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 56d24aff9fd34f3eb05fdf942cc7efc7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:23.828541 4167 raft_consensus.cc:2229] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:23.834019 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_507c4f42c22847e39b4c700687a1687c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.835005 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_507c4f42c22847e39b4c700687a1687c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:23.837215 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:23.837962 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.838807 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91dfb6056e5049bb8cca8a06b3a33b7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.843928 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab03996c5c624e5eb17267b124e27f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.851441 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 56d24aff9fd34f3eb05fdf942cc7efc7
I20260811 18:57:23.851991 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 56d24aff9fd34f3eb05fdf942cc7efc7 deleted and replaced by tablet 19d7716739f848d78b9cbbd3975f3d44
I20260811 18:57:23.853453 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a (DEFAULT_TABLE table=default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.855436 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.860581 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 133efffbd3d74cdca61efd8ad0225b41 (DEFAULT_TABLE table=default.Table_d18d57b020e046b8a069552ea63ae961 [id=ed42157134b94a488999a64b5f6b92ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.870934 5072 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:23.851918 4162 consensus_queue.cc:260] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.886518 3887 raft_consensus.cc:2454] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:23.886936 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 133efffbd3d74cdca61efd8ad0225b41 (DEFAULT_TABLE table=default.Table_d18d57b020e046b8a069552ea63ae961 [id=ed42157134b94a488999a64b5f6b92ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.888576 3860 tablet_service.cc:1467] Processing CreateTablet for tablet ae02c9b0b066474894546829aed304e3 (DEFAULT_TABLE table=default.Table_204c26eb8e554700a777796f2df5e7b1 [id=137cef1d67054dc6be05fcef9b22c375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.889775 3856 tablet_service.cc:1467] Processing CreateTablet for tablet bef0f6d56cda40249f12b25a8eb3ab6b (DEFAULT_TABLE table=default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.892341 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b16cb4c1796147fbb10109f52071505e (DEFAULT_TABLE table=default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d [id=35e57410bd7f4f1eba55e9abc899cc86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.892735 3676 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:23.897114 4162 ts_tablet_manager.cc:1428] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.145s user 0.004s sys 0.000s
I20260811 18:57:23.900486 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133efffbd3d74cdca61efd8ad0225b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.904501 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 844bcd16f06c4788af9c0d7d320464aa (DEFAULT_TABLE table=default.Table_e8554df7eb5f4fbbbe828834c1d45f25 [id=00e172dcefa0491f9f287857d60ba3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.905812 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649cb98f553a418b911e307fdaf49917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.905849 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e4e304a679345cd9289252278ddb281"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:23.906930 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e4e304a679345cd9289252278ddb281 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:23.907931 4167 raft_consensus.cc:2258] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:23.909795 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab03996c5c624e5eb17267b124e27f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.913936 4163 raft_consensus.cc:348] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.914817 4163 raft_consensus.cc:374] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.914978 4163 raft_consensus.cc:729] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.916828 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6317dbacc890428caf5adac65ad7895d" } modify_external_catalogs: true
I20260811 18:57:23.917060 4163 consensus_queue.cc:260] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.917250 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6317dbacc890428caf5adac65ad7895d" } modify_external_catalogs: true
W20260811 18:57:23.917425 3678 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT)
I20260811 18:57:23.917939 4163 ts_tablet_manager.cc:1428] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.096s user 0.004s sys 0.000s
I20260811 18:57:23.917910 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd203b7eb8643f493b4642c0396e258 (DEFAULT_TABLE table=default.Table_f7fd9b1d34c3439c997e0341237d5a82 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.921886 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09" } modify_external_catalogs: true
I20260811 18:57:23.922217 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef0f6d56cda40249f12b25a8eb3ab6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.922308 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5d80bb7d236c4fb58268cf8ca2cc5e09" } modify_external_catalogs: true
I20260811 18:57:23.924433 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_425203e0bd2944109d8f9ba2f0a7ba6d" } new_table_name: "default.Table_71aa50e0de9f470d9c9cceeb3b18cd90" modify_external_catalogs: true
I20260811 18:57:23.924819 5051 raft_consensus.cc:2781] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.925181 5051 raft_consensus.cc:686] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:23.925839 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 133efffbd3d74cdca61efd8ad0225b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.927124 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02c9b0b066474894546829aed304e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.898762 3723 tablet_service.cc:1467] Processing CreateTablet for tablet bef0f6d56cda40249f12b25a8eb3ab6b (DEFAULT_TABLE table=default.Table_9e2e268d15894407ab276766ff611c86 [id=a067d27ecf9c402db2631f2df5f0fea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.928206 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16cb4c1796147fbb10109f52071505e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.929428 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24955fa6843f46dc883be60173049a16" } modify_external_catalogs: true
I20260811 18:57:23.929749 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24955fa6843f46dc883be60173049a16" } modify_external_catalogs: true
I20260811 18:57:23.931586 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d5db1f0452e40eea01177ba93223341" } modify_external_catalogs: true
I20260811 18:57:23.931864 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4d5db1f0452e40eea01177ba93223341" } modify_external_catalogs: true
I20260811 18:57:23.935477 4167 ts_tablet_manager.cc:1905] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:23.938772 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 705605cd929940f8a252b6e95abbeb95 (DEFAULT_TABLE table=default.Table_db899a033c3144559d1d5c7bf1d92fbf [id=5593f1d77ce94befac70ddabfc422ba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.941453 4163 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:23.943341 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 (DEFAULT_TABLE table=default.Table_8dbfb0ccfdef4c018c7eca1d0452375a [id=15fde9b467f04384aa3a2ea431ca352b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.933897 5051 consensus_queue.cc:237] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:23.947141 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 844bcd16f06c4788af9c0d7d320464aa (DEFAULT_TABLE table=default.Table_e8554df7eb5f4fbbbe828834c1d45f25 [id=00e172dcefa0491f9f287857d60ba3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.947966 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844bcd16f06c4788af9c0d7d320464aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:23.948779 3824 leader_election.cc:334] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.068s (SENT)
I20260811 18:57:23.949064 3824 leader_election.cc:302] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:23.965447 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 19d7716739f848d78b9cbbd3975f3d44) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:23.968080 5037 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:23.968407 5037 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:23.969038 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 4dd203b7eb8643f493b4642c0396e258 (DEFAULT_TABLE table=default.Table_f7fd9b1d34c3439c997e0341237d5a82 [id=f51f38f3582f44db99007a265b665fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.969678 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd203b7eb8643f493b4642c0396e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.970749 3872 tablet_service.cc:1467] Processing CreateTablet for tablet f8ca801c79fc499ab50cfe76cdb03b00 (DEFAULT_TABLE table=default.Table_9de9363efd764bd4ac323f86fd70658d [id=7c19e7030e324615ab16dd34a65a2ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.971261 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ca801c79fc499ab50cfe76cdb03b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.975543 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d24aff9fd34f3eb05fdf942cc7efc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 19d7716739f848d78b9cbbd3975f3d44) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:23.977188 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 705605cd929940f8a252b6e95abbeb95 (DEFAULT_TABLE table=default.Table_db899a033c3144559d1d5c7bf1d92fbf [id=5593f1d77ce94befac70ddabfc422ba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.977972 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705605cd929940f8a252b6e95abbeb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.980849 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 (DEFAULT_TABLE table=default.Table_8dbfb0ccfdef4c018c7eca1d0452375a [id=15fde9b467f04384aa3a2ea431ca352b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.980873 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 86d690ec09524248bc147ad7408e2ee6 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.981976 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 (DEFAULT_TABLE table=default.Table_ed948e1e9ee14cffae82bcd5517709f8 [id=4275d081d66941b9b9f0a927e7a19eea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:23.982069 3824 raft_consensus.cc:2670] T dde27d8cc3eb4d09879e717fc90305fd P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:23.982441 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.983625 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa (DEFAULT_TABLE table=default.Table_853572d26c9642b9b8ad616f574301dc [id=cecb2799de544cb392a84211ab0d55d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:23.993201 5037 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:24.003268 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.004962 4162 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.014982 5048 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.015404 5048 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.017042 5048 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.017581 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e556811ad1643f1baa3bbf4edebd157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.019181 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a (DEFAULT_TABLE table=default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.021219 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f8ca801c79fc499ab50cfe76cdb03b00 (DEFAULT_TABLE table=default.Table_9de9363efd764bd4ac323f86fd70658d [id=7c19e7030e324615ab16dd34a65a2ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.027926 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844bcd16f06c4788af9c0d7d320464aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.028369 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa (DEFAULT_TABLE table=default.Table_853572d26c9642b9b8ad616f574301dc [id=cecb2799de544cb392a84211ab0d55d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.029731 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 (DEFAULT_TABLE table=default.Table_ed948e1e9ee14cffae82bcd5517709f8 [id=4275d081d66941b9b9f0a927e7a19eea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.019429 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d690ec09524248bc147ad7408e2ee6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.022250 3824 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT)
I20260811 18:57:24.022714 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:24.019512 4163 tablet_bootstrap.cc:654] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.035017 3887 raft_consensus.cc:2454] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
#21 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
#22 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
I20260811 18:57:24.030401 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 86d690ec09524248bc147ad7408e2ee6 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.047590 3676 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.048076 5048 raft_consensus.cc:2781] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:24.050132 5048 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:24.050443 5048 raft_consensus.cc:3037] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.051397 3824 leader_election.cc:302] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.056273 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.061767 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68808.4us
W20260811 18:57:24.062173 3824 leader_election.cc:334] T 6ce8db230c7e41b6a4792273a0f07e6e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.008s (SENT)
I20260811 18:57:24.071333 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:24.071786 3744 raft_consensus.cc:2454] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:24.072433 3824 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.805s (SENT)
I20260811 18:57:24.072763 3824 leader_election.cc:302] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.073065 5037 raft_consensus.cc:2726] T 866ed614511649459b22a9fce55220b2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:24.073410 3678 leader_election.cc:334] T b1ce1042ae974f4dbac964bd7ea16a94 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.367s (SENT)
I20260811 18:57:24.076977 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ca801c79fc499ab50cfe76cdb03b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.082751 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.087888 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d690ec09524248bc147ad7408e2ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.055529 5048 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.093858 3824 raft_consensus.cc:2670] T d3d6daff4b7540309ff58755758e9752 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.102583 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705605cd929940f8a252b6e95abbeb95. 1 dirs total, 0 dirs full, 0 dirs failed
#23 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#24 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
W20260811 18:57:24.114430 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46001.9us
#25 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:24.120321 4169 ts_tablet_manager.cc:1918] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:24.123493 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef0f6d56cda40249f12b25a8eb3ab6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.138459 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dd203b7eb8643f493b4642c0396e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.142634 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.145408 5048 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.154731 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e556811ad1643f1baa3bbf4edebd157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.179795 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ad5f39f59b8447cb3f15b9646e4b852" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:24.180212 3890 raft_consensus.cc:3037] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.186015 3824 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.187256 5037 raft_consensus.cc:2781] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:24.187489 5037 raft_consensus.cc:482] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:24.187719 5037 raft_consensus.cc:3037] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.198593 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed61380b12e548ce843c5056342085b9" } modify_external_catalogs: true
I20260811 18:57:24.198979 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed61380b12e548ce843c5056342085b9" } modify_external_catalogs: true
I20260811 18:57:24.199896 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_204c26eb8e554700a777796f2df5e7b1" } new_table_name: "default.Table_f6953ae59bab4944a8db87e863735f79" modify_external_catalogs: true
I20260811 18:57:24.203100 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db899a033c3144559d1d5c7bf1d92fbf" } new_table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" modify_external_catalogs: true
W20260811 18:57:24.230533 3678 leader_election.cc:334] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 6.262s (SENT)
I20260811 18:57:24.235040 4162 tablet_bootstrap.cc:654] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.237990 4986 consensus_queue.cc:1035] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260811 18:57:24.240342 3678 leader_election.cc:302] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.255725 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 53891.6us
W20260811 18:57:24.262226 3678 raft_consensus.cc:2670] T 5df7a07983174dd1aaf48951c9ff1a1b P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.269781 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 53188.7us
I20260811 18:57:24.280339 4163 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:24.280582 4163 ts_tablet_manager.cc:1397] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.362s user 0.004s sys 0.004s
I20260811 18:57:24.289487 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:24.289101 5037 raft_consensus.cc:504] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.301808 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7057b8a7bae249caaed304b5c66e6871
I20260811 18:57:24.303061 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0da5d975ff884b9bb82d239828666af7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.303923 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0da5d975ff884b9bb82d239828666af7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.309926 4163 raft_consensus.cc:348] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:24.310261 4163 raft_consensus.cc:374] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.310150 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23f41a2638e74ba19c1c527b9fc796df" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:24.310401 4163 raft_consensus.cc:729] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.310465 3744 raft_consensus.cc:3037] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:24.310793 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 14e34732bda44670ae9da8facc405221
W20260811 18:57:24.310789 3824 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.598s (SENT)
I20260811 18:57:24.311790 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0036c28f59cb4f229da243389602bbd9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.312829 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0036c28f59cb4f229da243389602bbd9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.314566 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 219474fc820d40f49810209ac1c40afe has no consensus state
I20260811 18:57:24.314124 4163 consensus_queue.cc:260] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:24.315009 3890 raft_consensus.cc:2454] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:24.315255 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a65337aa3b94589a51dd41812870aca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.315706 3676 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:24.316001 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a65337aa3b94589a51dd41812870aca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.316505 4986 raft_consensus.cc:2781] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:24.317734 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
I20260811 18:57:24.318749 4986 raft_consensus.cc:686] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:24.318698 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bc171757977f4404bb736a03a87b9ecd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.319123 4986 consensus_queue.cc:237] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.319468 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc171757977f4404bb736a03a87b9ecd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.320904 3824 leader_election.cc:302] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.320930 3744 raft_consensus.cc:2454] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:24.321223 3824 raft_consensus.cc:2670] T d9421bc220774f66b90bf537a47f1e13 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.321862 3678 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.022s (SENT)
I20260811 18:57:24.322103 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 64b922f2a71544e8a18533aa0a1ccb52
I20260811 18:57:24.322734 3824 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:24.323385 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.324227 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6ed856dda814eb0bd8eb76c2f83c70b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.324414 3678 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.818s (SENT)
I20260811 18:57:24.325706 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_559eff9548514310840eee6c9a02b364 [id=58d4d7c05ba44fc2b4c4a71746d340dd] alter complete (version 2)
I20260811 18:57:24.326690 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.327564 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.329375 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 705605cd929940f8a252b6e95abbeb95 consensus state has no leader
I20260811 18:57:24.329944 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d1c65c57764085a678b08ec4e1938b" } new_table_name: "default.Table_24cdaf08a18b43e3bf356bb8b04f5017" modify_external_catalogs: true
I20260811 18:57:24.340543 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddcc89fb831040ed9752150dc2ad4b51" } new_table_name: "default.Table_9b7ce6401d2c44b9937b7f442ccbd56d" modify_external_catalogs: true
I20260811 18:57:24.341779 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.342640 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.346796 4167 ts_tablet_manager.cc:1918] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.347262 4167 log.cc:1196] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e215759d48c24d8cb99bbc3fc9009a66
I20260811 18:57:24.348335 4167 ts_tablet_manager.cc:1939] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.349295 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 387) took 5595 ms. Trace:
I20260811 18:57:24.323276 5037 leader_election.cc:288] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:24.350361 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:24.354892 5037 raft_consensus.cc:2781] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:24.370350 5037 raft_consensus.cc:686] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:24.351842 4162 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.371822 4162 ts_tablet_manager.cc:1397] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.474s user 0.006s sys 0.010s
I20260811 18:57:24.372118 5037 consensus_queue.cc:237] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.373806 4162 raft_consensus.cc:348] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.374285 4162 raft_consensus.cc:374] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.374776 4162 raft_consensus.cc:729] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.375444 4162 consensus_queue.cc:260] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.377140 4169 log.cc:1196] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a443fe209556489abb47f4faa5d696e5
I20260811 18:57:24.379642 4169 ts_tablet_manager.cc:1939] T a443fe209556489abb47f4faa5d696e5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:24.380388 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 401) took 5590 ms. Trace:
I20260811 18:57:24.358955 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e34732bda44670ae9da8facc405221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.362470 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.381253 4167 rpcz_store.cc:276] 0811 18:57:18.753875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.753950 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:23.727721 (+4973771us) ts_tablet_manager.cc:1145] Deleting tablet e215759d48c24d8cb99bbc3fc9009a66
0811 18:57:23.727737 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.240708 (+512971us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.346587 (+105879us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.349243 (+ 2656us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4973160}
I20260811 18:57:24.387840 4162 ts_tablet_manager.cc:1428] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260811 18:57:24.390607 4162 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.388924 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 388) took 5618 ms. Trace:
I20260811 18:57:24.392478 4167 rpcz_store.cc:276] 0811 18:57:18.770410 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.770487 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:24.388540 (+5618053us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:24.388562 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.388774 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5617703}
I20260811 18:57:24.393592 4167 tablet_replica.cc:330] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:24.393703 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d00d943b778741b4aeec076ba46a9c40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d00d943b778741b4aeec076ba46a9c40 already in progress: creating tablet
I20260811 18:57:24.397061 4167 raft_consensus.cc:2229] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.399226 4167 raft_consensus.cc:2258] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.405963 4167 ts_tablet_manager.cc:1905] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.418445 4162 tablet_bootstrap.cc:654] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.416026 4169 rpcz_store.cc:276] 0811 18:57:18.789601 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.807351 (+ 17750us) service_pool.cc:227] Handling call
0811 18:57:23.633664 (+4826313us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:23.633676 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:23.833566 (+199890us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.120110 (+286544us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.380320 (+260210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4813011}
I20260811 18:57:24.450486 4169 tablet_replica.cc:330] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:24.452919 4169 raft_consensus.cc:2229] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:24.456672 4169 raft_consensus.cc:2258] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.457707 4169 ts_tablet_manager.cc:1905] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:24.463194 3678 leader_election.cc:334] T 8426ccbe530446fcbaa162a03cbb1892 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.936s (SENT)
I20260811 18:57:24.466076 4162 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.467408 4162 ts_tablet_manager.cc:1397] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.000s
I20260811 18:57:24.475271 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 3263da75f0d84528b7c8cc55dcaeaa46 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.476353 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8d7695d0859045409244bdd77ddd3931"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.477241 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d7695d0859045409244bdd77ddd3931 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.479215 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21054bdd3d7c400c95b1f41e41bafc79 consensus state has no leader
I20260811 18:57:24.482249 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c6e0d1eeb63493bbea4cd78dd085a3c from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:24.485603 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffb196eaa8c2483aa71418747adda95a has no consensus state
I20260811 18:57:24.488124 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2bd2a5750a454b88b5bff7b802541c89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.488960 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bd2a5750a454b88b5bff7b802541c89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.489099 4162 raft_consensus.cc:348] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.489670 4162 raft_consensus.cc:374] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.489850 4162 raft_consensus.cc:729] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.490828 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c4130777df64341a269a7954794efd2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.491746 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c4130777df64341a269a7954794efd2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.494040 4162 consensus_queue.cc:260] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.495335 4162 ts_tablet_manager.cc:1428] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:24.495901 4162 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.497817 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0322ec9fd9f24338bb9196b099f56643"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.498610 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0322ec9fd9f24338bb9196b099f56643 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.506059 4167 ts_tablet_manager.cc:1918] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.507120 4167 log.cc:1196] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f25ff8a9e3a944269e9799995c73ebe0
#26 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:24.510157 4167 ts_tablet_manager.cc:1939] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.520504 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 389) took 5742 ms. Trace:
I20260811 18:57:24.526002 4167 rpcz_store.cc:276] 0811 18:57:18.775166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.775403 (+ 237us) service_pool.cc:227] Handling call
0811 18:57:24.393409 (+5618006us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:24.393425 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.440558 (+ 47133us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.504526 (+ 63968us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.517596 (+ 13070us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5617663}
I20260811 18:57:24.532207 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 390) took 5743 ms. Trace:
I20260811 18:57:24.533941 4167 rpcz_store.cc:276] 0811 18:57:18.788588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.788653 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:24.530156 (+5741503us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:24.530178 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.532134 (+ 1956us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5741224}
I20260811 18:57:24.537583 4167 tablet_replica.cc:330] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.538250 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f25ff8a9e3a944269e9799995c73ebe0 (table default.Table_8397339a34974af9b4e8f20778beb8c7 [id=827e890cae784582aa5babbdeb2cdc4e]) successfully deleted
W20260811 18:57:24.538785 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a443fe209556489abb47f4faa5d696e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a443fe209556489abb47f4faa5d696e5 already in progress: creating tablet
I20260811 18:57:24.539345 4167 raft_consensus.cc:2229] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.539816 4167 raft_consensus.cc:2258] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.540827 4167 ts_tablet_manager.cc:1905] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.529095 4162 tablet_bootstrap.cc:654] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.542938 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b40e44a2f4940af891eddf6c3b6293c from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.555732 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee" } modify_external_catalogs: true
I20260811 18:57:24.556349 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdc11a6352eb4aac9ff657cf245aceee" } modify_external_catalogs: true
I20260811 18:57:24.557189 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea0cc7a576d64d1c934364316cae3c70" } modify_external_catalogs: true
I20260811 18:57:24.557552 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea0cc7a576d64d1c934364316cae3c70" } modify_external_catalogs: true
I20260811 18:57:24.558393 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_632b7ba5285744ee8f4bdc969820dae6" } new_table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" modify_external_catalogs: true
I20260811 18:57:24.560575 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e75a55498d4c4072968ee03aaf48b831" } new_table_name: "default.Table_7cabaea271024c3daf3ffc96c5825f1e" modify_external_catalogs: true
I20260811 18:57:24.564586 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1b2efb13eaf1468c92c8030667191b84" } new_table_name: "default.Table_34d2346219e14bd58aa02228ad109763" modify_external_catalogs: true
I20260811 18:57:24.566756 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" } modify_external_catalogs: true
I20260811 18:57:24.567185 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f581cf3ac8147ac844b27d7c201d6c4" } modify_external_catalogs: true
I20260811 18:57:24.568264 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a33a6c24205421f9fc488087672c6e0" } modify_external_catalogs: true
I20260811 18:57:24.568598 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a33a6c24205421f9fc488087672c6e0" } modify_external_catalogs: true
W20260811 18:57:24.456357 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 129574us
I20260811 18:57:24.584354 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.589056 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e34732bda44670ae9da8facc405221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.589993 4162 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.590276 4162 ts_tablet_manager.cc:1397] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.095s user 0.005s sys 0.004s
I20260811 18:57:24.592965 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.597210 4162 raft_consensus.cc:348] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.597535 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a443fe209556489abb47f4faa5d696e5 (table default.Table_bf28eaf7393d4d508363fbc8dd91686b [id=1a0803466858425cadc33f51f0933a95]) successfully deleted
I20260811 18:57:24.597674 4162 raft_consensus.cc:374] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.597898 4162 raft_consensus.cc:729] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:24.600040 3824 leader_election.cc:334] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.744s (SENT)
I20260811 18:57:24.600302 3824 leader_election.cc:302] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.601029 3824 raft_consensus.cc:2670] T a30bb384136f4a169a5751655437d5a3 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.602990 3824 leader_election.cc:334] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.866s (SENT)
I20260811 18:57:24.603415 3824 leader_election.cc:302] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.603787 3824 raft_consensus.cc:2670] T cf285cdf5235451bb1531cb294017101 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.604832 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 64b922f2a71544e8a18533aa0a1ccb52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:24.605235 3824 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.275s (SENT)
I20260811 18:57:24.605531 3824 leader_election.cc:302] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.606324 5080 raft_consensus.cc:2726] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:24.608431 3824 leader_election.cc:334] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.469s (SENT)
I20260811 18:57:24.608700 3824 leader_election.cc:302] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.609081 3824 raft_consensus.cc:2670] T 9f8c0444ec2242299482736a73551f80 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.610193 3824 leader_election.cc:334] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.219s (SENT)
I20260811 18:57:24.610616 3824 leader_election.cc:302] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.611001 3824 raft_consensus.cc:2670] T 0e1efffade0f486d8d627e56e0be6ff6 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.611495 4162 consensus_queue.cc:260] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:24.618466 3824 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.386s (SENT)
I20260811 18:57:24.618979 3824 leader_election.cc:302] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.619619 5080 raft_consensus.cc:2726] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:24.620155 4162 ts_tablet_manager.cc:1428] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.028s user 0.001s sys 0.003s
I20260811 18:57:24.620772 4162 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.624878 4162 tablet_bootstrap.cc:654] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:24.632856 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 84e152c74f33463e83843b92c0783803 consensus state has no leader
W20260811 18:57:24.633586 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
W20260811 18:57:24.634361 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4c7f500a5444bae9a015a5e903c0622 consensus state has no leader
I20260811 18:57:24.636521 4169 ts_tablet_manager.cc:1918] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.636947 4169 log.cc:1196] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/405773849fb94800be61ad8a7a6055a4
I20260811 18:57:24.639906 4169 ts_tablet_manager.cc:1939] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:24.640558 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 403) took 5716 ms. Trace:
I20260811 18:57:24.665561 4986 raft_consensus.cc:482] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.665889 4986 raft_consensus.cc:504] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.668322 4167 ts_tablet_manager.cc:1918] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.668694 4167 log.cc:1196] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d9421bc220774f66b90bf537a47f1e13
I20260811 18:57:24.669412 4167 ts_tablet_manager.cc:1939] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.675680 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 395) took 5740 ms. Trace:
I20260811 18:57:24.677517 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7e8b32170ff47eda203b84328064301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.678375 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7e8b32170ff47eda203b84328064301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.679387 4986 leader_election.cc:288] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.680115 4986 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:24.680787 4986 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.681689 4986 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.683406 4167 rpcz_store.cc:276] 0811 18:57:18.935117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.935416 (+ 299us) service_pool.cc:227] Handling call
0811 18:57:24.537454 (+5602038us) ts_tablet_manager.cc:1145] Deleting tablet d9421bc220774f66b90bf537a47f1e13
0811 18:57:24.537474 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.572436 (+ 34962us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.642530 (+ 70094us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.675615 (+ 33085us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5601762}
I20260811 18:57:24.685784 4167 tablet_replica.cc:330] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.685945 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d9421bc220774f66b90bf537a47f1e13 (table default.Table_b70decfaca9c4a9f87915d80838c6061 [id=d3a8cde4f30845a29d2b542fd425f4cb]) successfully deleted
I20260811 18:57:24.686622 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b654b59f369e429097b918236d8ddd08" } new_table_name: "default.Table_9194817120074797a6f60bf8e24c0636" modify_external_catalogs: true
I20260811 18:57:24.687809 4167 raft_consensus.cc:2229] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.688391 4167 raft_consensus.cc:2258] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.694308 3550 catalog_manager.cc:5495] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:24.698926 4167 ts_tablet_manager.cc:1905] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.668375 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2247194db704f9abd54a8a0df1ca566" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:24.683266 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:24.723671 3676 leader_election.cc:341] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:24.724375 3676 leader_election.cc:341] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:24.724560 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7f7c3f328d7e49db85f19e13aa462cd6 [id=0fee6d656a1b486cb1fb77abf4042dd8] alter complete (version 1)
I20260811 18:57:24.725972 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5580d47029684e6a8836966a2eee6853"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.727290 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5580d47029684e6a8836966a2eee6853 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.739465 4162 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.739792 4162 ts_tablet_manager.cc:1397] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.000s sys 0.008s
I20260811 18:57:24.741246 4162 raft_consensus.cc:348] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.741696 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 99b788d7e78c4945bb82fe86597aa634 (DEFAULT_TABLE table=default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a [id=2bc11a14c61347d5bee3feefee097dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.742477 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b788d7e78c4945bb82fe86597aa634. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.744107 3678 leader_election.cc:334] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:24.744511 3678 leader_election.cc:302] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.744832 3678 raft_consensus.cc:2670] T 151df87806694f46be5013592c68c01a P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.703991 4163 ts_tablet_manager.cc:1428] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.423s user 0.009s sys 0.013s
I20260811 18:57:24.745451 4163 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:24.741696 4162 raft_consensus.cc:374] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.746361 4162 raft_consensus.cc:729] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.746906 4162 consensus_queue.cc:260] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.749222 4162 ts_tablet_manager.cc:1428] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.003s
I20260811 18:57:24.751175 4162 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.752005 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 (DEFAULT_TABLE table=default.Table_69be92ae6ea74daf931e8ccd58a8dd4f [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.752815 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:24.711834 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17395ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:24.759400 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 (DEFAULT_TABLE table=default.Table_47326d9f3fac4cb7b76516843cf7bfd0 [id=33ec227ce4d54151bd3b046a7637b160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.759969 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2474ae972e9242b69f488b7a2fdbc481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.769438 4162 tablet_bootstrap.cc:654] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.769582 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0b40da8cb7324da9804e6fdafd8e8266
I20260811 18:57:24.770818 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 7be6d9578e7847c2822278486119390c (DEFAULT_TABLE table=default.Table_ea1ee62f594e4ee798829d582bc2b53f [id=62fea03d455b41d69f9e01c6da7f74fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.771898 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be6d9578e7847c2822278486119390c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.776978 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.777463 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 156781us
W20260811 18:57:24.778173 3824 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.135s (SENT)
I20260811 18:57:24.778730 3824 leader_election.cc:302] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.779109 3824 raft_consensus.cc:2670] T 405773849fb94800be61ad8a7a6055a4 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.780543 3824 leader_election.cc:334] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.388s (SENT)
I20260811 18:57:24.780781 3824 leader_election.cc:302] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:24.781785 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 19d7716739f848d78b9cbbd3975f3d44
I20260811 18:57:24.783689 4169 rpcz_store.cc:276] 0811 18:57:18.923927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.977560 (+ 53633us) service_pool.cc:227] Handling call
0811 18:57:24.420645 (+5443085us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:24.420659 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.491698 (+ 71039us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.636303 (+144605us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.640510 (+ 4207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5442379}
I20260811 18:57:24.784251 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.784538 3824 raft_consensus.cc:2670] T 2f9fa42654cf4ec18220e5fb173ddef7 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.787027 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2d0b59cafb154416952aecaa418e861f
W20260811 18:57:24.790515 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ffb196eaa8c2483aa71418747adda95a (table default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ffb196eaa8c2483aa71418747adda95a consensus state has no leader
I20260811 18:57:24.791738 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0b59cafb154416952aecaa418e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.796197 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fb3a9fdc3d594c1a84a5d8efa8490dfa consensus state has no leader
I20260811 18:57:24.797303 4169 tablet_replica.cc:330] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:24.797653 4169 raft_consensus.cc:2229] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:24.797923 4169 raft_consensus.cc:2258] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.800194 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3b40e44a2f4940af891eddf6c3b6293c
I20260811 18:57:24.800733 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 3b40e44a2f4940af891eddf6c3b6293c deleted and replaced by tablet 4da53cef719842f98d2190e487e8d653
I20260811 18:57:24.801088 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 7645a242bc4e4845ad49bfbe20b252f2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:24.803422 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 92e49846ab714dd294acdd09d49a83bd
I20260811 18:57:24.805277 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.806543 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 99b788d7e78c4945bb82fe86597aa634 (DEFAULT_TABLE table=default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a [id=2bc11a14c61347d5bee3feefee097dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.807631 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:24.807891 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 (DEFAULT_TABLE table=default.Table_69be92ae6ea74daf931e8ccd58a8dd4f [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.808800 4167 ts_tablet_manager.cc:1918] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.808617 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3baa014716584cb184d8a0e4299a8542"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.809523 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3baa014716584cb184d8a0e4299a8542 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.810822 4167 log.cc:1196] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:24.811322 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet f25ff8a9e3a944269e9799995c73ebe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.811338 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.811604 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c with delete_type TABLET_DATA_DELETED (replaced by tablet 4da53cef719842f98d2190e487e8d653) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.812069 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14b83cad2c2f4713b46f4d1fa3782f90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.812844 4167 ts_tablet_manager.cc:1939] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.815063 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 396) took 5878 ms. Trace:
I20260811 18:57:24.814973 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:24.815738 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.816385 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e49846ab714dd294acdd09d49a83bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.815199 4167 rpcz_store.cc:276] 0811 18:57:18.935352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.935739 (+ 387us) service_pool.cc:227] Handling call
0811 18:57:24.685525 (+5749786us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:24.685541 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.756663 (+ 71122us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.808548 (+ 51885us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.813355 (+ 4807us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5749599}
I20260811 18:57:24.820698 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.821203 5048 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:24.821593 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.821467 5048 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.822634 5048 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:24.823609 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_59f9d96102f349498c7f854e48e3d7c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.824237 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:24.824544 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59f9d96102f349498c7f854e48e3d7c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.824983 3676 leader_election.cc:341] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:24.826512 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 403) took 5777 ms. Trace:
I20260811 18:57:24.826738 4167 rpcz_store.cc:276] 0811 18:57:19.048915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.050433 (+ 1518us) service_pool.cc:227] Handling call
0811 18:57:24.826114 (+5775681us) ts_tablet_manager.cc:1145] Deleting tablet 0540e4e43ea34fbfbc66b6035c60684d
0811 18:57:24.826138 (+ 24us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.826289 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5754448}
I20260811 18:57:24.827018 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5fef93c0a9b546199a3217a8b8bd67dd" } new_table_name: "default.Table_a4daf4372b7a44babebcceb2ded88ce5" modify_external_catalogs: true
I20260811 18:57:24.814144 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e8da32e196c4b85943b62957165d308"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.830740 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 7be6d9578e7847c2822278486119390c (DEFAULT_TABLE table=default.Table_ea1ee62f594e4ee798829d582bc2b53f [id=62fea03d455b41d69f9e01c6da7f74fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.831364 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b40da8cb7324da9804e6fdafd8e8266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.832188 3549 catalog_manager.cc:5495] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:24.832669 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b40e44a2f4940af891eddf6c3b6293c with delete_type TABLET_DATA_DELETED (replaced by tablet 4da53cef719842f98d2190e487e8d653) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:24.834255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0540e4e43ea34fbfbc66b6035c60684d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0540e4e43ea34fbfbc66b6035c60684d already in progress: creating tablet
I20260811 18:57:24.834204 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.834587 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:24.834942 4167 tablet_replica.cc:330] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.835361 4167 raft_consensus.cc:2229] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.835879 4167 raft_consensus.cc:2258] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:24.835927 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e8da32e196c4b85943b62957165d308 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.839462 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:24.839915 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 (DEFAULT_TABLE table=default.Table_47326d9f3fac4cb7b76516843cf7bfd0 [id=33ec227ce4d54151bd3b046a7637b160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.840307 4167 ts_tablet_manager.cc:1905] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.840744 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.841363 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0b59cafb154416952aecaa418e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842005 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 92e49846ab714dd294acdd09d49a83bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842447 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:24.842988 5037 consensus_queue.cc:1035] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260811 18:57:24.843772 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99b788d7e78c4945bb82fe86597aa634. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.798810 4169 ts_tablet_manager.cc:1905] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:24.845090 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7f980c9b98994253b590548d86f6d145"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.845799 4163 tablet_bootstrap.cc:654] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:24.846505 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f980c9b98994253b590548d86f6d145 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:24.850486 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.854662 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be6d9578e7847c2822278486119390c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.855559 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.856287 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:24.856545 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 0190cbd859274fcfa09c6fab899c4843 with delete_type TABLET_DATA_DELETED (replaced by tablet 515d5b5afbd74ec5ab90a503aef2f4fb) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:24.858078 3676 consensus_peers.cc:487] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:24.858242 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2474ae972e9242b69f488b7a2fdbc481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.868125 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 1e65c28c0c0b4ae39b8438b4592e777b (DEFAULT_TABLE table=default.Table_a2ab815e5ecf491ab1ca7f1730ba148c [id=4d0cf7429dc94ee2ae7da259499680eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.868948 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e65c28c0c0b4ae39b8438b4592e777b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.902742 3720 tablet_service.cc:1467] Processing CreateTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.904753 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.915068 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.915810 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e120a45539f49bb82ab0fb95eac36f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.917829 4167 ts_tablet_manager.cc:1918] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:24.918076 4167 log.cc:1196] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:24.919030 4167 ts_tablet_manager.cc:1939] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:24.921872 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 404) took 5846 ms. Trace:
I20260811 18:57:24.922077 4167 rpcz_store.cc:276] 0811 18:57:19.075439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.075512 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:24.834674 (+5759162us) ts_tablet_manager.cc:1145] Deleting tablet 3744b3d608754de68c4c57b072d1dc15
0811 18:57:24.834690 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.890193 (+ 55503us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.917583 (+ 27390us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.921820 (+ 4237us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5758808}
I20260811 18:57:24.923022 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 405) took 5833 ms. Trace:
I20260811 18:57:24.923269 4167 rpcz_store.cc:276] 0811 18:57:19.089852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.091907 (+ 2055us) service_pool.cc:227] Handling call
0811 18:57:24.922662 (+5830755us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:24.922676 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.922887 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5830472}
I20260811 18:57:24.923647 4167 tablet_replica.cc:330] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:24.924074 4167 raft_consensus.cc:2229] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:24.924592 4167 raft_consensus.cc:2258] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:24.924998 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a4328549b8d4ad08d0135327263c811"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:24.925503 4167 ts_tablet_manager.cc:1905] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:24.925822 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a4328549b8d4ad08d0135327263c811 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:24.927475 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 27cda53b7ca54346a7a22bf26f5db990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27cda53b7ca54346a7a22bf26f5db990 already in progress: creating tablet
W20260811 18:57:24.929901 3676 consensus_peers.cc:487] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 405773849fb94800be61ad8a7a6055a4. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:24.934046 4162 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:24.934274 4162 ts_tablet_manager.cc:1397] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.183s user 0.007s sys 0.011s
I20260811 18:57:24.935489 4162 raft_consensus.cc:348] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.936133 4162 raft_consensus.cc:374] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:24.936420 4162 raft_consensus.cc:729] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:24.936834 4162 consensus_queue.cc:260] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:24.938015 4162 ts_tablet_manager.cc:1428] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:24.940471 4162 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:24.944043 4162 tablet_bootstrap.cc:654] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:24.949179 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca (DEFAULT_TABLE table=default.Table_03949bb266e742cbbe858496f37177f3 [id=73c7aa5e86eb4c468f6c967b2ac84f9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.949976 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.953562 3711 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4cc4dace944be894fe12690b18ccb (DEFAULT_TABLE table=default.Table_016d0800749f4083ad87c2a640e058c8 [id=c42c64b3e30043eca22cf0aebdf71d0c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:24.953930 3824 leader_election.cc:334] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.131s (SENT)
I20260811 18:57:24.954244 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4cc4dace944be894fe12690b18ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.954109 3824 leader_election.cc:302] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.961906 3824 raft_consensus.cc:2670] T f8eb041f3d704762868a048faec0e0fc P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.962877 3824 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.227s (SENT)
I20260811 18:57:24.963039 3824 leader_election.cc:302] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.963213 3824 raft_consensus.cc:2670] T 33d995d72f8142afaeb57a2c29910fa3 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.963939 3824 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.258s (SENT)
I20260811 18:57:24.964095 3824 leader_election.cc:302] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:24.964332 3824 raft_consensus.cc:2670] T 331396c9962f48ed9ea6dc66f1660902 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:24.967619 3678 leader_election.cc:334] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.049s (SENT)
I20260811 18:57:24.967809 3678 leader_election.cc:302] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:24.968060 3678 raft_consensus.cc:2670] T cf285cdf5235451bb1531cb294017101 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:24.969203 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 (DEFAULT_TABLE table=default.Table_f9693b748bd0490cb416e0cb36a62966 [id=4702ae470cf0451fa2fb675855aa6d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.969890 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37be4f7579fa4ab99ddbacfa80282b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.976966 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ae96e10a2ed4407c9d42a359d82d389e (DEFAULT_TABLE table=default.Table_dc16d2310b294ccb8c18945b693c6fce [id=b28c814fee894ee7a12d05872eea8e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.977598 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae96e10a2ed4407c9d42a359d82d389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:24.988941 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7bc942d59fd34beda3b8fb9e03e84cb2" } new_table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" modify_external_catalogs: true
W20260811 18:57:24.994097 3678 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.115s (SENT)
I20260811 18:57:24.990023 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61f28b085f594e1eaf7d7eb8905a7ba2" } new_table_name: "default.Table_3e8780219db840aeb566b570452e221b" modify_external_catalogs: true
I20260811 18:57:24.990909 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_609ebcc1b6e4405782a26228865b9460" } new_table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" modify_external_catalogs: true
W20260811 18:57:24.991501 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2b283002d0e4316844e482468f97c31 consensus state has no leader
I20260811 18:57:24.997946 3724 tablet_service.cc:1467] Processing CreateTablet for tablet be9288a76eab4606ba189d7de5813973 (DEFAULT_TABLE table=default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 [id=25867a79fa014eea81b48c95c9570ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:24.999465 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9288a76eab4606ba189d7de5813973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.011689 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b88a886800d34db0a94ec1fa36757f90 (DEFAULT_TABLE table=default.Table_2108d07d209a411da92158a80c5d08ab [id=6057586eaff54a40a6b1a4f50adf9cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.012494 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88a886800d34db0a94ec1fa36757f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.013417 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_919f7bdbd76f4707813e30d37dfc3767" } modify_external_catalogs: true
I20260811 18:57:25.013870 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_919f7bdbd76f4707813e30d37dfc3767" } modify_external_catalogs: true
I20260811 18:57:25.018965 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" } modify_external_catalogs: true
I20260811 18:57:25.019299 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c" } modify_external_catalogs: true
I20260811 18:57:25.021122 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5580d47029684e6a8836966a2eee6853" } modify_external_catalogs: true
I20260811 18:57:25.021524 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5580d47029684e6a8836966a2eee6853" } modify_external_catalogs: true
I20260811 18:57:25.028950 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab47bc9432074c05b97dce613fcbcdae" } new_table_name: "default.Table_540b89ef88904bb6b163c4b18480746e" modify_external_catalogs: true
I20260811 18:57:25.030284 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a0c48e6299546a2921a7a7db03f1032" } modify_external_catalogs: true
I20260811 18:57:25.030793 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6a0c48e6299546a2921a7a7db03f1032" } modify_external_catalogs: true
I20260811 18:57:25.032855 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_821a790ed161401bb9758d71a0760916" } modify_external_catalogs: true
I20260811 18:57:25.033110 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_821a790ed161401bb9758d71a0760916" } modify_external_catalogs: true
W20260811 18:57:25.051580 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260811 18:57:25.060714 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ffb196eaa8c2483aa71418747adda95a (DEFAULT_TABLE table=default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.061384 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb196eaa8c2483aa71418747adda95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.008134 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 1e65c28c0c0b4ae39b8438b4592e777b (DEFAULT_TABLE table=default.Table_a2ab815e5ecf491ab1ca7f1730ba148c [id=4d0cf7429dc94ee2ae7da259499680eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.062160 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e65c28c0c0b4ae39b8438b4592e777b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.069067 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8776bb47c4cb891ef13a2a1e36781 (DEFAULT_TABLE table=default.Table_d79d187445ee43d6864f03f905400405 [id=5c5f51b92b6242a3a9cb07db2e8b7daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.022881 4163 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.071064 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8776bb47c4cb891ef13a2a1e36781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.072609 4163 ts_tablet_manager.cc:1397] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.327s user 0.009s sys 0.000s
I20260811 18:57:25.073086 5091 consensus_queue.cc:1035] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:25.027243 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.074913 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.027825 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.078845 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e120a45539f49bb82ab0fb95eac36f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.028280 3854 tablet_service.cc:1467] Processing CreateTablet for tablet be9288a76eab4606ba189d7de5813973 (DEFAULT_TABLE table=default.Table_61f28b085f594e1eaf7d7eb8905a7ba2 [id=25867a79fa014eea81b48c95c9570ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.082895 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9288a76eab4606ba189d7de5813973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.029700 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca (DEFAULT_TABLE table=default.Table_03949bb266e742cbbe858496f37177f3 [id=73c7aa5e86eb4c468f6c967b2ac84f9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.084657 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.086930 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 3195082643264e0d90f70e1007ab9b24 (DEFAULT_TABLE table=default.Table_7075f060dbd942cca40c448af9eb5825 [id=40f69765767d4b23889527bc635f88d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.030217 3856 tablet_service.cc:1467] Processing CreateTablet for tablet e4f4cc4dace944be894fe12690b18ccb (DEFAULT_TABLE table=default.Table_016d0800749f4083ad87c2a640e058c8 [id=c42c64b3e30043eca22cf0aebdf71d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.090323 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f4cc4dace944be894fe12690b18ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.030766 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 (DEFAULT_TABLE table=default.Table_f9693b748bd0490cb416e0cb36a62966 [id=4702ae470cf0451fa2fb675855aa6d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.094851 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37be4f7579fa4ab99ddbacfa80282b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.031379 3861 tablet_service.cc:1467] Processing CreateTablet for tablet ae96e10a2ed4407c9d42a359d82d389e (DEFAULT_TABLE table=default.Table_dc16d2310b294ccb8c18945b693c6fce [id=b28c814fee894ee7a12d05872eea8e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.031903 3862 tablet_service.cc:1467] Processing CreateTablet for tablet b88a886800d34db0a94ec1fa36757f90 (DEFAULT_TABLE table=default.Table_2108d07d209a411da92158a80c5d08ab [id=6057586eaff54a40a6b1a4f50adf9cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.099466 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3195082643264e0d90f70e1007ab9b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.101473 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5a184b83373a4c089f2ca0169cbfcf66 (DEFAULT_TABLE table=default.Table_c7a814de93aa4d49a3ab5c7d2b167976 [id=ac945f840b754de791860a2415523df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.101841 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae96e10a2ed4407c9d42a359d82d389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.102393 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88a886800d34db0a94ec1fa36757f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.106592 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a184b83373a4c089f2ca0169cbfcf66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.106573 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.110623 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 810b9bdc330e40619f45c70e57724da0 (DEFAULT_TABLE table=default.Table_313ce0e9dd1042e2827c0eebceaf3729 [id=60db200412ce438194eda77eca5f7257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.117470 4162 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.120872 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d7716739f848d78b9cbbd3975f3d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.123520 3724 tablet_service.cc:1467] Processing CreateTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 (DEFAULT_TABLE table=default.Table_a1bf717c45ad424daa97f656cabd9db8 [id=6d01a59c66564bec9f1714a7a92c87d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.118373 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 219474fc820d40f49810209ac1c40afe (DEFAULT_TABLE table=default.Table_71aa50e0de9f470d9c9cceeb3b18cd90 [id=092f82e43fce4506883f0c8471022cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.132864 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1896700e724204a37439f134a4d988 (DEFAULT_TABLE table=default.Table_061ae2e2037348cab0a2499a98d273a6 [id=d3d44035163a4320bc0b5b79a4354fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.143640 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef (DEFAULT_TABLE table=default.Table_598311ac8416435286cfa4a6301380bd [id=cc4eea8d0a0c454ebb9e93d8ae619066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.143498 4162 ts_tablet_manager.cc:1397] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.006s
I20260811 18:57:25.143675 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810b9bdc330e40619f45c70e57724da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.143723 4167 ts_tablet_manager.cc:1918] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.143592 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ca82f01a33564be489962ef522cc6ed5 (DEFAULT_TABLE table=default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 [id=3c7a820c8c3944dd9d0cd6c8ed701f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.074203 4163 raft_consensus.cc:348] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.147630 4163 raft_consensus.cc:374] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.147787 4163 raft_consensus.cc:729] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.151211 4163 consensus_queue.cc:260] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.154806 4163 ts_tablet_manager.cc:1428] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.082s user 0.002s sys 0.000s
I20260811 18:57:25.155498 3714 tablet_service.cc:1467] Processing CreateTablet for tablet d11b889e9d044a049d0586382c263951 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.156041 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c74abe9dcf642e9b4d40df6ff3639ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.157711 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 125348us
I20260811 18:57:25.161823 3864 tablet_service.cc:1467] Processing CreateTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 (DEFAULT_TABLE table=default.Table_a1bf717c45ad424daa97f656cabd9db8 [id=6d01a59c66564bec9f1714a7a92c87d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.162061 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ffb196eaa8c2483aa71418747adda95a (DEFAULT_TABLE table=default.Table_24cdaf08a18b43e3bf356bb8b04f5017 [id=ccba6c748b6d42b298497668d922a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.162328 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21eb02db3de4c97878fa8badccaa6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.165256 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11b889e9d044a049d0586382c263951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.166173 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219474fc820d40f49810209ac1c40afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.162760 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 19d7716739f848d78b9cbbd3975f3d44 (DEFAULT_TABLE table=default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.166498 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb196eaa8c2483aa71418747adda95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.166779 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d7716739f848d78b9cbbd3975f3d44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.167340 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1896700e724204a37439f134a4d988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.167834 4167 log.cc:1196] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/331396c9962f48ed9ea6dc66f1660902
I20260811 18:57:25.168484 4167 ts_tablet_manager.cc:1939] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.170306 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 406) took 6079 ms. Trace:
I20260811 18:57:25.163166 3865 tablet_service.cc:1467] Processing CreateTablet for tablet ca82f01a33564be489962ef522cc6ed5 (DEFAULT_TABLE table=default.Table_de17e2e7bbbe429c9dea56a8b734f6b2 [id=3c7a820c8c3944dd9d0cd6c8ed701f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.170884 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca82f01a33564be489962ef522cc6ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.170713 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21eb02db3de4c97878fa8badccaa6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.171839 4162 raft_consensus.cc:348] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.175153 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef (DEFAULT_TABLE table=default.Table_598311ac8416435286cfa4a6301380bd [id=cc4eea8d0a0c454ebb9e93d8ae619066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.175650 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c74abe9dcf642e9b4d40df6ff3639ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.178799 3870 tablet_service.cc:1467] Processing CreateTablet for tablet d11b889e9d044a049d0586382c263951 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.179198 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d11b889e9d044a049d0586382c263951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.184165 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 39bfbcfc74834cbd9f4363e8c545d5a8 (DEFAULT_TABLE table=default.Table_ab47bc9432074c05b97dce613fcbcdae [id=3ecc967d0758417cbc1be9002c49d3f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.185226 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bfbcfc74834cbd9f4363e8c545d5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.172212 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca82f01a33564be489962ef522cc6ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.197212 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 39bfbcfc74834cbd9f4363e8c545d5a8 (DEFAULT_TABLE table=default.Table_ab47bc9432074c05b97dce613fcbcdae [id=3ecc967d0758417cbc1be9002c49d3f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.197839 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bfbcfc74834cbd9f4363e8c545d5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.198696 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ca37dd77c42a19146a99452ff87b0 (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.199256 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ca37dd77c42a19146a99452ff87b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.170946 4167 rpcz_store.cc:276] 0811 18:57:19.090018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.092235 (+ 2217us) service_pool.cc:227] Handling call
0811 18:57:24.923523 (+5831288us) ts_tablet_manager.cc:1145] Deleting tablet 331396c9962f48ed9ea6dc66f1660902
0811 18:57:24.923536 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.047717 (+124181us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.143119 (+ 95402us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.169187 (+ 26068us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5831020}
I20260811 18:57:25.205626 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 407) took 6090 ms. Trace:
I20260811 18:57:25.205847 4167 rpcz_store.cc:276] 0811 18:57:19.114792 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.114857 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:25.205104 (+6090247us) ts_tablet_manager.cc:1145] Deleting tablet a0116997a1c64e658045c319f1c45861
0811 18:57:25.205116 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.205426 (+ 310us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6090001}
I20260811 18:57:25.206704 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 408) took 6055 ms. Trace:
I20260811 18:57:25.206915 4167 rpcz_store.cc:276] 0811 18:57:19.151498 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.151928 (+ 430us) service_pool.cc:227] Handling call
0811 18:57:25.206312 (+6054384us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:25.206323 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.206493 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6054128}
I20260811 18:57:25.207482 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 420) took 5659 ms. Trace:
I20260811 18:57:25.207702 4167 rpcz_store.cc:276] 0811 18:57:19.547527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.547609 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:25.207189 (+5659580us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:25.207199 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.207314 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5646190}
I20260811 18:57:25.208552 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 331396c9962f48ed9ea6dc66f1660902 (table default.Table_11582479665b4ae58e3bfc2103bddde9 [id=bfb9674bd02e47df945ddd55d5fadfc0]) successfully deleted
I20260811 18:57:25.172533 4162 raft_consensus.cc:374] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:25.209010 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a0116997a1c64e658045c319f1c45861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0116997a1c64e658045c319f1c45861 already in progress: creating tablet
W20260811 18:57:25.209389 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ce94d37efd2245bf8cc71fdd90bd9735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce94d37efd2245bf8cc71fdd90bd9735 already in progress: creating tablet
I20260811 18:57:25.214599 4167 tablet_replica.cc:330] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.215000 4167 raft_consensus.cc:2229] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.218410 3678 leader_election.cc:334] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.466s (SENT)
I20260811 18:57:25.218673 3678 leader_election.cc:302] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.220942 3678 raft_consensus.cc:2670] T 2f9fa42654cf4ec18220e5fb173ddef7 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.221974 5037 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.222167 5037 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:25.223119 5037 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:25.223449 4167 raft_consensus.cc:2258] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.224370 4162 raft_consensus.cc:729] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.226832 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 (DEFAULT_TABLE table=default.Table_df08bd98f2ef430eaf0e10876b9e265e [id=ec725d0625094fc994ee36547333988d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.227882 4167 ts_tablet_manager.cc:1905] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.228789 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb831bb10f47749a2cfd3c88cf3785 (DEFAULT_TABLE table=default.Table_507c4f42c22847e39b4c700687a1687c [id=9648b8cbb8ad49669020fc67e803c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.230820 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2f5ce2b1d14a9a988cff644e196088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.239274 4162 consensus_queue.cc:260] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.242458 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 810b9bdc330e40619f45c70e57724da0 (DEFAULT_TABLE table=default.Table_313ce0e9dd1042e2827c0eebceaf3729 [id=60db200412ce438194eda77eca5f7257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.244415 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 810b9bdc330e40619f45c70e57724da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.243260 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 5a184b83373a4c089f2ca0169cbfcf66 (DEFAULT_TABLE table=default.Table_c7a814de93aa4d49a3ab5c7d2b167976 [id=ac945f840b754de791860a2415523df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.246829 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a184b83373a4c089f2ca0169cbfcf66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.243589 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 219474fc820d40f49810209ac1c40afe (DEFAULT_TABLE table=default.Table_71aa50e0de9f470d9c9cceeb3b18cd90 [id=092f82e43fce4506883f0c8471022cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.251423 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219474fc820d40f49810209ac1c40afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.242964 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 3195082643264e0d90f70e1007ab9b24 (DEFAULT_TABLE table=default.Table_7075f060dbd942cca40c448af9eb5825 [id=40f69765767d4b23889527bc635f88d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.252516 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3195082643264e0d90f70e1007ab9b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.244966 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8776bb47c4cb891ef13a2a1e36781 (DEFAULT_TABLE table=default.Table_d79d187445ee43d6864f03f905400405 [id=5c5f51b92b6242a3a9cb07db2e8b7daa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.254925 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8776bb47c4cb891ef13a2a1e36781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.259577 4162 ts_tablet_manager.cc:1428] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.107s user 0.002s sys 0.002s
I20260811 18:57:25.243991 4163 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:24.976162 4169 ts_tablet_manager.cc:1918] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.261334 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce521c08754bf391215ab98230fb9d (DEFAULT_TABLE table=default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 [id=d3b4765ced4341a2b9944a39cf19019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.257833 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1896700e724204a37439f134a4d988 (DEFAULT_TABLE table=default.Table_061ae2e2037348cab0a2499a98d273a6 [id=d3d44035163a4320bc0b5b79a4354fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.261690 4169 log.cc:1196] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e7259cf84f5c4c89a1b4510f743c2bc5
I20260811 18:57:25.268878 4169 ts_tablet_manager.cc:1939] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.270115 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1896700e724204a37439f134a4d988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.283921 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 402) took 6360 ms. Trace:
I20260811 18:57:25.284428 4169 rpcz_store.cc:276] 0811 18:57:18.923768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:18.996147 (+ 72379us) service_pool.cc:227] Handling call
0811 18:57:24.784116 (+5787969us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:24.797034 (+ 12918us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:24.972019 (+174985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:24.975986 (+ 3967us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.283890 (+307904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5787643}
I20260811 18:57:25.285118 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 414) took 6234 ms. Trace:
I20260811 18:57:25.262411 4162 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.256119 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b7b48a35acf1486bbb63680ca5da31dd (DEFAULT_TABLE table=default.Table_91dfb6056e5049bb8cca8a06b3a33b7f [id=68958402d14841a494933aa943a07ec0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:25.273311 3678 leader_election.cc:334] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.619s (SENT)
I20260811 18:57:25.285275 4169 rpcz_store.cc:276] 0811 18:57:19.050383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.084405 (+ 34022us) service_pool.cc:227] Handling call
0811 18:57:25.284953 (+6200548us) ts_tablet_manager.cc:1145] Deleting tablet 0540e4e43ea34fbfbc66b6035c60684d
0811 18:57:25.284966 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.285096 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6173607}
I20260811 18:57:25.285491 3678 leader_election.cc:302] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.285738 3678 raft_consensus.cc:2670] T dde27d8cc3eb4d09879e717fc90305fd P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.285750 4169 tablet_replica.cc:330] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.287209 4163 tablet_bootstrap.cc:654] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.283738 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb831bb10f47749a2cfd3c88cf3785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.289793 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cb3c2f76034640b398a94e7536595277"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.291200 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb3c2f76034640b398a94e7536595277 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.294112 4169 raft_consensus.cc:2229] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.294380 4169 raft_consensus.cc:2258] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.298976 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:25.300848 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b48a35acf1486bbb63680ca5da31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.307811 4162 tablet_bootstrap.cc:654] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.307757 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e7259cf84f5c4c89a1b4510f743c2bc5 (table default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]) successfully deleted
W20260811 18:57:25.308302 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0540e4e43ea34fbfbc66b6035c60684d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0540e4e43ea34fbfbc66b6035c60684d already in progress: creating tablet
I20260811 18:57:25.313279 4167 ts_tablet_manager.cc:1918] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:25.313503 4167 log.cc:1196] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.313848 5048 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.313985 4167 ts_tablet_manager.cc:1939] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.314010 5048 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.314706 5048 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.316565 4169 ts_tablet_manager.cc:1905] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.320616 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f389d2b7ff354ac4bab5d68b9ff201c6 consensus state has no leader
W20260811 18:57:25.320475 3824 leader_election.cc:341] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.325280 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
I20260811 18:57:25.326295 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 8304ac7709ca4392baec4cfa78ef5b41 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:25.327014 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:25.327734 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_75b5605dc2a14c9daf6d68e7094bcb39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.328745 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75b5605dc2a14c9daf6d68e7094bcb39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.329684 4162 tablet_bootstrap.cc:492] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.329854 4162 ts_tablet_manager.cc:1397] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.070s user 0.001s sys 0.005s
I20260811 18:57:25.326534 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:25.330861 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1dd4434500a64abcb81ee6eca6398ab8
W20260811 18:57:25.330873 3678 leader_election.cc:334] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.280s (SENT)
I20260811 18:57:25.331104 3678 leader_election.cc:302] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.331347 3678 raft_consensus.cc:2670] T 0e1efffade0f486d8d627e56e0be6ff6 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:25.331414 3676 leader_election.cc:341] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.331579 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b012268c924347d8a87ea52350010dc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.332463 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b012268c924347d8a87ea52350010dc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.334453 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
I20260811 18:57:25.335381 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b08e57f933b24cd0aee53441350e6775"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.336102 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b08e57f933b24cd0aee53441350e6775 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.337525 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 (DEFAULT_TABLE table=default.Table_df08bd98f2ef430eaf0e10876b9e265e [id=ec725d0625094fc994ee36547333988d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.337785 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 380c1f53b3ac4509a09b25a2800028af
I20260811 18:57:25.338644 4162 raft_consensus.cc:348] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.338729 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_614252f40e34465080bb490a1a7050ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.338971 4162 raft_consensus.cc:374] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.339037 4162 raft_consensus.cc:729] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:25.339385 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_614252f40e34465080bb490a1a7050ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.340034 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 421) took 5768 ms. Trace:
I20260811 18:57:25.340152 4167 rpcz_store.cc:276] 0811 18:57:19.571590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.571721 (+ 131us) service_pool.cc:227] Handling call
0811 18:57:25.208022 (+5636301us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:25.208033 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.267672 (+ 59639us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.313157 (+ 45485us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.340007 (+ 26850us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3427,"tablet-delete.queue_time_us":5635734}
I20260811 18:57:25.340898 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:25.341837 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be655558b83041fcace57520a599b8a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.342799 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:25.344055 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2f5ce2b1d14a9a988cff644e196088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.343981 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.344779 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.344935 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 422) took 5753 ms. Trace:
I20260811 18:57:25.345795 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb831bb10f47749a2cfd3c88cf3785 (DEFAULT_TABLE table=default.Table_507c4f42c22847e39b4c700687a1687c [id=9648b8cbb8ad49669020fc67e803c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.346187 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb831bb10f47749a2cfd3c88cf3785. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.346289 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1811e5f62a714f8388460dfcb1717e81 consensus state has no leader
I20260811 18:57:25.346890 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e8da32e196c4b85943b62957165d308" } new_table_name: "default.Table_51f392e10eb84f838bd93a0a30fe55f0" modify_external_catalogs: true
I20260811 18:57:25.347272 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ca37dd77c42a19146a99452ff87b0 (DEFAULT_TABLE table=default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.347611 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ca37dd77c42a19146a99452ff87b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.345054 4167 rpcz_store.cc:276] 0811 18:57:19.591115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.593952 (+ 2837us) service_pool.cc:227] Handling call
0811 18:57:25.340660 (+5746708us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:25.340691 (+ 31us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.344911 (+ 4220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5746058}
I20260811 18:57:25.348593 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 423) took 5753 ms. Trace:
I20260811 18:57:25.348708 4167 rpcz_store.cc:276] 0811 18:57:19.594898 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.597229 (+ 2331us) service_pool.cc:227] Handling call
0811 18:57:25.348308 (+5751079us) ts_tablet_manager.cc:1145] Deleting tablet 8d1e244addf5462499fcadaa3332cba1
0811 18:57:25.348327 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.348555 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5735902}
I20260811 18:57:25.349004 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c2247194db704f9abd54a8a0df1ca566 (table default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]) successfully deleted
W20260811 18:57:25.349418 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fc3f120758c6405dafd2ea0bf54c81d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3f120758c6405dafd2ea0bf54c81d8 already in progress: creating tablet
I20260811 18:57:25.349797 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.349846 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8d1e244addf5462499fcadaa3332cba1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d1e244addf5462499fcadaa3332cba1 already in progress: creating tablet
I20260811 18:57:25.339354 4162 consensus_queue.cc:260] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.352907 5048 consensus_queue.cc:1035] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:25.354095 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.356895 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be655558b83041fcace57520a599b8a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.359321 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 45fde25bcb40493aafb083e161aa6932
I20260811 18:57:25.359882 4162 ts_tablet_manager.cc:1428] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.030s user 0.002s sys 0.002s
I20260811 18:57:25.360321 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d86b05dec10e4f3da6b9c48860fe6d88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.361081 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d86b05dec10e4f3da6b9c48860fe6d88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.360956 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce521c08754bf391215ab98230fb9d (DEFAULT_TABLE table=default.Table_55c8f3c61fb24a7b8536cee1b85c9fc2 [id=d3b4765ced4341a2b9944a39cf19019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.361475 4162 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.362720 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 424) took 5708 ms. Trace:
I20260811 18:57:25.363443 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 380c1f53b3ac4509a09b25a2800028af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.362830 4167 rpcz_store.cc:276] 0811 18:57:19.654759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.655356 (+ 597us) service_pool.cc:227] Handling call
0811 18:57:25.349012 (+5693656us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:25.349026 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.362699 (+ 13673us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5693174}
I20260811 18:57:25.368431 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 425) took 5672 ms. Trace:
W20260811 18:57:25.368533 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 515d5b5afbd74ec5ab90a503aef2f4fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 515d5b5afbd74ec5ab90a503aef2f4fb already in progress: creating tablet
I20260811 18:57:25.371400 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.373468 3870 tablet_service.cc:1467] Processing CreateTablet for tablet b7b48a35acf1486bbb63680ca5da31dd (DEFAULT_TABLE table=default.Table_91dfb6056e5049bb8cca8a06b3a33b7f [id=68958402d14841a494933aa943a07ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.374166 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b48a35acf1486bbb63680ca5da31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.374854 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce521c08754bf391215ab98230fb9d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:25.375669 3824 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.371s (SENT)
I20260811 18:57:25.375785 3824 leader_election.cc:302] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.385061 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce521c08754bf391215ab98230fb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:25.368618 4167 rpcz_store.cc:276] 0811 18:57:19.695936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.706446 (+ 10510us) service_pool.cc:227] Handling call
0811 18:57:25.368174 (+5661728us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:25.368192 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.368401 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5649575}
W20260811 18:57:25.390691 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e18242f762d8407db0e2f8716890feb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e18242f762d8407db0e2f8716890feb4 already in progress: creating tablet
I20260811 18:57:25.390838 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 433) took 5486 ms. Trace:
I20260811 18:57:25.401304 5037 raft_consensus.cc:2726] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:25.405313 4162 tablet_bootstrap.cc:654] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.390942 4167 rpcz_store.cc:276] 0811 18:57:19.904502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.904736 (+ 234us) service_pool.cc:227] Handling call
0811 18:57:25.390683 (+5485947us) ts_tablet_manager.cc:1145] Deleting tablet 0bc6cce2c19746219b8b0fe54b13dcd6
0811 18:57:25.390698 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.390815 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5467517}
W20260811 18:57:25.407397 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bc6cce2c19746219b8b0fe54b13dcd6 already in progress: creating tablet
I20260811 18:57:25.407419 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 435) took 5495 ms. Trace:
I20260811 18:57:25.407543 4167 rpcz_store.cc:276] 0811 18:57:19.911648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.911737 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:25.407239 (+5495502us) ts_tablet_manager.cc:1145] Deleting tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
0811 18:57:25.407256 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.407393 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5484028}
W20260811 18:57:25.408119 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 already in progress: creating tablet
I20260811 18:57:25.408116 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 434) took 5501 ms. Trace:
I20260811 18:57:25.408224 4167 rpcz_store.cc:276] 0811 18:57:19.906407 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.906466 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:25.407987 (+5501521us) ts_tablet_manager.cc:1145] Deleting tablet 7a0e0d139c924f1dafe42da0bedada72
0811 18:57:25.408000 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.408098 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5481021}
I20260811 18:57:25.408797 4167 tablet_replica.cc:330] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:25.408813 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7a0e0d139c924f1dafe42da0bedada72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a0e0d139c924f1dafe42da0bedada72 already in progress: creating tablet
I20260811 18:57:25.409145 4167 raft_consensus.cc:2229] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.409528 4167 raft_consensus.cc:2258] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.419128 4167 ts_tablet_manager.cc:1905] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.424501 4162 tablet_bootstrap.cc:492] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.424700 4162 ts_tablet_manager.cc:1397] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.000s
I20260811 18:57:25.425925 4162 raft_consensus.cc:348] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:25.426247 4162 raft_consensus.cc:374] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.427120 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.427383 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.426319 4162 raft_consensus.cc:729] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.431052 4162 consensus_queue.cc:260] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:25.431288 3824 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.987s (SENT)
I20260811 18:57:25.431703 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.433209 4162 ts_tablet_manager.cc:1428] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260811 18:57:25.433514 4162 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.436091 3678 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.142s (SENT)
I20260811 18:57:25.431425 3824 leader_election.cc:302] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:25.443662 3824 raft_consensus.cc:2670] T f25ff8a9e3a944269e9799995c73ebe0 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.446967 4162 tablet_bootstrap.cc:654] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.452055 3678 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.412s (SENT)
I20260811 18:57:25.456877 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 380c1f53b3ac4509a09b25a2800028af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.459035 5048 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.459298 5048 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.460948 5048 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.461354 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:25.462002 3676 leader_election.cc:341] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:25.464601 4162 tablet_bootstrap.cc:492] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.464778 4162 ts_tablet_manager.cc:1397] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.001s
I20260811 18:57:25.465022 4167 ts_tablet_manager.cc:1918] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.466571 4167 log.cc:1196] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/33d995d72f8142afaeb57a2c29910fa3
I20260811 18:57:25.467876 4167 ts_tablet_manager.cc:1939] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:25.468691 3678 leader_election.cc:334] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.323s (SENT)
I20260811 18:57:25.488426 4163 tablet_bootstrap.cc:492] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.466120 4162 raft_consensus.cc:348] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.488602 4163 ts_tablet_manager.cc:1397] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.245s user 0.006s sys 0.002s
I20260811 18:57:25.488556 4162 raft_consensus.cc:374] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.488669 4162 raft_consensus.cc:729] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.488902 4162 consensus_queue.cc:260] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.489645 4162 ts_tablet_manager.cc:1428] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260811 18:57:25.490007 4162 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.490442 4163 raft_consensus.cc:348] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.490916 4163 raft_consensus.cc:374] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.491030 4163 raft_consensus.cc:729] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.491556 5091 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.491726 5091 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.491972 4163 consensus_queue.cc:260] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.492419 5091 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.488556 3678 leader_election.cc:302] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:25.492844 3678 raft_consensus.cc:2670] T a30bb384136f4a169a5751655437d5a3 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.493945 4162 tablet_bootstrap.cc:654] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.494302 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 443) took 5222 ms. Trace:
I20260811 18:57:25.494406 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:25.494889 3891 raft_consensus.cc:2454] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:25.495540 3676 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.495978 4167 rpcz_store.cc:276] 0811 18:57:20.271386 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.271453 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:25.408738 (+5137285us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:25.408748 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.432968 (+ 24220us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.464883 (+ 31915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.494278 (+ 29395us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5136635}
I20260811 18:57:25.496764 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 33d995d72f8142afaeb57a2c29910fa3 (table default.Table_2ace95690ebf4f619a57d3d7bd47bec5 [id=8baa9ddf0cd6467588b014d6e32d9272]) successfully deleted
I20260811 18:57:25.497190 5048 raft_consensus.cc:2781] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:25.497298 5048 raft_consensus.cc:482] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:25.497382 5048 raft_consensus.cc:3037] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.498492 4167 tablet_replica.cc:330] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.499699 4167 raft_consensus.cc:2229] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.501246 4167 raft_consensus.cc:2258] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.503517 4167 ts_tablet_manager.cc:1905] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.501758 5048 raft_consensus.cc:504] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.517437 5048 leader_election.cc:288] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:25.518134 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460e03cc14444c83b6c08ca194ca5a8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:25.518549 3891 raft_consensus.cc:3037] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.537071 4167 ts_tablet_manager.cc:1918] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:25.543530 3678 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.052s (SENT)
I20260811 18:57:25.553838 4167 log.cc:1196] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/405773849fb94800be61ad8a7a6055a4
I20260811 18:57:25.555231 4167 ts_tablet_manager.cc:1939] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.562282 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 444) took 5286 ms. Trace:
I20260811 18:57:25.563211 4167 rpcz_store.cc:276] 0811 18:57:20.272297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.272355 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:25.498436 (+5226081us) ts_tablet_manager.cc:1145] Deleting tablet 405773849fb94800be61ad8a7a6055a4
0811 18:57:25.498449 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.523359 (+ 24910us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.536930 (+ 13571us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.558314 (+ 21384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5225722}
I20260811 18:57:25.565042 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 405773849fb94800be61ad8a7a6055a4 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:25.565503 4167 tablet_replica.cc:330] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.567605 4167 raft_consensus.cc:2229] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.569563 4167 raft_consensus.cc:2258] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.572297 4167 ts_tablet_manager.cc:1905] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.536455 5037 consensus_queue.cc:1035] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:25.582281 4163 ts_tablet_manager.cc:1428] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.094s user 0.004s sys 0.000s
I20260811 18:57:25.582894 4163 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.591195 3891 raft_consensus.cc:2454] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:25.592170 3676 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.592729 5113 raft_consensus.cc:2781] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:25.593003 5113 raft_consensus.cc:686] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:25.593503 5113 consensus_queue.cc:237] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.630128 4167 ts_tablet_manager.cc:1918] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.632937 4167 log.cc:1196] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e7259cf84f5c4c89a1b4510f743c2bc5
I20260811 18:57:25.636049 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91d4fb4116244d9e8c08ce623bea3a2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.637161 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91d4fb4116244d9e8c08ce623bea3a2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.638288 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d995d72f8142afaeb57a2c29910fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:05 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.638623 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 460e03cc14444c83b6c08ca194ca5a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:06 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.639629 4167 ts_tablet_manager.cc:1939] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.642669 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 445) took 5369 ms. Trace:
I20260811 18:57:25.645859 4167 rpcz_store.cc:276] 0811 18:57:20.272923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.272984 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:25.565424 (+5292440us) ts_tablet_manager.cc:1145] Deleting tablet e7259cf84f5c4c89a1b4510f743c2bc5
0811 18:57:25.565446 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.598416 (+ 32970us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.629984 (+ 31568us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.642645 (+ 12661us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5279348}
I20260811 18:57:25.647450 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e7259cf84f5c4c89a1b4510f743c2bc5 (table default.Table_593bb155f3de4ed9a2913e3114d49e5b [id=038bbdafd74749b18d15986bdbe9a9b1]) successfully deleted
I20260811 18:57:25.648360 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 446) took 5361 ms. Trace:
I20260811 18:57:25.649287 4167 rpcz_store.cc:276] 0811 18:57:20.286892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.286958 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:25.647534 (+5360576us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:25.647546 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.648340 (+ 794us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5346867}
W20260811 18:57:25.651006 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6f3060822021431584a494e6941be756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f3060822021431584a494e6941be756 already in progress: creating tablet
I20260811 18:57:25.652355 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 447) took 5340 ms. Trace:
I20260811 18:57:25.652721 4163 tablet_bootstrap.cc:654] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.653440 4167 rpcz_store.cc:276] 0811 18:57:20.311638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.311710 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:25.651119 (+5339409us) ts_tablet_manager.cc:1145] Deleting tablet 7645a242bc4e4845ad49bfbe20b252f2
0811 18:57:25.651136 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.652333 (+ 1197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5338956}
I20260811 18:57:25.655570 4167 tablet_replica.cc:330] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.657433 4167 raft_consensus.cc:2229] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.659276 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7645a242bc4e4845ad49bfbe20b252f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7645a242bc4e4845ad49bfbe20b252f2 already in progress: creating tablet
I20260811 18:57:25.660595 4167 raft_consensus.cc:2258] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.661715 4169 ts_tablet_manager.cc:1918] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.661865 4169 log.cc:1196] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3744b3d608754de68c4c57b072d1dc15
I20260811 18:57:25.662303 4169 ts_tablet_manager.cc:1939] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.662729 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 415) took 6552 ms. Trace:
I20260811 18:57:25.662839 4169 rpcz_store.cc:276] 0811 18:57:19.110104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.111393 (+ 1289us) service_pool.cc:227] Handling call
0811 18:57:25.285623 (+6174230us) ts_tablet_manager.cc:1145] Deleting tablet 3744b3d608754de68c4c57b072d1dc15
0811 18:57:25.285645 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.472014 (+186369us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.661576 (+189562us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.662704 (+ 1128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6173769}
I20260811 18:57:25.663578 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 417) took 6544 ms. Trace:
I20260811 18:57:25.663684 4169 rpcz_store.cc:276] 0811 18:57:19.118709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.122812 (+ 4103us) service_pool.cc:227] Handling call
0811 18:57:25.663393 (+6540581us) ts_tablet_manager.cc:1145] Deleting tablet a0116997a1c64e658045c319f1c45861
0811 18:57:25.663408 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.663559 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6540238}
I20260811 18:57:25.663854 4162 tablet_bootstrap.cc:492] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:25.664234 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a0116997a1c64e658045c319f1c45861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0116997a1c64e658045c319f1c45861 already in progress: creating tablet
I20260811 18:57:25.664618 4162 ts_tablet_manager.cc:1397] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.175s user 0.013s sys 0.025s
I20260811 18:57:25.664958 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 416) took 6554 ms. Trace:
I20260811 18:57:25.665073 4169 rpcz_store.cc:276] 0811 18:57:19.110255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.150981 (+ 40726us) service_pool.cc:227] Handling call
0811 18:57:25.664812 (+6513831us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:25.664826 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.664937 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6513575}
I20260811 18:57:25.665580 4167 ts_tablet_manager.cc:1905] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.665688 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 27cda53b7ca54346a7a22bf26f5db990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27cda53b7ca54346a7a22bf26f5db990 already in progress: creating tablet
I20260811 18:57:25.666563 4162 raft_consensus.cc:348] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.666931 4162 raft_consensus.cc:374] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.667016 4162 raft_consensus.cc:729] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.667304 4162 consensus_queue.cc:260] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:25.668197 4162 ts_tablet_manager.cc:1428] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:25.674993 4162 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:25.683590 4163 tablet_bootstrap.cc:492] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.683766 4163 ts_tablet_manager.cc:1397] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.101s user 0.003s sys 0.002s
W20260811 18:57:25.683666 3678 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.083s (SENT)
I20260811 18:57:25.684933 4163 raft_consensus.cc:348] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.685242 4163 raft_consensus.cc:374] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.685297 4163 raft_consensus.cc:729] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.685539 4163 consensus_queue.cc:260] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.690917 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 418) took 6539 ms. Trace:
I20260811 18:57:25.691075 4169 rpcz_store.cc:276] 0811 18:57:19.151545 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.154980 (+ 3435us) service_pool.cc:227] Handling call
0811 18:57:25.690641 (+6535661us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:25.690660 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.690884 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6535293}
I20260811 18:57:25.691654 4169 tablet_replica.cc:330] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.691890 4169 raft_consensus.cc:2229] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:25.691980 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ce94d37efd2245bf8cc71fdd90bd9735 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce94d37efd2245bf8cc71fdd90bd9735 already in progress: creating tablet
I20260811 18:57:25.692093 4169 raft_consensus.cc:2258] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.692822 4169 ts_tablet_manager.cc:1905] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.699019 4162 tablet_bootstrap.cc:654] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.709841 4163 ts_tablet_manager.cc:1428] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.026s user 0.001s sys 0.002s
I20260811 18:57:25.712801 4163 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:25.715947 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 51839.3us
W20260811 18:57:25.718801 3678 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.094s (SENT)
I20260811 18:57:25.725694 4167 ts_tablet_manager.cc:1918] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:25.727201 4167 log.cc:1196] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:25.728574 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.728914 4167 ts_tablet_manager.cc:1939] T 61c8443df21949ff9bdc63fa6aa34108 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.729993 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 448) took 5417 ms. Trace:
I20260811 18:57:25.730090 4167 rpcz_store.cc:276] 0811 18:57:20.312333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.312382 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:25.655481 (+5343099us) ts_tablet_manager.cc:1145] Deleting tablet 61c8443df21949ff9bdc63fa6aa34108
0811 18:57:25.655494 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.673545 (+ 18051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.710255 (+ 36710us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.729968 (+ 19713us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5342873}
I20260811 18:57:25.738391 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cee8eb42df9b40cd85eb35169133787a" } new_table_name: "default.Table_e8871173d4364a52afb59e9b90e09f2c" modify_external_catalogs: true
I20260811 18:57:25.739435 4167 tablet_replica.cc:330] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.739773 4167 raft_consensus.cc:2229] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.740202 4167 raft_consensus.cc:2258] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.740919 4167 ts_tablet_manager.cc:1905] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.746877 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]) successfully deleted
W20260811 18:57:25.747567 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 has no consensus state
I20260811 18:57:25.748819 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_853572d26c9642b9b8ad616f574301dc" } modify_external_catalogs: true
I20260811 18:57:25.749854 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 86d690ec09524248bc147ad7408e2ee6 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.751068 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8304ac7709ca4392baec4cfa78ef5b41
I20260811 18:57:25.751993 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 8304ac7709ca4392baec4cfa78ef5b41 deleted and replaced by tablet d6c81850cdc74b4ab727bd2279162477
I20260811 18:57:25.753285 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.753690 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.754141 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5f4d9f42e99c454e88964d343262e534" } modify_external_catalogs: true
I20260811 18:57:25.754554 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5f4d9f42e99c454e88964d343262e534" } modify_external_catalogs: true
I20260811 18:57:25.755728 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_86e4bb4e07694c3a96ff484bd7944fd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.756212 4163 tablet_bootstrap.cc:654] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.757504 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e4bb4e07694c3a96ff484bd7944fd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.757967 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eff26ab40dd947859e4f83ff55b793d4" } new_table_name: "default.Table_fe2f572c66cd4b9a8656e0fd439d50c7" modify_external_catalogs: true
I20260811 18:57:25.758445 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6a3985ef18b44138437cd5cd54f34ed" } new_table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" modify_external_catalogs: true
I20260811 18:57:25.759087 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_853572d26c9642b9b8ad616f574301dc" } modify_external_catalogs: true
I20260811 18:57:25.769313 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet a443fe209556489abb47f4faa5d696e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.769623 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d3822fc2d4b569f223238a6464e0a with delete_type TABLET_DATA_DELETED (replaced by tablet 9bc5448a555944dfba8ccae09e25cbe9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.770677 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_176c852b84c14616bc0ea0f110671639" } modify_external_catalogs: true
I20260811 18:57:25.771175 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_176c852b84c14616bc0ea0f110671639" } modify_external_catalogs: true
I20260811 18:57:25.793570 4169 ts_tablet_manager.cc:1918] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.793751 4169 log.cc:1196] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:25.794250 4169 ts_tablet_manager.cc:1939] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:25.796452 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c05597a50f6e4aea9c286eae0cf9a54d" } new_table_name: "default.Table_c3c124426696404c9e9cb5200f63055e" modify_external_catalogs: true
I20260811 18:57:25.799937 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.800232 4163 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.802223 4163 ts_tablet_manager.cc:1397] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.090s user 0.002s sys 0.006s
I20260811 18:57:25.803313 4163 raft_consensus.cc:348] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.803599 4163 raft_consensus.cc:374] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.803645 4163 raft_consensus.cc:729] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.803879 4163 consensus_queue.cc:260] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.804417 4162 tablet_bootstrap.cc:492] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.804550 4162 ts_tablet_manager.cc:1397] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.130s user 0.000s sys 0.008s
I20260811 18:57:25.805531 4162 raft_consensus.cc:348] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.805755 4162 raft_consensus.cc:374] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.805809 4162 raft_consensus.cc:729] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:25.805961 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bfb54175c834517b7af291c942333cd consensus state has no leader
W20260811 18:57:25.806041 3824 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.608s (SENT)
I20260811 18:57:25.806174 3824 leader_election.cc:302] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.806051 4162 consensus_queue.cc:260] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:25.806321 3824 raft_consensus.cc:2670] T 987b9ac6859444e3a86c56af786cc84e P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.806710 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f" } modify_external_catalogs: true
I20260811 18:57:25.806991 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91dfb6056e5049bb8cca8a06b3a33b7f" } modify_external_catalogs: true
I20260811 18:57:25.808039 4163 ts_tablet_manager.cc:1428] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260811 18:57:25.808182 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3b6378d1219433cb724148f1ae74b25 consensus state has no leader
I20260811 18:57:25.808346 4163 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.808727 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fc78208e540445719638da3757163df5" } modify_external_catalogs: true
I20260811 18:57:25.809036 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fc78208e540445719638da3757163df5" } modify_external_catalogs: true
W20260811 18:57:25.809886 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35ebc32eb5d5443eaaeba245b566f98a consensus state has no leader
I20260811 18:57:25.810379 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 84e152c74f33463e83843b92c0783803 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.811923 4162 ts_tablet_manager.cc:1428] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 18:57:25.812242 4162 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.814085 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_7cabaea271024c3daf3ffc96c5825f1e [id=bd974444a6c546959eb6442a80543187], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
I20260811 18:57:25.814782 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:25.816026 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.816223 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 8d1e244addf5462499fcadaa3332cba1 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:25.816313 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d564ad80064912a8ce8e31c1b6c4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.822225 4167 ts_tablet_manager.cc:1918] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:25.822428 4167 log.cc:1196] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6381a951a966473c80670b0dd1aff062
I20260811 18:57:25.823112 4167 ts_tablet_manager.cc:1939] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.823716 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 450) took 5314 ms. Trace:
I20260811 18:57:25.823808 4167 rpcz_store.cc:276] 0811 18:57:20.509040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.509106 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:25.739355 (+5230249us) ts_tablet_manager.cc:1145] Deleting tablet 6381a951a966473c80670b0dd1aff062
0811 18:57:25.739375 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.758619 (+ 19244us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.762282 (+ 3663us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.823694 (+ 61412us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5225900}
I20260811 18:57:25.824364 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_4da51fec1a044237ab13da18366dde64 [id=263eb849fd614011a9dd641aeb9126d0]) successfully deleted
I20260811 18:57:25.824420 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 463) took 5001 ms. Trace:
I20260811 18:57:25.824501 4167 rpcz_store.cc:276] 0811 18:57:20.822748 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.822839 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:25.824277 (+5001438us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:25.824289 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.824402 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4996681}
W20260811 18:57:25.825165 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 43ad6993b63e4bbd98c8cc52f1a53451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43ad6993b63e4bbd98c8cc52f1a53451 already in progress: creating tablet
I20260811 18:57:25.827071 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 464) took 4993 ms. Trace:
I20260811 18:57:25.827206 4167 rpcz_store.cc:276] 0811 18:57:20.833426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.833568 (+ 142us) service_pool.cc:227] Handling call
0811 18:57:25.826932 (+4993364us) ts_tablet_manager.cc:1145] Deleting tablet 91818d0cbe144a67a458a6648968bf5b
0811 18:57:25.826943 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.827050 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4991372}
I20260811 18:57:25.827718 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 467) took 4960 ms. Trace:
I20260811 18:57:25.827812 4167 rpcz_store.cc:276] 0811 18:57:20.867330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.867586 (+ 256us) service_pool.cc:227] Handling call
0811 18:57:25.827596 (+4960010us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:25.827608 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.827699 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957788}
I20260811 18:57:25.828078 5037 raft_consensus.cc:482] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:25.828289 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 91818d0cbe144a67a458a6648968bf5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91818d0cbe144a67a458a6648968bf5b already in progress: creating tablet
I20260811 18:57:25.828249 5037 raft_consensus.cc:504] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:25.828827 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 059b325df1db4fde9d4b26b559121316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059b325df1db4fde9d4b26b559121316 already in progress: creating tablet
I20260811 18:57:25.829075 5037 leader_election.cc:288] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:25.829492 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae consensus state has no leader
I20260811 18:57:25.830240 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 465) took 4989 ms. Trace:
I20260811 18:57:25.830330 4167 rpcz_store.cc:276] 0811 18:57:20.840379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.840503 (+ 124us) service_pool.cc:227] Handling call
0811 18:57:25.830033 (+4989530us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:25.830043 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.830222 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4959989}
W20260811 18:57:25.830936 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b6f36ab6bb774257b5c2692c64eff63d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6f36ab6bb774257b5c2692c64eff63d already in progress: creating tablet
W20260811 18:57:25.831367 3824 leader_election.cc:341] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.833345 4167 tablet_replica.cc:330] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.833576 4167 raft_consensus.cc:2229] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.833174 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1af5a92fcee4bc4898572e55da333b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.833884 4167 raft_consensus.cc:2258] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.833891 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 419) took 6568 ms. Trace:
I20260811 18:57:25.833976 4169 rpcz_store.cc:276] 0811 18:57:19.265679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.265773 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:25.691601 (+6425828us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:25.691613 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.747207 (+ 55594us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.793439 (+ 46232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.833869 (+ 40430us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6376377}
I20260811 18:57:25.834615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 431) took 6355 ms. Trace:
I20260811 18:57:25.834532 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 828993fb34bb4c62bce2a9322d7c992a (table default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]) successfully deleted
I20260811 18:57:25.834753 4169 rpcz_store.cc:276] 0811 18:57:19.479453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.485654 (+ 6201us) service_pool.cc:227] Handling call
0811 18:57:25.834470 (+6348816us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:25.834486 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.834596 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6322417}
I20260811 18:57:25.835242 4169 tablet_replica.cc:330] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:25.835650 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1af5a92fcee4bc4898572e55da333b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.835927 4169 raft_consensus.cc:2229] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.836140 4169 raft_consensus.cc:2258] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.836872 4169 ts_tablet_manager.cc:1905] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.845615 4162 tablet_bootstrap.cc:654] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.848906 4163 tablet_bootstrap.cc:654] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.855180 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15425ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:25.857146 4167 ts_tablet_manager.cc:1905] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.857858 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 6d28c486e408451a990ae3905ef3c2ff from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:25.873435 3824 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.118s (SENT)
I20260811 18:57:25.873608 3824 leader_election.cc:302] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:25.874076 5037 raft_consensus.cc:2726] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:25.873903 4163 tablet_bootstrap.cc:492] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:25.876745 4163 ts_tablet_manager.cc:1397] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.068s user 0.006s sys 0.001s
W20260811 18:57:25.885480 3824 leader_election.cc:334] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.245s (SENT)
I20260811 18:57:25.885659 3824 leader_election.cc:302] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:25.885874 3824 raft_consensus.cc:2670] T f1a55a42d1e64f04ad7f433614a3a1aa P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:25.885505 4163 raft_consensus.cc:348] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.887008 4163 raft_consensus.cc:374] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.887087 4163 raft_consensus.cc:729] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.888481 5037 raft_consensus.cc:482] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.888665 5037 raft_consensus.cc:504] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.889678 5037 leader_election.cc:288] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:25.890810 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "283c3fe4f41f4185ab4488d774e46299" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:25.891149 3744 raft_consensus.cc:2454] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:25.891829 3824 leader_election.cc:302] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.891922 4167 ts_tablet_manager.cc:1918] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:25.892040 4167 log.cc:1196] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:25.892316 5037 raft_consensus.cc:2781] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:25.892396 5037 raft_consensus.cc:482] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:25.892454 5037 raft_consensus.cc:3037] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.892509 4167 ts_tablet_manager.cc:1939] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:25.892961 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 466) took 5041 ms. Trace:
I20260811 18:57:25.893042 4167 rpcz_store.cc:276] 0811 18:57:20.851895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.852357 (+ 462us) service_pool.cc:227] Handling call
0811 18:57:25.833284 (+4980927us) ts_tablet_manager.cc:1145] Deleting tablet a732c700ff79432aaf1f0d1c1e90aabe
0811 18:57:25.833298 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.861672 (+ 28374us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.891806 (+ 30134us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.892944 (+ 1138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4921218}
I20260811 18:57:25.893671 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a732c700ff79432aaf1f0d1c1e90aabe (table default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]) successfully deleted
I20260811 18:57:25.896575 5037 raft_consensus.cc:504] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.898761 4163 consensus_queue.cc:260] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.899502 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.899844 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
W20260811 18:57:25.900275 3678 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.062s (SENT)
I20260811 18:57:25.901022 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "283c3fe4f41f4185ab4488d774e46299" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:25.901224 3744 raft_consensus.cc:3037] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:25.902822 4163 ts_tablet_manager.cc:1428] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.026s user 0.001s sys 0.002s
I20260811 18:57:25.903101 4163 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:25.904909 3744 raft_consensus.cc:2454] T 283c3fe4f41f4185ab4488d774e46299 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:25.911271 4162 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:25.911423 4162 ts_tablet_manager.cc:1397] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.099s user 0.001s sys 0.009s
I20260811 18:57:25.916056 4162 raft_consensus.cc:348] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.916365 4162 raft_consensus.cc:374] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:25.916427 4162 raft_consensus.cc:729] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:25.916663 4162 consensus_queue.cc:260] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.917419 4162 ts_tablet_manager.cc:1428] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:25.917708 4162 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:25.918903 3678 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.235s (SENT)
I20260811 18:57:25.919701 3824 leader_election.cc:302] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:25.922755 5113 raft_consensus.cc:482] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:25.922921 5113 raft_consensus.cc:504] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.923638 5113 leader_election.cc:288] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:25.925386 3676 leader_election.cc:341] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: a443fe209556489abb47f4faa5d696e5
I20260811 18:57:25.927587 4162 tablet_bootstrap.cc:654] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:25.929455 4167 tablet_replica.cc:330] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:25.930559 4167 raft_consensus.cc:2229] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:25.930761 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
I20260811 18:57:25.932545 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.933039 5037 leader_election.cc:288] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:25.933267 4167 raft_consensus.cc:2258] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.933005 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e89523faca1046669de08be2f6b963dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.933763 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e89523faca1046669de08be2f6b963dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.934036 4167 ts_tablet_manager.cc:1905] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:25.937073 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.944876 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d7e8ba858b0c44d588499e1463673957"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.945350 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.920194 5080 raft_consensus.cc:2781] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:25.945657 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7e8ba858b0c44d588499e1463673957 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.945675 5080 raft_consensus.cc:686] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:25.945993 5080 consensus_queue.cc:237] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:25.947744 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e3a4280bd164399a5346ffaa6f40b8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.948199 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.948426 4163 tablet_bootstrap.cc:654] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:25.948577 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e3a4280bd164399a5346ffaa6f40b8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.948962 4169 ts_tablet_manager.cc:1918] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:25.949090 4169 log.cc:1196] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c2247194db704f9abd54a8a0df1ca566
I20260811 18:57:25.951275 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b069c186694049998175e3d83241b318"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.951694 4169 ts_tablet_manager.cc:1939] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:25.952060 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b069c186694049998175e3d83241b318 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:25.952776 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 84e152c74f33463e83843b92c0783803
I20260811 18:57:25.953190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 432) took 6416 ms. Trace:
I20260811 18:57:25.953415 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 84e152c74f33463e83843b92c0783803 deleted and replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780
I20260811 18:57:25.953306 4169 rpcz_store.cc:276] 0811 18:57:19.537253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.540139 (+ 2886us) service_pool.cc:227] Handling call
0811 18:57:25.835188 (+6295049us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:25.835202 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.871425 (+ 36223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.948849 (+ 77424us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:25.953169 (+ 4320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6234725}
I20260811 18:57:25.953824 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 434) took 6353 ms. Trace:
I20260811 18:57:25.953913 4169 rpcz_store.cc:276] 0811 18:57:19.600317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.602562 (+ 2245us) service_pool.cc:227] Handling call
0811 18:57:25.953653 (+6351091us) ts_tablet_manager.cc:1145] Deleting tablet 8d1e244addf5462499fcadaa3332cba1
0811 18:57:25.953667 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.953805 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6350661}
I20260811 18:57:25.954677 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 649cb98f553a418b911e307fdaf49917
I20260811 18:57:25.954918 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e152c74f33463e83843b92c0783803 with delete_type TABLET_DATA_DELETED (replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.956223 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 649cb98f553a418b911e307fdaf49917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.956271 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1cd8ec7574294bc6886f42d65935c0a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.957266 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 86d690ec09524248bc147ad7408e2ee6
I20260811 18:57:25.958748 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 86d690ec09524248bc147ad7408e2ee6 deleted and replaced by tablet b6c70b4163634984bd02dd9b47ffb472
I20260811 18:57:25.959193 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88a784338b5e4928be15e4679b3f3304"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:25.959474 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d28c486e408451a990ae3905ef3c2ff
I20260811 18:57:25.959724 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 84e152c74f33463e83843b92c0783803 with delete_type TABLET_DATA_DELETED (replaced by tablet b27f9dbb52f34687a33c2ebf20e8f780) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960297 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d690ec09524248bc147ad7408e2ee6 with delete_type TABLET_DATA_DELETED (replaced by tablet b6c70b4163634984bd02dd9b47ffb472) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960403 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 649cb98f553a418b911e307fdaf49917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.954272 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 433) took 6354 ms. Trace:
I20260811 18:57:25.960629 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d28c486e408451a990ae3905ef3c2ff with delete_type TABLET_DATA_DELETED (replaced by tablet bb1cb45730b14b998a8f547983b83823) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:25.960610 4169 rpcz_store.cc:276] 0811 18:57:19.600151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.607558 (+ 7407us) service_pool.cc:227] Handling call
0811 18:57:25.954166 (+6346608us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:25.954178 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.954257 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6346240}
I20260811 18:57:25.961313 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 435) took 6298 ms. Trace:
I20260811 18:57:25.961428 4169 rpcz_store.cc:276] 0811 18:57:19.663125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.663192 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:25.961075 (+6297883us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:25.961089 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.961292 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6291061}
I20260811 18:57:25.961581 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 6d28c486e408451a990ae3905ef3c2ff deleted and replaced by tablet bb1cb45730b14b998a8f547983b83823
I20260811 18:57:25.961828 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 436) took 6281 ms. Trace:
I20260811 18:57:25.961925 4169 rpcz_store.cc:276] 0811 18:57:19.680169 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.680241 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:25.961712 (+6281471us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:25.961725 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.961811 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6281088}
I20260811 18:57:25.962270 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 442) took 6057 ms. Trace:
I20260811 18:57:25.962373 4169 rpcz_store.cc:276] 0811 18:57:19.904745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.904860 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:25.962162 (+6057302us) ts_tablet_manager.cc:1145] Deleting tablet 0bc6cce2c19746219b8b0fe54b13dcd6
0811 18:57:25.962174 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.962254 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6056381}
I20260811 18:57:25.962741 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 444) took 6054 ms. Trace:
I20260811 18:57:25.962811 4169 rpcz_store.cc:276] 0811 18:57:19.908425 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.908510 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:25.962656 (+6054146us) ts_tablet_manager.cc:1145] Deleting tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4
0811 18:57:25.962665 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.962729 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6035313}
I20260811 18:57:25.962911 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c2247194db704f9abd54a8a0df1ca566 (table default.Table_e4b52f563c5f4629a03019c85bfbb653 [id=bbb15eceae984178b8b903105c78afc4]) successfully deleted
I20260811 18:57:25.963037 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 443) took 6055 ms. Trace:
I20260811 18:57:25.963093 4169 rpcz_store.cc:276] 0811 18:57:19.907149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:19.907988 (+ 839us) service_pool.cc:227] Handling call
0811 18:57:25.962965 (+6054977us) ts_tablet_manager.cc:1145] Deleting tablet 7a0e0d139c924f1dafe42da0bedada72
0811 18:57:25.962973 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963027 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6035112}
W20260811 18:57:25.963307 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8d1e244addf5462499fcadaa3332cba1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d1e244addf5462499fcadaa3332cba1 already in progress: creating tablet
I20260811 18:57:25.963361 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 451) took 5726 ms. Trace:
I20260811 18:57:25.963429 4169 rpcz_store.cc:276] 0811 18:57:20.236628 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.237034 (+ 406us) service_pool.cc:227] Handling call
0811 18:57:25.963252 (+5726218us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:25.963260 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963345 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5706677}
I20260811 18:57:25.963753 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 452) took 5713 ms. Trace:
I20260811 18:57:25.963845 4169 rpcz_store.cc:276] 0811 18:57:20.250514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.250583 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:25.963648 (+5713065us) ts_tablet_manager.cc:1145] Deleting tablet 7645a242bc4e4845ad49bfbe20b252f2
0811 18:57:25.963659 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.963737 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5696989}
I20260811 18:57:25.964433 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d690ec09524248bc147ad7408e2ee6 with delete_type TABLET_DATA_DELETED (replaced by tablet b6c70b4163634984bd02dd9b47ffb472) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.964825 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b7b48a35acf1486bbb63680ca5da31dd
I20260811 18:57:25.964954 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d28c486e408451a990ae3905ef3c2ff with delete_type TABLET_DATA_DELETED (replaced by tablet bb1cb45730b14b998a8f547983b83823) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:25.965935 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b48a35acf1486bbb63680ca5da31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:25.966748 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fc3f120758c6405dafd2ea0bf54c81d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc3f120758c6405dafd2ea0bf54c81d8 already in progress: creating tablet
W20260811 18:57:25.967113 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 515d5b5afbd74ec5ab90a503aef2f4fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 515d5b5afbd74ec5ab90a503aef2f4fb already in progress: creating tablet
I20260811 18:57:25.958441 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a905052284004a77adf260546090e666"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:25.969015 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a905052284004a77adf260546090e666 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.970794 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cd8ec7574294bc6886f42d65935c0a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.972198 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88a784338b5e4928be15e4679b3f3304 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:25.973557 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e18242f762d8407db0e2f8716890feb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e18242f762d8407db0e2f8716890feb4 already in progress: creating tablet
W20260811 18:57:25.973994 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bc6cce2c19746219b8b0fe54b13dcd6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bc6cce2c19746219b8b0fe54b13dcd6 already in progress: creating tablet
W20260811 18:57:25.974493 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d7812f2c2d44ef4b0f7be0fc33ac8a4 already in progress: creating tablet
I20260811 18:57:25.974500 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b48a35acf1486bbb63680ca5da31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:25.974871 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7a0e0d139c924f1dafe42da0bedada72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a0e0d139c924f1dafe42da0bedada72 already in progress: creating tablet
W20260811 18:57:25.975479 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6f3060822021431584a494e6941be756 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f3060822021431584a494e6941be756 already in progress: creating tablet
W20260811 18:57:25.975775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7645a242bc4e4845ad49bfbe20b252f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7645a242bc4e4845ad49bfbe20b252f2 already in progress: creating tablet
I20260811 18:57:25.975199 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69be92ae6ea74daf931e8ccd58a8dd4f" } new_table_name: "default.Table_e98a0fba26cb4171bf8eb772249c3e47" modify_external_catalogs: true
W20260811 18:57:25.978710 3678 leader_election.cc:334] T d9421bc220774f66b90bf537a47f1e13 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.383s (SENT)
I20260811 18:57:25.982561 4169 tablet_replica.cc:330] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:25.982606 3550 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 23f41a2638e74ba19c1c527b9fc796df (table default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c]). Expected version 2 got 0
I20260811 18:57:25.982872 4169 raft_consensus.cc:2229] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:25.983295 4169 raft_consensus.cc:2258] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:25.983104 3550 catalog_manager.cc:5495] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:25.984115 4169 ts_tablet_manager.cc:1905] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:25.991371 3678 leader_election.cc:334] T 331396c9962f48ed9ea6dc66f1660902 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.281s (SENT)
I20260811 18:57:25.993602 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a3209bc9f1194850ae8fde67d25c8a0d (DEFAULT_TABLE table=default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d [id=ccecc852942941bc9e47d9af9c07495f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:25.994124 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3209bc9f1194850ae8fde67d25c8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.001608 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bcb9560450d14ad392e0db4330f60317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.002462 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcb9560450d14ad392e0db4330f60317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.003962 3711 tablet_service.cc:1467] Processing CreateTablet for tablet a3209bc9f1194850ae8fde67d25c8a0d (DEFAULT_TABLE table=default.Table_5a4ebf675d3d44f8a8fa6a8e1fa9a39d [id=ccecc852942941bc9e47d9af9c07495f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.004491 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3209bc9f1194850ae8fde67d25c8a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.009420 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 2523fa07587246d994eca722c7032105 (DEFAULT_TABLE table=default.Table_3baa014716584cb184d8a0e4299a8542 [id=35354689f6714dbc929e448b7b00d640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.010004 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2523fa07587246d994eca722c7032105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.011042 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 (DEFAULT_TABLE table=default.Table_51f392e10eb84f838bd93a0a30fe55f0 [id=f1048594851646469a3f7664e216d924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.011536 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.014925 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543 (DEFAULT_TABLE table=default.Table_51f392e10eb84f838bd93a0a30fe55f0 [id=f1048594851646469a3f7664e216d924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.015343 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ddcdd987c40e3927b7b8ddb6f0543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.019209 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2a926bfdbff42feaa7cfed27e0c04a0" } new_table_name: "default.Table_ab9ee42f727a415ab19b78c0e5bd013a" modify_external_catalogs: true
I20260811 18:57:26.021100 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 05cba6b392b642e59321570aac3541d0 (DEFAULT_TABLE table=default.Table_59f9d96102f349498c7f854e48e3d7c5 [id=0b705b356df3452a966e56ef0d684159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.021282 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4b52f563c5f4629a03019c85bfbb653" } new_table_name: "default.Table_f2f1e0bdeb7a45e3b9e6041aa2ac2578" modify_external_catalogs: true
I20260811 18:57:26.021564 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cba6b392b642e59321570aac3541d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.023445 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_771bec99eb584420a45d22e9b038229a" } new_table_name: "default.Table_21287a00ce0a446fb09ba45cacede2c2" modify_external_catalogs: true
I20260811 18:57:26.025233 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_85a94fea83f84948a14090e2deeac956"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.025558 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 05cba6b392b642e59321570aac3541d0 (DEFAULT_TABLE table=default.Table_59f9d96102f349498c7f854e48e3d7c5 [id=0b705b356df3452a966e56ef0d684159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.026042 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05cba6b392b642e59321570aac3541d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.026191 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85a94fea83f84948a14090e2deeac956 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.031221 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 2523fa07587246d994eca722c7032105 (DEFAULT_TABLE table=default.Table_3baa014716584cb184d8a0e4299a8542 [id=35354689f6714dbc929e448b7b00d640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.032091 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2523fa07587246d994eca722c7032105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.036257 4162 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.036413 4162 ts_tablet_manager.cc:1397] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.119s user 0.002s sys 0.003s
I20260811 18:57:26.037451 4162 raft_consensus.cc:348] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.037376 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9e2e268d15894407ab276766ff611c86" } new_table_name: "default.Table_102e5367d375402c9800357142894fa0" modify_external_catalogs: true
I20260811 18:57:26.037665 4162 raft_consensus.cc:374] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.037725 4162 raft_consensus.cc:729] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.039409 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 (DEFAULT_TABLE table=default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 [id=c9676ca3a1a8471bb1dfc3dbf19d7861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.040067 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8146f47d9ae48889ac7efc954d4a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.040831 4162 consensus_queue.cc:260] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.042960 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 consensus state has no leader
I20260811 18:57:26.045965 3715 tablet_service.cc:1467] Processing CreateTablet for tablet f13a14bd64704798bf6de4fc05201a31 (DEFAULT_TABLE table=default.Table_14b83cad2c2f4713b46f4d1fa3782f90 [id=1fa3a7f5ea7646428fc8d129dbf153fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.046427 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13a14bd64704798bf6de4fc05201a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.051090 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.053561 4162 ts_tablet_manager.cc:1428] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260811 18:57:26.053997 4162 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.056367 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f13a14bd64704798bf6de4fc05201a31 (DEFAULT_TABLE table=default.Table_14b83cad2c2f4713b46f4d1fa3782f90 [id=1fa3a7f5ea7646428fc8d129dbf153fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.037415 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13" } modify_external_catalogs: true
I20260811 18:57:26.056818 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13a14bd64704798bf6de4fc05201a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.060091 4167 ts_tablet_manager.cc:1918] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:26.060075 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 761902f7366d43ffbde226a0ade2e188 (DEFAULT_TABLE table=default.Table_0036c28f59cb4f229da243389602bbd9 [id=9b456a91de20444dbe8abd84177b3073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.060552 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761902f7366d43ffbde226a0ade2e188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.061918 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 99e9474ef8814514832f0e48c2b3495f (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.058161 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd454d73c8da4bf79e68487caa3dd843"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.062265 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e9474ef8814514832f0e48c2b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.062572 4167 log.cc:1196] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f8eb041f3d704762868a048faec0e0fc
W20260811 18:57:26.062831 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd454d73c8da4bf79e68487caa3dd843 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.063457 5113 raft_consensus.cc:482] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.063632 5113 raft_consensus.cc:504] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.063786 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffad2d401c8f42f3b7a65afbb71b7c13" } modify_external_catalogs: true
I20260811 18:57:26.064392 5113 leader_election.cc:288] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.057204 3870 tablet_service.cc:1467] Processing CreateTablet for tablet fcec797e73bf4399aeb2bfd4b02949c0 (DEFAULT_TABLE table=default.Table_9c4130777df64341a269a7954794efd2 [id=c52e87ba38e24c0ea8bc1ed616ba9a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.068409 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21054bdd3d7c400c95b1f41e41bafc79" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.068671 3891 raft_consensus.cc:2454] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:26.069012 3824 leader_election.cc:334] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.736s (SENT)
I20260811 18:57:26.069218 3676 leader_election.cc:302] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.069201 3824 leader_election.cc:302] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:26.068681 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcec797e73bf4399aeb2bfd4b02949c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.069432 3824 raft_consensus.cc:2670] T e215759d48c24d8cb99bbc3fc9009a66 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:26.070055 3678 leader_election.cc:334] T d3d6daff4b7540309ff58755758e9752 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.151s (SENT)
I20260811 18:57:26.070814 5113 raft_consensus.cc:2781] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.070897 5113 raft_consensus.cc:482] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.070957 5113 raft_consensus.cc:3037] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.073770 5113 raft_consensus.cc:504] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.074059 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 99e9474ef8814514832f0e48c2b3495f (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.074460 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e9474ef8814514832f0e48c2b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.075655 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 (DEFAULT_TABLE table=default.Table_0da5d975ff884b9bb82d239828666af7 [id=c082db9ee1db46e2bc2b0ff9d94d8fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.075817 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21054bdd3d7c400c95b1f41e41bafc79" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:26.076004 3891 raft_consensus.cc:3037] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.076040 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de8ce45e514ff291cb9e5c0a4c00d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.078666 5113 leader_election.cc:288] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.082013 3891 raft_consensus.cc:2454] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:26.082558 3676 leader_election.cc:302] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.082973 5113 raft_consensus.cc:2781] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.083244 5113 raft_consensus.cc:686] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:26.083534 5113 consensus_queue.cc:237] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.085542 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ea4ad482bde44d6bab5ec809054aaebd
I20260811 18:57:26.086529 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 761902f7366d43ffbde226a0ade2e188 (DEFAULT_TABLE table=default.Table_0036c28f59cb4f229da243389602bbd9 [id=9b456a91de20444dbe8abd84177b3073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.086994 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761902f7366d43ffbde226a0ade2e188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.087432 5080 raft_consensus.cc:482] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.089002 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4ad482bde44d6bab5ec809054aaebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.089720 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 46505ba544104458888dbc2a37115dcc (DEFAULT_TABLE table=default.Table_1e4e304a679345cd9289252278ddb281 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.090054 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46505ba544104458888dbc2a37115dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.092334 3716 tablet_service.cc:1467] Processing CreateTablet for tablet fcec797e73bf4399aeb2bfd4b02949c0 (DEFAULT_TABLE table=default.Table_9c4130777df64341a269a7954794efd2 [id=c52e87ba38e24c0ea8bc1ed616ba9a2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.092792 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcec797e73bf4399aeb2bfd4b02949c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.073770 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 (DEFAULT_TABLE table=default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7 [id=c9676ca3a1a8471bb1dfc3dbf19d7861]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.096122 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8146f47d9ae48889ac7efc954d4a6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.097407 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a255a0922a92419191f18730d8b335bc (DEFAULT_TABLE table=default.Table_7f980c9b98994253b590548d86f6d145 [id=e4ed3f906b884fffbb40dfa89e10ddd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.097774 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a255a0922a92419191f18730d8b335bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.099014 4167 ts_tablet_manager.cc:1939] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.099606 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 471) took 5182 ms. Trace:
W20260811 18:57:26.103856 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:26.087568 5080 raft_consensus.cc:504] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.106555 4167 rpcz_store.cc:276] 0811 18:57:20.917227 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.918975 (+ 1748us) service_pool.cc:227] Handling call
0811 18:57:25.929408 (+5010433us) ts_tablet_manager.cc:1145] Deleting tablet f8eb041f3d704762868a048faec0e0fc
0811 18:57:25.929417 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.965332 (+ 35915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.059979 (+ 94647us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.099587 (+ 39608us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4991598}
I20260811 18:57:26.107110 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 472) took 5124 ms. Trace:
I20260811 18:57:26.107204 4167 rpcz_store.cc:276] 0811 18:57:20.983111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.983861 (+ 750us) service_pool.cc:227] Handling call
0811 18:57:26.106951 (+5123090us) ts_tablet_manager.cc:1145] Deleting tablet 3468b39bd73c4272a84dad37fcfd6f27
0811 18:57:26.106965 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.107078 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5104954}
I20260811 18:57:26.107576 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 473) took 5119 ms. Trace:
I20260811 18:57:26.107645 4167 rpcz_store.cc:276] 0811 18:57:20.988320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.988903 (+ 583us) service_pool.cc:227] Handling call
0811 18:57:26.107457 (+5118554us) ts_tablet_manager.cc:1145] Deleting tablet bf5749d4eebd42319df1b52eeba4c062
0811 18:57:26.107466 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.107560 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5104237}
I20260811 18:57:26.108084 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 475) took 5023 ms. Trace:
I20260811 18:57:26.108170 4167 rpcz_store.cc:276] 0811 18:57:21.084332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.084404 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:26.107935 (+5023531us) ts_tablet_manager.cc:1145] Deleting tablet fdeec0c5672b4b3190d7d7cf341cb118
0811 18:57:26.107952 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108068 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5022582}
I20260811 18:57:26.108279 5080 leader_election.cc:288] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.108547 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 474) took 5024 ms. Trace:
I20260811 18:57:26.108632 4167 rpcz_store.cc:276] 0811 18:57:21.084146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.084900 (+ 754us) service_pool.cc:227] Handling call
0811 18:57:26.108388 (+5023488us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:26.108397 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108529 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5022901}
I20260811 18:57:26.109015 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 476) took 4782 ms. Trace:
I20260811 18:57:26.109112 4167 rpcz_store.cc:276] 0811 18:57:21.326640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.326702 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:26.108866 (+4782164us) ts_tablet_manager.cc:1145] Deleting tablet 289e607f10e14a2983bc6ecc93ce9bbc
0811 18:57:26.108874 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.108998 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4781740}
I20260811 18:57:26.109576 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 477) took 4609 ms. Trace:
I20260811 18:57:26.109634 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e63ee9b9fd4b12ae791b9d5bc2e648" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.109650 4167 rpcz_store.cc:276] 0811 18:57:21.499808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.500275 (+ 467us) service_pool.cc:227] Handling call
0811 18:57:26.109323 (+4609048us) ts_tablet_manager.cc:1145] Deleting tablet d3d6daff4b7540309ff58755758e9752
0811 18:57:26.109348 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.109561 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4583519}
I20260811 18:57:26.109969 3744 raft_consensus.cc:2454] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.110041 4167 tablet_replica.cc:330] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.110253 4167 raft_consensus.cc:2229] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.105230 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3721f75a96bf418ea7bd86ec90126b52 (DEFAULT_TABLE table=default.Table_8d7695d0859045409244bdd77ddd3931 [id=36fa8d1e4ec84fe286b5b4153857999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.105640 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 40898ceb18cc4f2d9104818798049ae2 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.110610 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f8eb041f3d704762868a048faec0e0fc (table default.Table_45c439603b1140e998b5d9199026da49 [id=32914b6ff485486f843da54878e2d49b]) successfully deleted
I20260811 18:57:26.110772 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40898ceb18cc4f2d9104818798049ae2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.111356 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3468b39bd73c4272a84dad37fcfd6f27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3468b39bd73c4272a84dad37fcfd6f27 already in progress: creating tablet
W20260811 18:57:26.111661 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf5749d4eebd42319df1b52eeba4c062 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf5749d4eebd42319df1b52eeba4c062 already in progress: creating tablet
W20260811 18:57:26.111997 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fdeec0c5672b4b3190d7d7cf341cb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdeec0c5672b4b3190d7d7cf341cb118 already in progress: creating tablet
W20260811 18:57:26.112264 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 42405aec21e3405486b6c658438b5fb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42405aec21e3405486b6c658438b5fb2 already in progress: creating tablet
W20260811 18:57:26.112641 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 289e607f10e14a2983bc6ecc93ce9bbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 289e607f10e14a2983bc6ecc93ce9bbc already in progress: creating tablet
W20260811 18:57:26.113010 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d3d6daff4b7540309ff58755758e9752 because the tablet was not found. No further retry: Not found: Tablet not found: d3d6daff4b7540309ff58755758e9752
I20260811 18:57:26.110791 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4ad482bde44d6bab5ec809054aaebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.112319 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 46505ba544104458888dbc2a37115dcc (DEFAULT_TABLE table=default.Table_1e4e304a679345cd9289252278ddb281 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.115031 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46505ba544104458888dbc2a37115dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.116647 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 (DEFAULT_TABLE table=default.Table_0da5d975ff884b9bb82d239828666af7 [id=c082db9ee1db46e2bc2b0ff9d94d8fc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.117408 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de8ce45e514ff291cb9e5c0a4c00d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.118083 4162 tablet_bootstrap.cc:654] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.120316 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 40898ceb18cc4f2d9104818798049ae2 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.120795 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a255a0922a92419191f18730d8b335bc (DEFAULT_TABLE table=default.Table_7f980c9b98994253b590548d86f6d145 [id=e4ed3f906b884fffbb40dfa89e10ddd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.121196 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a255a0922a92419191f18730d8b335bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.127168 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 3721f75a96bf418ea7bd86ec90126b52 (DEFAULT_TABLE table=default.Table_8d7695d0859045409244bdd77ddd3931 [id=36fa8d1e4ec84fe286b5b4153857999f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.127708 3824 leader_election.cc:302] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.128027 5080 raft_consensus.cc:2781] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.128120 5080 raft_consensus.cc:482] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.128183 5080 raft_consensus.cc:3037] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.130656 4169 ts_tablet_manager.cc:1918] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 18:57:26.134735 3678 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.151s (SENT)
I20260811 18:57:26.134733 4169 log.cc:1196] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/61c8443df21949ff9bdc63fa6aa34108
I20260811 18:57:26.136425 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40898ceb18cc4f2d9104818798049ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.137502 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3721f75a96bf418ea7bd86ec90126b52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.137848 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e407d34d3c984ecb8c8ffde304f23ce6 consensus state has no leader
W20260811 18:57:26.138543 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bef0f6d56cda40249f12b25a8eb3ab6b consensus state has no leader
W20260811 18:57:26.139528 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
W20260811 18:57:26.140347 3678 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.518s (SENT)
I20260811 18:57:26.142035 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" } modify_external_catalogs: true
I20260811 18:57:26.144870 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3721f75a96bf418ea7bd86ec90126b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.145148 3546 catalog_manager.cc:5495] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:26.145627 3678 leader_election.cc:334] T 6381a951a966473c80670b0dd1aff062 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.579s (SENT)
I20260811 18:57:26.145764 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 21054bdd3d7c400c95b1f41e41bafc79 (table default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4]). Expected version 4 got 0
I20260811 18:57:26.146998 3865 tablet_service.cc:1467] Processing CreateTablet for tablet c282501396154e3088b39691c60cd35b (DEFAULT_TABLE table=default.Table_d7e8b32170ff47eda203b84328064301 [id=f33a076a8b49435d89198c35da0b1831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.147079 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1427a17411684dc280e5e063f4221bf9" } modify_external_catalogs: true
I20260811 18:57:26.147485 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c282501396154e3088b39691c60cd35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.150985 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 99ecc0d308d540948de39dfe12199597 (DEFAULT_TABLE table=default.Table_2bd2a5750a454b88b5bff7b802541c89 [id=e659e4ff88894137ba1c37c450230df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.151515 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ecc0d308d540948de39dfe12199597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.155253 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet ff1be026ca404eada53f60f257ea367c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.155521 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet c2247194db704f9abd54a8a0df1ca566 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dce6143309948d49e229dc9890d904f) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.156030 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b (DEFAULT_TABLE table=default.Table_0322ec9fd9f24338bb9196b099f56643 [id=3dac4cc0f73442c9a173f0766e0e9874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.156453 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a5fdab68694761b7493b42a0bf7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.158929 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 (DEFAULT_TABLE table=default.Table_c6ed856dda814eb0bd8eb76c2f83c70b [id=ffd5f1ae63224181bb83a8c11717ad3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.160957 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22abc6f8e9ef4c32ab69af31bdaad872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.162423 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" } modify_external_catalogs: true
I20260811 18:57:26.166998 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7b7f47e131444447a4ac2d8377da6090" } modify_external_catalogs: true
I20260811 18:57:26.168004 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d" } modify_external_catalogs: true
I20260811 18:57:26.168365 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf764ce4b1c46b8a9d7d88f7ba19d8d" } modify_external_catalogs: true
I20260811 18:57:26.170032 4167 raft_consensus.cc:2258] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.170753 4167 ts_tablet_manager.cc:1905] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.171361 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 4da53cef719842f98d2190e487e8d653 (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.171833 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da53cef719842f98d2190e487e8d653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.176107 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 4da53cef719842f98d2190e487e8d653 (DEFAULT_TABLE table=default.Table_66121f6795f240ea88c01d7c0ae02e00 [id=d7dca3f4b6d84c1380f2c4cef8207a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.176734 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da53cef719842f98d2190e487e8d653. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.178980 3678 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.025s (SENT)
I20260811 18:57:26.180171 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 02bf791926d84522b1099c1242acddcd (DEFAULT_TABLE table=default.Table_bc171757977f4404bb736a03a87b9ecd [id=41452d01375a443ba5c239475692f69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.180780 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02bf791926d84522b1099c1242acddcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.183338 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1d81e93739f44d9e99b6ebc647e1d0e4
I20260811 18:57:26.183838 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 54fa32f8fa1144c4bd9dcd00a550001a
W20260811 18:57:26.184456 3678 leader_election.cc:334] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.372s (SENT)
I20260811 18:57:26.185343 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b16cb4c1796147fbb10109f52071505e
I20260811 18:57:26.188802 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.188932 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet b16cb4c1796147fbb10109f52071505e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.194115 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_256aa74bb2884a0d83f0d17b159b9e49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.194435 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 99ecc0d308d540948de39dfe12199597 (DEFAULT_TABLE table=default.Table_2bd2a5750a454b88b5bff7b802541c89 [id=e659e4ff88894137ba1c37c450230df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.194950 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ecc0d308d540948de39dfe12199597. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.195263 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_256aa74bb2884a0d83f0d17b159b9e49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.194177 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9d18547a0114b83bfbed5110e565661"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.196755 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9d18547a0114b83bfbed5110e565661 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.198361 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b (DEFAULT_TABLE table=default.Table_0322ec9fd9f24338bb9196b099f56643 [id=3dac4cc0f73442c9a173f0766e0e9874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.198845 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a5fdab68694761b7493b42a0bf7c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.206940 4169 ts_tablet_manager.cc:1939] T 61c8443df21949ff9bdc63fa6aa34108 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.207415 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 453) took 5947 ms. Trace:
I20260811 18:57:26.207545 4169 rpcz_store.cc:276] 0811 18:57:20.260026 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.260121 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:25.982489 (+5722368us) ts_tablet_manager.cc:1145] Deleting tablet 61c8443df21949ff9bdc63fa6aa34108
0811 18:57:25.982508 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:25.992576 (+ 10068us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.130514 (+137938us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.207388 (+ 76874us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5695530}
I20260811 18:57:26.208176 4169 tablet_replica.cc:330] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:26.208149 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 61c8443df21949ff9bdc63fa6aa34108 (table default.Table_c391f66ce98e48f3bf4a736969eca1af [id=8931a7530e1b483d864b7c8a64612f45]) successfully deleted
I20260811 18:57:26.208449 4169 raft_consensus.cc:2229] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.208715 4169 raft_consensus.cc:2258] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.209720 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 02bf791926d84522b1099c1242acddcd (DEFAULT_TABLE table=default.Table_bc171757977f4404bb736a03a87b9ecd [id=41452d01375a443ba5c239475692f69f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.210182 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02bf791926d84522b1099c1242acddcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.210537 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 54fa32f8fa1144c4bd9dcd00a550001a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.210949 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 (DEFAULT_TABLE table=default.Table_c6ed856dda814eb0bd8eb76c2f83c70b [id=ffd5f1ae63224181bb83a8c11717ad3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.211431 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22abc6f8e9ef4c32ab69af31bdaad872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.215492 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e80925854f7a430c93376f6448db3632"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.216524 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e80925854f7a430c93376f6448db3632 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.217891 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.219027 3716 tablet_service.cc:1467] Processing CreateTablet for tablet c282501396154e3088b39691c60cd35b (DEFAULT_TABLE table=default.Table_d7e8b32170ff47eda203b84328064301 [id=f33a076a8b49435d89198c35da0b1831]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.219722 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c282501396154e3088b39691c60cd35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.219975 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b5da605dec7141f8aa11be6246b92608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.220729 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5da605dec7141f8aa11be6246b92608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.221266 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb62e05c51bb4f10b1058d170a2202e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.221926 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb62e05c51bb4f10b1058d170a2202e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.226511 5080 raft_consensus.cc:504] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.227257 5080 leader_election.cc:288] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.228148 4169 ts_tablet_manager.cc:1905] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.229704 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e63ee9b9fd4b12ae791b9d5bc2e648" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:26.230060 3744 raft_consensus.cc:3037] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.232908 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b16cb4c1796147fbb10109f52071505e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.234460 3744 raft_consensus.cc:2454] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:26.236297 4163 tablet_bootstrap.cc:492] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:26.236459 4163 ts_tablet_manager.cc:1397] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.333s user 0.007s sys 0.007s
I20260811 18:57:26.240433 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.235340 3824 leader_election.cc:302] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.244968 4162 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.245276 4162 ts_tablet_manager.cc:1397] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.002s
I20260811 18:57:26.246727 4162 raft_consensus.cc:348] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.247138 4162 raft_consensus.cc:374] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.247249 4162 raft_consensus.cc:729] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.247546 4162 consensus_queue.cc:260] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.247987 4167 ts_tablet_manager.cc:1918] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.248127 4167 log.cc:1196] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:26.248600 4167 ts_tablet_manager.cc:1939] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.249148 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 478) took 4749 ms. Trace:
I20260811 18:57:26.249269 4167 rpcz_store.cc:276] 0811 18:57:21.500188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.500471 (+ 283us) service_pool.cc:227] Handling call
0811 18:57:26.109999 (+4609528us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:26.110009 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.203854 (+ 93845us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.247872 (+ 44018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.249125 (+ 1253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4583936}
I20260811 18:57:26.249794 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 480) took 4685 ms. Trace:
I20260811 18:57:26.249900 4167 rpcz_store.cc:276] 0811 18:57:21.564859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.564935 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:26.249606 (+4684671us) ts_tablet_manager.cc:1145] Deleting tablet d8aea8dc31914052832bdc83d638461f
0811 18:57:26.249621 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.249776 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4684217}
I20260811 18:57:26.250073 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 36724a83bf9844b9a27a88ca36a9bf50 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:26.250660 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d8aea8dc31914052832bdc83d638461f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8aea8dc31914052832bdc83d638461f already in progress: creating tablet
W20260811 18:57:26.251204 3678 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.089s (SENT)
I20260811 18:57:26.253587 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7c395f6c4c364b7eb463f78e16675847"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.254989 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c395f6c4c364b7eb463f78e16675847 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.255445 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.256213 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d42d349a68447e39dbdda4dea1dd35b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.256467 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_66121f6795f240ea88c01d7c0ae02e00" } new_table_name: "default.Table_949e95c6313942e5b97401b457b0c160" modify_external_catalogs: true
I20260811 18:57:26.259938 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1811e5f62a714f8388460dfcb1717e81
I20260811 18:57:26.261620 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1811e5f62a714f8388460dfcb1717e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.262630 5080 raft_consensus.cc:2781] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.262789 5080 raft_consensus.cc:686] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:26.263057 5080 consensus_queue.cc:237] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:26.270849 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 1811e5f62a714f8388460dfcb1717e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.280434 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 481) took 4715 ms. Trace:
I20260811 18:57:26.281366 4167 rpcz_store.cc:276] 0811 18:57:21.565084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.565441 (+ 357us) service_pool.cc:227] Handling call
0811 18:57:26.250236 (+4684795us) ts_tablet_manager.cc:1145] Deleting tablet bf2816cc2dfe45f0815ce50a5a929324
0811 18:57:26.250250 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.280398 (+ 30148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4684261}
W20260811 18:57:26.285741 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf2816cc2dfe45f0815ce50a5a929324 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf2816cc2dfe45f0815ce50a5a929324 already in progress: creating tablet
I20260811 18:57:26.286108 4162 ts_tablet_manager.cc:1428] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.041s user 0.000s sys 0.002s
I20260811 18:57:26.286610 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 482) took 4699 ms. Trace:
I20260811 18:57:26.286805 4167 rpcz_store.cc:276] 0811 18:57:21.587392 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.587482 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:26.282725 (+4695243us) ts_tablet_manager.cc:1145] Deleting tablet ffa8a4a1c0cb4834be501bdd7d468472
0811 18:57:26.286410 (+ 3685us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.286586 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4692253}
I20260811 18:57:26.288249 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 499) took 4237 ms. Trace:
I20260811 18:57:26.288453 4167 rpcz_store.cc:276] 0811 18:57:22.050959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.052357 (+ 1398us) service_pool.cc:227] Handling call
0811 18:57:26.287445 (+4235088us) ts_tablet_manager.cc:1145] Deleting tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
0811 18:57:26.287462 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.288224 (+ 762us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4195274}
W20260811 18:57:26.289140 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ffa8a4a1c0cb4834be501bdd7d468472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa8a4a1c0cb4834be501bdd7d468472 already in progress: creating tablet
I20260811 18:57:26.289398 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 500) took 4238 ms. Trace:
I20260811 18:57:26.289557 4167 rpcz_store.cc:276] 0811 18:57:22.051170 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.052282 (+ 1112us) service_pool.cc:227] Handling call
0811 18:57:26.289071 (+4236789us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:26.289086 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.289374 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4190803}
W20260811 18:57:26.289702 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e already in progress: creating tablet
I20260811 18:57:26.290432 5037 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.290603 5037 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.290807 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7b404e99e9414c67b3cd5aff12203a53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b404e99e9414c67b3cd5aff12203a53 already in progress: creating tablet
I20260811 18:57:26.291397 5037 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.291942 4163 raft_consensus.cc:348] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.292152 4163 raft_consensus.cc:374] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.292227 4163 raft_consensus.cc:729] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.292596 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.293064 3744 raft_consensus.cc:2454] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.293597 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6914252b8edc40e79af29a5145201b08" } modify_external_catalogs: true
I20260811 18:57:26.293696 3824 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.293895 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6914252b8edc40e79af29a5145201b08" } modify_external_catalogs: true
I20260811 18:57:26.294159 5037 raft_consensus.cc:2781] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.294241 5037 raft_consensus.cc:482] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.294329 5037 raft_consensus.cc:3037] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.290208 4167 tablet_replica.cc:330] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.295027 4167 raft_consensus.cc:2229] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.295323 4167 raft_consensus.cc:2258] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.297569 4167 ts_tablet_manager.cc:1905] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.292436 4163 consensus_queue.cc:260] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.304172 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7db24c0702d141f08ea7023227018d81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.304881 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7db24c0702d141f08ea7023227018d81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.307785 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e5ca40ed9be45ee85dd2681b4523778" } new_table_name: "default.Table_fcb655a52f4449839a4919d317bc6cfe" modify_external_catalogs: true
I20260811 18:57:26.300107 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.314455 4162 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.327785 4162 tablet_bootstrap.cc:654] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.338469 5113 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.338699 5113 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.339461 5113 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.344767 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.326717 5037 raft_consensus.cc:504] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.350900 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c1ceb39258a049f08a1f48c40df65d8b" } new_table_name: "default.Table_61b2dc398e804cdabeda17315814c849" modify_external_catalogs: true
I20260811 18:57:26.352226 4162 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.352402 4162 ts_tablet_manager.cc:1397] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.005s
I20260811 18:57:26.352735 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e3a4280bd164399a5346ffaa6f40b8f" } new_table_name: "default.Table_8e407f239f804e5cbd2c9ecbc2c170c5" modify_external_catalogs: true
I20260811 18:57:26.354249 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_07499669fc704a31830a06152cd615cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.354869 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07499669fc704a31830a06152cd615cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.356482 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_41c44c966ece466eb232aeab834c5fe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.357012 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41c44c966ece466eb232aeab834c5fe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.358795 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_88a784338b5e4928be15e4679b3f3304" } new_table_name: "default.Table_7309030d82da426ab87ff64346efd25b" modify_external_catalogs: true
W20260811 18:57:26.360303 3676 leader_election.cc:341] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:26.360249 4162 raft_consensus.cc:348] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.365687 4167 ts_tablet_manager.cc:1918] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.370029 4167 log.cc:1196] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:26.371028 4167 ts_tablet_manager.cc:1939] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.373119 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 502) took 4229 ms. Trace:
I20260811 18:57:26.377923 4167 rpcz_store.cc:276] 0811 18:57:22.143771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.148904 (+ 5133us) service_pool.cc:227] Handling call
0811 18:57:26.290151 (+4141247us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:26.290163 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.335909 (+ 45746us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.365536 (+ 29627us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.373094 (+ 7558us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4113185}
I20260811 18:57:26.379417 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 94943cfbe41a47b3a36c81957a3adaeb (table default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]) successfully deleted
I20260811 18:57:26.380623 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 506) took 4233 ms. Trace:
I20260811 18:57:26.381083 4167 rpcz_store.cc:276] 0811 18:57:22.146453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.149317 (+ 2864us) service_pool.cc:227] Handling call
0811 18:57:26.379498 (+4230181us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:26.379518 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.379764 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4201885}
W20260811 18:57:26.382604 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6432abd4fc5c4f9ca95971ca4795e1cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6432abd4fc5c4f9ca95971ca4795e1cf already in progress: creating tablet
I20260811 18:57:26.382905 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 507) took 4236 ms. Trace:
I20260811 18:57:26.383265 5037 leader_election.cc:288] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.384517 4167 rpcz_store.cc:276] 0811 18:57:22.146633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.149032 (+ 2399us) service_pool.cc:227] Handling call
0811 18:57:26.382683 (+4233651us) ts_tablet_manager.cc:1145] Deleting tablet 9ec4bba841dd446aa3db751358ce7f32
0811 18:57:26.382701 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.382879 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4200737}
W20260811 18:57:26.385951 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9ec4bba841dd446aa3db751358ce7f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ec4bba841dd446aa3db751358ce7f32 already in progress: creating tablet
I20260811 18:57:26.386157 4167 tablet_replica.cc:330] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.388224 4167 raft_consensus.cc:2229] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.389539 4167 raft_consensus.cc:2258] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.392019 4167 ts_tablet_manager.cc:1905] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:26.392354 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4da53cef719842f98d2190e487e8d653 consensus state has no leader
I20260811 18:57:26.393170 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69803efc857941408d32f7da625acfb6" } modify_external_catalogs: true
I20260811 18:57:26.393602 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69803efc857941408d32f7da625acfb6" } modify_external_catalogs: true
I20260811 18:57:26.395334 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" } modify_external_catalogs: true
I20260811 18:57:26.395859 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19c0f99ce46a480e9c8a60b4c2837eca" } modify_external_catalogs: true
W20260811 18:57:26.397795 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4059e5907b1e488ab9e1d04e563f96cd consensus state has no leader
I20260811 18:57:26.398934 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_709cd55123504b35877d469f96354d28" } modify_external_catalogs: true
I20260811 18:57:26.399261 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_709cd55123504b35877d469f96354d28" } modify_external_catalogs: true
I20260811 18:57:26.401479 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 335fc15ea75d483990e6ae5a5872682f
I20260811 18:57:26.405035 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fc15ea75d483990e6ae5a5872682f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.410542 5113 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.410754 5113 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.411655 5113 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.412736 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.413908 3676 leader_election.cc:341] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:26.416000 4167 ts_tablet_manager.cc:1918] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:26.428194 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85140a3800f34a23b716703fd4e73d17 consensus state has no leader
I20260811 18:57:26.419934 5113 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.428850 4167 log.cc:1196] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e51a08e6dbf04e3fa58689ee0cd45db3
W20260811 18:57:26.428872 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9af9fb059984c9680e78b68120b3726 has no consensus state
I20260811 18:57:26.428884 5113 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.429467 4167 ts_tablet_manager.cc:1939] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.429769 5113 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.434713 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 505) took 4288 ms. Trace:
I20260811 18:57:26.433909 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.434991 3891 raft_consensus.cc:2454] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:26.435751 3676 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.437335 5113 raft_consensus.cc:2781] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.437441 5113 raft_consensus.cc:482] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.437521 5113 raft_consensus.cc:3037] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.439190 4167 rpcz_store.cc:276] 0811 18:57:22.146278 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.155537 (+ 9259us) service_pool.cc:227] Handling call
0811 18:57:26.386083 (+4230546us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:26.386100 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.412228 (+ 26128us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.415879 (+ 3651us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.434675 (+ 18796us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4202584}
I20260811 18:57:26.440658 4167 tablet_replica.cc:330] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.440934 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e51a08e6dbf04e3fa58689ee0cd45db3 (table default.Table_acb61e38f2a0442688865698892cc5c9 [id=3c7daa88956747719fb78e9860e7039f]) successfully deleted
I20260811 18:57:26.441514 4167 raft_consensus.cc:2229] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.442142 4167 raft_consensus.cc:2258] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.442044 5113 raft_consensus.cc:504] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.444883 5113 leader_election.cc:288] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.445005 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d3822fc2d4b569f223238a6464e0a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:26.445283 3891 raft_consensus.cc:3037] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.445746 4167 ts_tablet_manager.cc:1905] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.455417 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 705605cd929940f8a252b6e95abbeb95
I20260811 18:57:26.456539 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 705605cd929940f8a252b6e95abbeb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:26.456740 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c3ac1bc93404622a9b6a74aea8b7c1c has no consensus state
I20260811 18:57:26.469676 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" } modify_external_catalogs: true
I20260811 18:57:26.470062 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e52a767b81cd4dd186dacdeb1d4120cb" } modify_external_catalogs: true
I20260811 18:57:26.471695 5113 consensus_queue.cc:1035] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.473007 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
I20260811 18:57:26.474332 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.484606 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5eb58df7cc2042f69300d10a7999719b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.485841 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5eb58df7cc2042f69300d10a7999719b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.489389 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17690ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:26.489215 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f1e69da8480425682a8348cad2ffa51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.489935 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f1e69da8480425682a8348cad2ffa51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#27 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
I20260811 18:57:26.506853 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f6cd18a5e464333bbdc36290a29eaea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.508033 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f6cd18a5e464333bbdc36290a29eaea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.509892 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9df3de8d1ff649e8a41bff8b3de9d92f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.510593 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9df3de8d1ff649e8a41bff8b3de9d92f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.513814 4162 raft_consensus.cc:374] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.513944 4162 raft_consensus.cc:729] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.514268 4162 consensus_queue.cc:260] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.516850 4162 ts_tablet_manager.cc:1428] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.164s user 0.023s sys 0.000s
I20260811 18:57:26.517251 4162 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.518878 4162 tablet_bootstrap.cc:654] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.520721 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.523128 4167 ts_tablet_manager.cc:1918] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.523415 4167 log.cc:1196] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/828993fb34bb4c62bce2a9322d7c992a
I20260811 18:57:26.523973 4167 ts_tablet_manager.cc:1939] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:26.524390 3678 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.244s (SENT)
I20260811 18:57:26.525079 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 503) took 4379 ms. Trace:
I20260811 18:57:26.525264 4167 rpcz_store.cc:276] 0811 18:57:22.145857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.155446 (+ 9589us) service_pool.cc:227] Handling call
0811 18:57:26.440604 (+4285158us) ts_tablet_manager.cc:1145] Deleting tablet 828993fb34bb4c62bce2a9322d7c992a
0811 18:57:26.440617 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.467831 (+ 27214us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.523003 (+ 55172us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.525057 (+ 2054us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4256681}
I20260811 18:57:26.526090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 501) took 4388 ms. Trace:
I20260811 18:57:26.526257 4167 rpcz_store.cc:276] 0811 18:57:22.137769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.148492 (+ 10723us) service_pool.cc:227] Handling call
0811 18:57:26.525957 (+4377465us) ts_tablet_manager.cc:1145] Deleting tablet 7991dabe263b44f6b7b7d1d43b96ee44
0811 18:57:26.525969 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.526073 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4339300}
I20260811 18:57:26.526683 4167 tablet_replica.cc:330] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.526985 4167 raft_consensus.cc:2229] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.527402 4167 raft_consensus.cc:2258] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.527645 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 828993fb34bb4c62bce2a9322d7c992a (table default.Table_d22535c5f02a4044aad07546138fca10 [id=0ee8f3888c8b4aa9a70425c399460f3d]) successfully deleted
W20260811 18:57:26.528101 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7991dabe263b44f6b7b7d1d43b96ee44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7991dabe263b44f6b7b7d1d43b96ee44 already in progress: creating tablet
#28 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
I20260811 18:57:26.538668 3891 raft_consensus.cc:2454] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:26.538980 4163 ts_tablet_manager.cc:1428] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.302s user 0.003s sys 0.012s
I20260811 18:57:26.542469 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_035060c2fcfd476196e06c23978f156f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.545209 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b50aff06b9214a069e8b74b23089e9bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.539453 3676 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:26.551908 3678 leader_election.cc:334] T a732c700ff79432aaf1f0d1c1e90aabe P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.456s (SENT)
I20260811 18:57:26.553133 4163 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:26.553191 5091 raft_consensus.cc:2781] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.553332 5091 raft_consensus.cc:686] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:26.553565 5091 consensus_queue.cc:237] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.553999 4167 ts_tablet_manager.cc:1905] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:26.576776 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b50aff06b9214a069e8b74b23089e9bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.577690 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_035060c2fcfd476196e06c23978f156f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:26.591111 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 242915us
I20260811 18:57:26.593034 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff1be026ca404eada53f60f257ea367c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:26.593281 3744 raft_consensus.cc:3037] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.597357 3744 raft_consensus.cc:2454] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:26.601552 4162 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.601728 4162 ts_tablet_manager.cc:1397] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.000s
I20260811 18:57:26.601831 4163 tablet_bootstrap.cc:654] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.599820 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 335fc15ea75d483990e6ae5a5872682f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.602551 4162 raft_consensus.cc:348] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.602802 4162 raft_consensus.cc:374] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.602867 4162 raft_consensus.cc:729] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.603081 4162 consensus_queue.cc:260] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.603827 4162 ts_tablet_manager.cc:1428] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:26.604130 4162 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.605494 4162 tablet_bootstrap.cc:654] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#29 0x7fc7af50f61e in clone (/lib/x86_64-linux-gnu/libc.so.6+0x12161e)
0x6190000bf988 is located 8 bytes inside of 1048-byte region [0x6190000bf980,0x6190000bfd98)
freed by thread T137 (tablet-delete [) here:
I20260811 18:57:26.613170 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a795d5395cc64155b84057a53c701064" } new_table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" modify_external_catalogs: true
I20260811 18:57:26.620214 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1204e0b30877462982c245f439f0bae4" } new_table_name: "default.Table_e425950244124732b380187898a2b68f" modify_external_catalogs: true
#0 0x8c4680 in operator delete(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:160:3
I20260811 18:57:26.641196 4162 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.641372 4162 ts_tablet_manager.cc:1397] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.001s
#1 0x7fc7bfa20d2c in std::_Sp_counted_ptr<kudu::tablet::Tablet*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() ../../../include/c++/8/bits/shared_ptr_base.h:377:9
I20260811 18:57:26.642488 4162 raft_consensus.cc:348] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.642930 4162 raft_consensus.cc:374] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.643013 4162 raft_consensus.cc:729] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.643146 4167 ts_tablet_manager.cc:1918] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:26.643335 4167 log.cc:1196] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:26.643287 4162 consensus_queue.cc:260] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.643896 4167 ts_tablet_manager.cc:1939] T 0ace00158b744bd8a97dc1c47fc6049c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.644846 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 508) took 4378 ms. Trace:
I20260811 18:57:26.644991 4167 rpcz_store.cc:276] 0811 18:57:22.266101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.267499 (+ 1398us) service_pool.cc:227] Handling call
0811 18:57:26.526620 (+4259121us) ts_tablet_manager.cc:1145] Deleting tablet 0ace00158b744bd8a97dc1c47fc6049c
0811 18:57:26.526645 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.590419 (+ 63774us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.643028 (+ 52609us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.644819 (+ 1791us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4222084}
I20260811 18:57:26.643901 4162 ts_tablet_manager.cc:1428] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:26.646808 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0f271e4958af4bfbb65af29a053ac2d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.649590 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_558f1f69ec7547c3b029b4672f04c6e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.650563 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_558f1f69ec7547c3b029b4672f04c6e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.652010 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_102e5367d375402c9800357142894fa0 [id=a067d27ecf9c402db2631f2df5f0fea4]) successfully deleted
I20260811 18:57:26.656530 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_540b89ef88904bb6b163c4b18480746e" } new_table_name: "default.Table_661f1ebd9f4c4cb9be591c5e1c203a86" modify_external_catalogs: true
W20260811 18:57:26.657572 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f271e4958af4bfbb65af29a053ac2d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.664049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 514) took 4324 ms. Trace:
I20260811 18:57:26.664165 4167 rpcz_store.cc:276] 0811 18:57:22.339980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.340052 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:26.645488 (+4305436us) ts_tablet_manager.cc:1145] Deleting tablet 2605007cc2454599868f44c717deb2d2
0811 18:57:26.663844 (+ 18356us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.664018 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4304931}
I20260811 18:57:26.666200 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_365efecf4f8e4f4993ba5579614a3049" } modify_external_catalogs: true
I20260811 18:57:26.666594 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_365efecf4f8e4f4993ba5579614a3049" } modify_external_catalogs: true
W20260811 18:57:26.667969 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2605007cc2454599868f44c717deb2d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2605007cc2454599868f44c717deb2d2 already in progress: creating tablet
W20260811 18:57:26.668823 3678 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.472s (SENT)
I20260811 18:57:26.672760 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 516) took 4331 ms. Trace:
I20260811 18:57:26.672923 4167 rpcz_store.cc:276] 0811 18:57:22.341791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.341838 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:26.664619 (+4322781us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:26.672573 (+ 7954us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.672734 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4322392}
W20260811 18:57:26.673599 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3263da75f0d84528b7c8cc55dcaeaa46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3263da75f0d84528b7c8cc55dcaeaa46 already in progress: creating tablet
I20260811 18:57:26.677662 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 517) took 4317 ms. Trace:
I20260811 18:57:26.677857 4167 rpcz_store.cc:276] 0811 18:57:22.360023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.360094 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:26.673435 (+4313341us) ts_tablet_manager.cc:1145] Deleting tablet e14f8717bcd94d6d88295fd8be130ca5
0811 18:57:26.677502 (+ 4067us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.677640 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4313044}
I20260811 18:57:26.678563 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 521) took 4251 ms. Trace:
I20260811 18:57:26.678632 4162 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.678727 4167 rpcz_store.cc:276] 0811 18:57:22.426942 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.427011 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:26.678455 (+4251444us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:26.678465 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.678545 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4250551}
W20260811 18:57:26.680333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a847581412494e34b2c76e605ff11772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a847581412494e34b2c76e605ff11772 already in progress: creating tablet
I20260811 18:57:26.680465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 522) took 4252 ms. Trace:
I20260811 18:57:26.680546 4167 rpcz_store.cc:276] 0811 18:57:22.428289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.428347 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:26.680351 (+4252004us) ts_tablet_manager.cc:1145] Deleting tablet c6e9da8dfb4a4c6c9275c092fae7d910
0811 18:57:26.680363 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.680450 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4251628}
I20260811 18:57:26.681011 4167 tablet_replica.cc:330] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.681052 4162 tablet_bootstrap.cc:654] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.681212 4167 raft_consensus.cc:2229] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:26.681274 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c6e9da8dfb4a4c6c9275c092fae7d910 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6e9da8dfb4a4c6c9275c092fae7d910 already in progress: creating tablet
I20260811 18:57:26.681500 4167 raft_consensus.cc:2258] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.682171 4167 ts_tablet_manager.cc:1905] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.684201 5113 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.684404 5113 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.685559 5113 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:26.686478 5113 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.686669 5113 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.688719 5113 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:26.697396 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50af4e9b20a74ed983b26281c3c6eda1 consensus state has no leader
I20260811 18:57:26.698294 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b9115c298dfa4ecb970e76c7de2cd8e9" } new_table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" modify_external_catalogs: true
W20260811 18:57:26.700102 3678 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.746s (SENT)
W20260811 18:57:26.700119 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6d988ef7e9dd43158d107998ea4d82b9 consensus state has no leader
I20260811 18:57:26.700949 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4f7f1249bb248359c032a4e20f7c9b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.703142 4162 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.703315 4162 ts_tablet_manager.cc:1397] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.005s
I20260811 18:57:26.704293 4162 raft_consensus.cc:348] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.704573 4162 raft_consensus.cc:374] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.704628 4162 raft_consensus.cc:729] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.705623 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9693b748bd0490cb416e0cb36a62966" } modify_external_catalogs: true
I20260811 18:57:26.706027 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9693b748bd0490cb416e0cb36a62966" } modify_external_catalogs: true
I20260811 18:57:26.707093 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:26.707391 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:26.702199 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4f7f1249bb248359c032a4e20f7c9b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.707391 4162 consensus_queue.cc:260] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.708518 4162 ts_tablet_manager.cc:1428] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260811 18:57:26.717991 4162 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.719295 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.720160 4162 tablet_bootstrap.cc:654] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.720217 3824 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.720789 5133 raft_consensus.cc:2781] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:26.720966 5133 raft_consensus.cc:686] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:26.721496 5133 consensus_queue.cc:237] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:26.727643 3678 leader_election.cc:334] T 987b9ac6859444e3a86c56af786cc84e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
I20260811 18:57:26.728232 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 705605cd929940f8a252b6e95abbeb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:26.729420 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.729898 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.731333 3676 leader_election.cc:341] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.731593 3676 leader_election.cc:341] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.748888 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
I20260811 18:57:26.752213 4167 ts_tablet_manager.cc:1918] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.752380 4167 log.cc:1196] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:26.752607 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fb62e05c51bb4f10b1058d170a2202e6" } new_table_name: "default.Table_0790be2c84c14fb3aeb2add67667646e" modify_external_catalogs: true
I20260811 18:57:26.752840 4167 ts_tablet_manager.cc:1939] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:26.757105 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2962915ef36940f8b6dee3a420d74c59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.758023 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2962915ef36940f8b6dee3a420d74c59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.759935 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e695533c42647128860db601dcc0b5f" } modify_external_catalogs: true
I20260811 18:57:26.760254 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e695533c42647128860db601dcc0b5f" } modify_external_catalogs: true
W20260811 18:57:26.760957 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6009c9f6bbb0493ebe125120a1a47f3f consensus state has no leader
I20260811 18:57:26.761456 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_69f270a4181547b2a2cc604a814c82c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:26.762213 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69f270a4181547b2a2cc604a814c82c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:26.777087 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 524) took 4334 ms. Trace:
I20260811 18:57:26.777261 4167 rpcz_store.cc:276] 0811 18:57:22.442551 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.454522 (+ 11971us) service_pool.cc:227] Handling call
0811 18:57:26.680963 (+4226441us) ts_tablet_manager.cc:1145] Deleting tablet 494939a4eb5a45d6a123d174ad513bcb
0811 18:57:26.680974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.722214 (+ 41240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.752053 (+ 29839us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.777047 (+ 24994us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4225294}
I20260811 18:57:26.784061 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdd7e4b15384ec8844f2e31a5496d8a (DEFAULT_TABLE table=default.Table_85a94fea83f84948a14090e2deeac956 [id=c844a598a3c1437cbcdafa534b77ede7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.784583 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdd7e4b15384ec8844f2e31a5496d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.784811 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 494939a4eb5a45d6a123d174ad513bcb (table default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]) successfully deleted
I20260811 18:57:26.784898 3715 tablet_service.cc:1467] Processing CreateTablet for tablet c4ffd3f7c4274466bb5bde604ecf720a (DEFAULT_TABLE table=default.Table_bcb9560450d14ad392e0db4330f60317 [id=24c96cb40e7541b2a65a2b94553f009b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.785286 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ffd3f7c4274466bb5bde604ecf720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.786670 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 09366eccf38a4876840cc5ddddbbb103 (DEFAULT_TABLE table=default.Table_d7e8ba858b0c44d588499e1463673957 [id=307c31b5346949adb9b0c0db5da22ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.787081 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09366eccf38a4876840cc5ddddbbb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.788575 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.788932 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.789505 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:26.790771 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 (DEFAULT_TABLE table=default.Table_e89523faca1046669de08be2f6b963dd [id=10d38692f62648acb9248d2668bdc173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.791206 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c50dcffac54c7bac210eeaed340ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.792320 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 538) took 4224 ms. Trace:
I20260811 18:57:26.792423 4167 rpcz_store.cc:276] 0811 18:57:22.567767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.569699 (+ 1932us) service_pool.cc:227] Handling call
0811 18:57:26.792174 (+4222475us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:26.792186 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.792296 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164385}
I20260811 18:57:26.792907 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f24c8db84f04484584ad1ba5d97f96d4 (DEFAULT_TABLE table=default.Table_a905052284004a77adf260546090e666 [id=dd14415dd4a448178cb65d59d9deabc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.793244 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f24c8db84f04484584ad1ba5d97f96d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.794644 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b9af9fb059984c9680e78b68120b3726 (DEFAULT_TABLE table=default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.795013 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9af9fb059984c9680e78b68120b3726. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.796340 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 452ee450778e4d33adfa7e2745b5d970 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 452ee450778e4d33adfa7e2745b5d970 already in progress: creating tablet
I20260811 18:57:26.797175 4167 tablet_replica.cc:330] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:26.797398 4167 raft_consensus.cc:2229] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:26.798632 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f2255ff17edd4b81b18423473cc52218 (DEFAULT_TABLE table=default.Table_1cd8ec7574294bc6886f42d65935c0a5 [id=5b789b53abce4da480695c1f1c0562c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.799150 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2255ff17edd4b81b18423473cc52218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.802552 4167 raft_consensus.cc:2258] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.802778 3722 tablet_service.cc:1467] Processing CreateTablet for tablet fd2baeed833d4a62adc2b89e71050c79 (DEFAULT_TABLE table=default.Table_86e4bb4e07694c3a96ff484bd7944fd3 [id=6ae4c8c6cd59421ab00af37326d7e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.803195 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2baeed833d4a62adc2b89e71050c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.803267 4167 ts_tablet_manager.cc:1905] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.806926 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 00645f522efa4076832758b48fe566f3 (DEFAULT_TABLE table=default.Table_b08e57f933b24cd0aee53441350e6775 [id=c920e237cc6c4dfeac73235ddcff872a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.807353 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00645f522efa4076832758b48fe566f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.807575 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 69059a0667234a408cc1c09585df63bc (DEFAULT_TABLE table=default.Table_614252f40e34465080bb490a1a7050ed [id=63acd1bb983b453e910b79f7479cf2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.807871 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69059a0667234a408cc1c09585df63bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.810621 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f (DEFAULT_TABLE table=default.Table_91d4fb4116244d9e8c08ce623bea3a2e [id=e638fe436b5941f5a0633046786e2da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.810725 3706 tablet_service.cc:1467] Processing CreateTablet for tablet a135bd92ab5b4f76bad9b3eedcfada29 (DEFAULT_TABLE table=default.Table_75b5605dc2a14c9daf6d68e7094bcb39 [id=df10702558ae46c3b03adaa4d1af4896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.784487 3724 tablet_service.cc:1467] Processing CreateTablet for tablet a49b8b344177487fb5bd096ee9cdea2f (DEFAULT_TABLE table=default.Table_bd454d73c8da4bf79e68487caa3dd843 [id=654fb49529b84dcab699903b1b482115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.811097 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a135bd92ab5b4f76bad9b3eedcfada29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.811249 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49b8b344177487fb5bd096ee9cdea2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.814723 3710 tablet_service.cc:1467] Processing CreateTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 (DEFAULT_TABLE table=default.Table_e1af5a92fcee4bc4898572e55da333b5 [id=a0bf7b47be7e440789b03a33e06b8b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.815097 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c944d5cfd9b3405b8151abfc9f2fd998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.815762 3865 tablet_service.cc:1467] Processing CreateTablet for tablet a49b8b344177487fb5bd096ee9cdea2f (DEFAULT_TABLE table=default.Table_bd454d73c8da4bf79e68487caa3dd843 [id=654fb49529b84dcab699903b1b482115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.816315 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a49b8b344177487fb5bd096ee9cdea2f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.818732 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6009c9f6bbb0493ebe125120a1a47f3f (table default.Table_6cc21ee738d3430ead80d93044f49334 [id=f1e29c96194e4ca0bc9b14c0f8811a98], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6009c9f6bbb0493ebe125120a1a47f3f consensus state has no leader
I20260811 18:57:26.819007 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 5d8cf245bdc9410ab460f455bc7f7943 (DEFAULT_TABLE table=default.Table_cb3c2f76034640b398a94e7536595277 [id=118bf5f196cb4e37bfa10e1cf1608d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.819355 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d8cf245bdc9410ab460f455bc7f7943. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:26.822451 3678 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.658s (SENT)
I20260811 18:57:26.823436 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b38d3b9570467a9128bd7fe73acf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.827379 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0cdd7e4b15384ec8844f2e31a5496d8a (DEFAULT_TABLE table=default.Table_85a94fea83f84948a14090e2deeac956 [id=c844a598a3c1437cbcdafa534b77ede7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.828016 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cdd7e4b15384ec8844f2e31a5496d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.831393 3863 tablet_service.cc:1467] Processing CreateTablet for tablet c4ffd3f7c4274466bb5bde604ecf720a (DEFAULT_TABLE table=default.Table_bcb9560450d14ad392e0db4330f60317 [id=24c96cb40e7541b2a65a2b94553f009b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.831892 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ffd3f7c4274466bb5bde604ecf720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.836920 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 (DEFAULT_TABLE table=default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.837468 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd4ec30acde4585a1c3451927217cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.840221 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.840703 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.842761 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f24c8db84f04484584ad1ba5d97f96d4 (DEFAULT_TABLE table=default.Table_a905052284004a77adf260546090e666 [id=dd14415dd4a448178cb65d59d9deabc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.843148 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f24c8db84f04484584ad1ba5d97f96d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.845225 5113 raft_consensus.cc:482] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.845367 5113 raft_consensus.cc:504] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:26.846112 5113 leader_election.cc:288] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:26.846727 3864 tablet_service.cc:1467] Processing CreateTablet for tablet b9af9fb059984c9680e78b68120b3726 (DEFAULT_TABLE table=default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.847244 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9af9fb059984c9680e78b68120b3726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.847286 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08436f33db043b1bc6395cd92102e59" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:26.847930 3676 leader_election.cc:341] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:26.848366 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 105595us
I20260811 18:57:26.850804 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f2255ff17edd4b81b18423473cc52218 (DEFAULT_TABLE table=default.Table_1cd8ec7574294bc6886f42d65935c0a5 [id=5b789b53abce4da480695c1f1c0562c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.851320 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2255ff17edd4b81b18423473cc52218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.853627 4163 tablet_bootstrap.cc:492] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:26.853816 4163 ts_tablet_manager.cc:1397] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.301s user 0.001s sys 0.005s
I20260811 18:57:26.855120 4163 raft_consensus.cc:348] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.855399 4163 raft_consensus.cc:374] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.855553 4163 raft_consensus.cc:729] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.855862 4163 consensus_queue.cc:260] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.863601 3888 raft_consensus.cc:1261] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Refusing update from remote peer bd8c4bdbd811417ab011a7131f17d283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:26.864413 5140 consensus_queue.cc:1035] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.868844 4163 ts_tablet_manager.cc:1428] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.000s sys 0.003s
I20260811 18:57:26.869308 4163 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:26.871563 4163 tablet_bootstrap.cc:654] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.871830 4169 ts_tablet_manager.cc:1918] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:26.871973 4169 log.cc:1196] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6381a951a966473c80670b0dd1aff062
I20260811 18:57:26.872426 4169 ts_tablet_manager.cc:1939] T 6381a951a966473c80670b0dd1aff062 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.873077 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 09366eccf38a4876840cc5ddddbbb103 (DEFAULT_TABLE table=default.Table_d7e8ba858b0c44d588499e1463673957 [id=307c31b5346949adb9b0c0db5da22ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.873631 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09366eccf38a4876840cc5ddddbbb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.875691 5080 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.875810 5080 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.876657 5080 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.885032 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 (DEFAULT_TABLE table=default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48 [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.885553 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd4ec30acde4585a1c3451927217cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.890614 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f (DEFAULT_TABLE table=default.Table_91d4fb4116244d9e8c08ce623bea3a2e [id=e638fe436b5941f5a0633046786e2da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.891170 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b38d3b9570467a9128bd7fe73acf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.898882 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 00645f522efa4076832758b48fe566f3 (DEFAULT_TABLE table=default.Table_b08e57f933b24cd0aee53441350e6775 [id=c920e237cc6c4dfeac73235ddcff872a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.899422 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00645f522efa4076832758b48fe566f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.899526 4162 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:26.899683 4162 ts_tablet_manager.cc:1397] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.182s user 0.005s sys 0.001s
I20260811 18:57:26.900435 4162 raft_consensus.cc:348] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.900640 4162 raft_consensus.cc:374] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:26.900697 4162 raft_consensus.cc:729] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:26.901047 4162 consensus_queue.cc:260] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.903030 3864 tablet_service.cc:1467] Processing CreateTablet for tablet fd2baeed833d4a62adc2b89e71050c79 (DEFAULT_TABLE table=default.Table_86e4bb4e07694c3a96ff484bd7944fd3 [id=6ae4c8c6cd59421ab00af37326d7e181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.903558 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2baeed833d4a62adc2b89e71050c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.904505 4162 ts_tablet_manager.cc:1428] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:26.904899 4162 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:26.910420 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 (DEFAULT_TABLE table=default.Table_e89523faca1046669de08be2f6b963dd [id=10d38692f62648acb9248d2668bdc173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.911217 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c50dcffac54c7bac210eeaed340ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.911217 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 454) took 6404 ms. Trace:
I20260811 18:57:26.911461 4169 rpcz_store.cc:276] 0811 18:57:20.507042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.507116 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:26.208113 (+5700997us) ts_tablet_manager.cc:1145] Deleting tablet 6381a951a966473c80670b0dd1aff062
0811 18:57:26.208130 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.646569 (+438439us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.871715 (+225146us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.911197 (+ 39482us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5700613}
I20260811 18:57:26.912091 4169 tablet_replica.cc:330] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:26.912343 4169 raft_consensus.cc:2229] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:26.912533 4169 raft_consensus.cc:2258] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:26.913123 4169 ts_tablet_manager.cc:1905] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:26.920068 5152 tablet.cc:1722] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:26.920595 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a135bd92ab5b4f76bad9b3eedcfada29 (DEFAULT_TABLE table=default.Table_75b5605dc2a14c9daf6d68e7094bcb39 [id=df10702558ae46c3b03adaa4d1af4896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.921324 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a135bd92ab5b4f76bad9b3eedcfada29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.928146 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 5d8cf245bdc9410ab460f455bc7f7943 (DEFAULT_TABLE table=default.Table_cb3c2f76034640b398a94e7536595277 [id=118bf5f196cb4e37bfa10e1cf1608d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.930784 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:26.931048 3744 raft_consensus.cc:2454] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:26.933046 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6381a951a966473c80670b0dd1aff062 (table default.Table_4da51fec1a044237ab13da18366dde64 [id=263eb849fd614011a9dd641aeb9126d0]) successfully deleted
I20260811 18:57:26.933918 3856 tablet_service.cc:1467] Processing CreateTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 (DEFAULT_TABLE table=default.Table_e1af5a92fcee4bc4898572e55da333b5 [id=a0bf7b47be7e440789b03a33e06b8b5f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:26.935763 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT) [suppressed 57 similar messages]
W20260811 18:57:26.954841 3678 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.756s (SENT)
I20260811 18:57:26.960258 4162 tablet_bootstrap.cc:654] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:26.966490 5091 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:26.966706 5091 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.967590 5091 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
#2 0x8e30e6 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() ../../../include/c++/8/bits/shared_ptr_base.h:155:6
W20260811 18:57:26.974805 3824 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.272s (SENT)
I20260811 18:57:26.974977 3824 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:26.975286 5133 raft_consensus.cc:2726] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:26.975831 3824 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:26.975905 5133 consensus_queue.cc:1035] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:26.976492 5133 raft_consensus.cc:2781] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:26.976426 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 69059a0667234a408cc1c09585df63bc (DEFAULT_TABLE table=default.Table_614252f40e34465080bb490a1a7050ed [id=63acd1bb983b453e910b79f7479cf2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.976593 5133 raft_consensus.cc:482] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:26.976682 5133 raft_consensus.cc:3037] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:26.980188 4169 ts_tablet_manager.cc:1918] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:26.980994 5133 raft_consensus.cc:504] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:26.982237 5133 leader_election.cc:288] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:26.982544 4169 log.cc:1196] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/36724a83bf9844b9a27a88ca36a9bf50
I20260811 18:57:26.987035 4169 ts_tablet_manager.cc:1939] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:26.987249 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:26.990272 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:26.989645 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d8cf245bdc9410ab460f455bc7f7943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.994751 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c944d5cfd9b3405b8151abfc9f2fd998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.995030 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da (DEFAULT_TABLE table=default.Table_8a4328549b8d4ad08d0135327263c811 [id=7668053328524584b72ecd40ed55ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:26.995522 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9608d6ddcad749cd9e72dd6f265db0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:26.998736 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69059a0667234a408cc1c09585df63bc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.000442 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_120c3bf5084e4fd1a967d4564e474f6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.001859 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c2053e3676584227a3ea13c4f912e315"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.003607 4162 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.003826 4162 ts_tablet_manager.cc:1397] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.099s user 0.002s sys 0.002s
I20260811 18:57:27.004252 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba6246e6d29418cbca5f751b34f4ef1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.004457 3743 raft_consensus.cc:3037] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.004776 4162 raft_consensus.cc:348] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.004993 4162 raft_consensus.cc:374] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.005074 4162 raft_consensus.cc:729] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.005312 4162 consensus_queue.cc:260] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.005893 4162 ts_tablet_manager.cc:1428] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:27.006202 4162 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.008111 3743 raft_consensus.cc:2454] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.011809 3824 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.012329 5133 raft_consensus.cc:2781] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.012667 5133 raft_consensus.cc:686] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.013010 5133 consensus_queue.cc:237] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.014002 3676 leader_election.cc:341] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.016059 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 459) took 6423 ms. Trace:
#3 0x8e2fc1 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() ../../../include/c++/8/bits/shared_ptr_base.h:728:11
I20260811 18:57:27.017678 4169 rpcz_store.cc:276] 0811 18:57:20.593000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.593064 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:26.912040 (+6318976us) ts_tablet_manager.cc:1145] Deleting tablet 36724a83bf9844b9a27a88ca36a9bf50
0811 18:57:26.912052 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.942202 (+ 30150us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:26.980061 (+ 37859us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.016035 (+ 35974us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6317618}
W20260811 18:57:27.021130 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2053e3676584227a3ea13c4f912e315 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.018632 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da (DEFAULT_TABLE table=default.Table_8a4328549b8d4ad08d0135327263c811 [id=7668053328524584b72ecd40ed55ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.023488 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9608d6ddcad749cd9e72dd6f265db0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.023794 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b6c70b4163634984bd02dd9b47ffb472 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.024326 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c70b4163634984bd02dd9b47ffb472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.026428 4162 tablet_bootstrap.cc:654] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.032001 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 469) took 6321 ms. Trace:
I20260811 18:57:27.032783 4169 rpcz_store.cc:276] 0811 18:57:20.710381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.717598 (+ 7217us) service_pool.cc:227] Handling call
0811 18:57:27.031461 (+6313863us) ts_tablet_manager.cc:1145] Deleting tablet 91818d0cbe144a67a458a6648968bf5b
0811 18:57:27.031784 (+ 323us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.031964 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6299852}
I20260811 18:57:27.036720 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 (DEFAULT_TABLE table=default.Table_3652770b3e134affa80ff22fdfc1a0a1 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.037312 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27f9dbb52f34687a33c2ebf20e8f780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.034255 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 468) took 6324 ms. Trace:
I20260811 18:57:27.040589 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 36724a83bf9844b9a27a88ca36a9bf50 (table default.Table_c038c8ee6e694c6f99d5cb717ba90ca9 [id=7a048f5d108d462ba534f3398356d38d]) successfully deleted
W20260811 18:57:27.041687 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 91818d0cbe144a67a458a6648968bf5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91818d0cbe144a67a458a6648968bf5b already in progress: creating tablet
I20260811 18:57:27.041945 5091 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.042199 5091 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.042853 5091 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.035193 5153 tablet.cc:1722] T 21054bdd3d7c400c95b1f41e41bafc79 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:27.044554 5133 consensus_queue.cc:1035] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.045842 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.046069 3678 leader_election.cc:334] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.950s (SENT)
I20260811 18:57:27.046198 3678 leader_election.cc:302] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:27.046396 3678 raft_consensus.cc:2670] T a300a7aca2814da6b999eb4d865fea66 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:27.046613 3676 leader_election.cc:341] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.047138 3865 tablet_service.cc:1467] Processing CreateTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 (DEFAULT_TABLE table=default.Table_3652770b3e134affa80ff22fdfc1a0a1 [id=ae35cbd76db748e2bd87803adac842bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.047616 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27f9dbb52f34687a33c2ebf20e8f780. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.048398 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15600ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:27.060597 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 37be4f7579fa4ab99ddbacfa80282b42
I20260811 18:57:27.061815 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.065145 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0aee6beb2e70421083841520733087f8
I20260811 18:57:27.066094 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9722572533de436cb9149e6230570382
W20260811 18:57:27.066143 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4f21761b37b42dbb814bd54cca89452 has no consensus state
I20260811 18:57:27.066504 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7251204325dd424192a6f58e18e71c34
I20260811 18:57:27.068069 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 28af2d2c61254b0ca399dddb05e9ee70
I20260811 18:57:27.068181 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.068414 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 9722572533de436cb9149e6230570382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.068292 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cc00e047acd44bf8ef4e6a3bc1b4d69" } new_table_name: "default.Table_de8a2d61a3cc411a97ad8e8df11c7d89" modify_external_catalogs: true
I20260811 18:57:27.071062 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_706c784347b146089c032d699f6a9abe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.071468 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.071852 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.070199 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_035060c2fcfd476196e06c23978f156f" } new_table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" modify_external_catalogs: true
W20260811 18:57:27.072567 3824 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.412s (SENT)
W20260811 18:57:27.072613 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_706c784347b146089c032d699f6a9abe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.072697 3824 leader_election.cc:302] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.073504 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 37be4f7579fa4ab99ddbacfa80282b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.073644 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.073961 4162 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.074046 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.074103 4162 ts_tablet_manager.cc:1397] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.002s
I20260811 18:57:27.074610 5133 raft_consensus.cc:2726] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.061641 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_863d848a099c49edae9fa42c7b7db8ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.075229 4162 raft_consensus.cc:348] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.075507 4162 raft_consensus.cc:374] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.075568 4162 raft_consensus.cc:729] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:27.075578 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_863d848a099c49edae9fa42c7b7db8ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.075805 4162 consensus_queue.cc:260] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.076531 4162 ts_tablet_manager.cc:1428] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:57:27.076844 4162 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.079164 3856 tablet_service.cc:1467] Processing CreateTablet for tablet b6c70b4163634984bd02dd9b47ffb472 (DEFAULT_TABLE table=default.Table_971d1dda6c9c4e65a732e83f6528e2de [id=5083f806f9a54ca4ac7c1d77b1fd8000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.079731 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c70b4163634984bd02dd9b47ffb472. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.080583 3678 leader_election.cc:334] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.136s (SENT)
I20260811 18:57:27.080796 3678 leader_election.cc:302] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:27.080982 3678 raft_consensus.cc:2670] T 5ae200f3814e431a8288c30fe94bffcf P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.080912 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_068dc550d67d413c916d1384c3cb811c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.081610 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_068dc550d67d413c916d1384c3cb811c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.038028 4169 rpcz_store.cc:276] 0811 18:57:20.710084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.710744 (+ 660us) service_pool.cc:227] Handling call
0811 18:57:27.034030 (+6323286us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:27.034044 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.034228 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6299001}
I20260811 18:57:27.090950 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 470) took 6380 ms. Trace:
I20260811 18:57:27.091176 4169 rpcz_store.cc:276] 0811 18:57:20.710641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.717651 (+ 7010us) service_pool.cc:227] Handling call
0811 18:57:27.090780 (+6373129us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:27.090798 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.090926 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6354950}
I20260811 18:57:27.091490 4169 tablet_replica.cc:330] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:27.091719 4169 raft_consensus.cc:2229] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:27.091918 4169 raft_consensus.cc:2258] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.092648 4169 ts_tablet_manager.cc:1905] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.094589 4163 tablet_bootstrap.cc:492] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.094753 4163 ts_tablet_manager.cc:1397] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.226s user 0.004s sys 0.000s
I20260811 18:57:27.095631 4163 raft_consensus.cc:348] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.095968 4163 raft_consensus.cc:374] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.096045 4163 raft_consensus.cc:729] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.096606 4163 consensus_queue.cc:260] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.097600 5091 raft_consensus.cc:482] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.097779 5091 raft_consensus.cc:504] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.097702 4163 ts_tablet_manager.cc:1428] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:27.098542 5091 leader_election.cc:288] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.100220 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 28af2d2c61254b0ca399dddb05e9ee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.100862 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 9722572533de436cb9149e6230570382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.103632 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.104163 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9621bbd0f94223a5d36bc01487f615. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.104121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 43ad6993b63e4bbd98c8cc52f1a53451 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43ad6993b63e4bbd98c8cc52f1a53451 already in progress: creating tablet
W20260811 18:57:27.104086 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 44107.8us
W20260811 18:57:27.104530 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b6f36ab6bb774257b5c2692c64eff63d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6f36ab6bb774257b5c2692c64eff63d already in progress: creating tablet
I20260811 18:57:27.109311 4167 ts_tablet_manager.cc:1918] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.109548 4167 log.cc:1196] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/125757b8c1a6448b841c63339f392339
I20260811 18:57:27.114672 4167 ts_tablet_manager.cc:1939] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.114806 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 90576c068ce4468a9c572307dcb79b10 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.116416 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbf6ce167fb348be8d61372791f0cbca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.117192 3676 leader_election.cc:341] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.119216 4163 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.121418 4163 tablet_bootstrap.cc:654] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.122555 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3cc27811ad29451daaebbf31fb3ad198" } new_table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" modify_external_catalogs: true
W20260811 18:57:27.128909 3678 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.058s (SENT)
I20260811 18:57:27.130849 4162 tablet_bootstrap.cc:654] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.132226 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 3c142d8ef40947e29f3c94e1e8e4c517 (DEFAULT_TABLE table=default.Table_b012268c924347d8a87ea52350010dc3 [id=de9e6027bb194adebeba8a4b3253ff9b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.132730 3824 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.977s (SENT)
I20260811 18:57:27.132858 3824 leader_election.cc:302] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:27.133042 3824 raft_consensus.cc:2670] T e7259cf84f5c4c89a1b4510f743c2bc5 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:27.133659 3824 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.104s (SENT)
I20260811 18:57:27.133762 3824 leader_election.cc:302] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.134828 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c142d8ef40947e29f3c94e1e8e4c517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.134760 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3c142d8ef40947e29f3c94e1e8e4c517 (DEFAULT_TABLE table=default.Table_b012268c924347d8a87ea52350010dc3 [id=de9e6027bb194adebeba8a4b3253ff9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.135195 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c142d8ef40947e29f3c94e1e8e4c517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.137662 5080 raft_consensus.cc:2726] T 9ad616256b9949ceaf8b300f660d00c5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.144893 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 539) took 4576 ms. Trace:
I20260811 18:57:27.145567 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.148464 3719 tablet_service.cc:1467] Processing CreateTablet for tablet d6c81850cdc74b4ab727bd2279162477 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.147527 4167 rpcz_store.cc:276] 0811 18:57:22.567943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.569754 (+ 1811us) service_pool.cc:227] Handling call
0811 18:57:26.797127 (+4227373us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:26.797139 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:26.971057 (+173918us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.109192 (+138135us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.144866 (+ 35674us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4169147}
I20260811 18:57:27.148913 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c81850cdc74b4ab727bd2279162477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.153705 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9621bbd0f94223a5d36bc01487f615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.155922 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bb1cb45730b14b998a8f547983b83823 (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.156378 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1cb45730b14b998a8f547983b83823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.166441 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c (DEFAULT_TABLE table=default.Table_7309030d82da426ab87ff64346efd25b [id=6e7f157b26ec44fba47381cbcaff1f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.167044 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.148437 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 98ad733726594d2a8087f857f155dbe2 (DEFAULT_TABLE table=default.Table_be655558b83041fcace57520a599b8a0 [id=411f1ba1a39c4f10b9aaea7b10799e0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.169090 3856 tablet_service.cc:1467] Processing CreateTablet for tablet bb1cb45730b14b998a8f547983b83823 (DEFAULT_TABLE table=default.Table_af28562bfbf94af98e28afef24fda608 [id=794884a7937e42e6b98e1db67d99551a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.170117 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1cb45730b14b998a8f547983b83823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.169457 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ad733726594d2a8087f857f155dbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.173800 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 98ad733726594d2a8087f857f155dbe2 (DEFAULT_TABLE table=default.Table_be655558b83041fcace57520a599b8a0 [id=411f1ba1a39c4f10b9aaea7b10799e0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.175377 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c (DEFAULT_TABLE table=default.Table_7309030d82da426ab87ff64346efd25b [id=6e7f157b26ec44fba47381cbcaff1f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.175817 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.176421 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 540) took 4576 ms. Trace:
I20260811 18:57:27.177809 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ad733726594d2a8087f857f155dbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.182492 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3453632ca8274e459b096036508fbb8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.182530 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5d93073a69c143ff9460248848a108d4 has no consensus state
W20260811 18:57:27.184194 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3453632ca8274e459b096036508fbb8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.184851 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:27.185807 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.186546 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a35f3caa50e849abac2fc4d5711ea9fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.188371 3824 consensus_peers.cc:487] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94943cfbe41a47b3a36c81957a3adaeb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:27.192803 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7fc31b7c8d824f879807f105af51df48" } new_table_name: "default.Table_fb8dd203bf3941fbacfd7dad1418e566" modify_external_catalogs: true
I20260811 18:57:27.192848 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.195218 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bea4820fb5f4e65acb2a1a36332ac2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.197731 4162 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.197901 4162 ts_tablet_manager.cc:1397] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.121s user 0.003s sys 0.004s
I20260811 18:57:27.199178 4162 raft_consensus.cc:348] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.199437 4162 raft_consensus.cc:374] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.199527 4162 raft_consensus.cc:729] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.199858 4162 consensus_queue.cc:260] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.200672 4162 ts_tablet_manager.cc:1428] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:27.176538 4167 rpcz_store.cc:276] 0811 18:57:22.599852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.613269 (+ 13417us) service_pool.cc:227] Handling call
0811 18:57:27.176267 (+4562998us) ts_tablet_manager.cc:1145] Deleting tablet b20872d80c9545e5822bfc82d496ce97
0811 18:57:27.176280 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.176400 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4536703}
W20260811 18:57:27.205539 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b20872d80c9545e5822bfc82d496ce97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b20872d80c9545e5822bfc82d496ce97 already in progress: creating tablet
W20260811 18:57:27.206488 3824 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.822s (SENT)
I20260811 18:57:27.206656 3824 leader_election.cc:302] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.207158 5080 raft_consensus.cc:2726] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.218881 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 90576c068ce4468a9c572307dcb79b10
W20260811 18:57:27.219532 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfea0cd619024204b9bff1e2dd62b7b0 consensus state has no leader
I20260811 18:57:27.219730 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 90576c068ce4468a9c572307dcb79b10 deleted and replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331
W20260811 18:57:27.222589 3678 leader_election.cc:334] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.710s (SENT)
I20260811 18:57:27.226778 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 90576c068ce4468a9c572307dcb79b10 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.227815 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet b16cb4c1796147fbb10109f52071505e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.232724 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 90576c068ce4468a9c572307dcb79b10 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e44c67a0eb7485fb8dcde09416cd331) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.177947 5133 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.233183 5133 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.234059 5133 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.177938 3864 tablet_service.cc:1467] Processing CreateTablet for tablet d6c81850cdc74b4ab727bd2279162477 (DEFAULT_TABLE table=default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.234901 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c81850cdc74b4ab727bd2279162477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.235172 4163 tablet_bootstrap.cc:492] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.235352 4163 ts_tablet_manager.cc:1397] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.116s user 0.002s sys 0.001s
I20260811 18:57:27.236455 4163 raft_consensus.cc:348] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.236718 4163 raft_consensus.cc:374] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.236784 4163 raft_consensus.cc:729] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.240697 4162 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.242202 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.242571 3744 raft_consensus.cc:2454] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.243520 3824 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.243961 5133 raft_consensus.cc:2781] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.244055 5133 raft_consensus.cc:482] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.244168 5133 raft_consensus.cc:3037] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.246183 4162 tablet_bootstrap.cc:654] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.246537 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 545) took 4019 ms. Trace:
I20260811 18:57:27.246634 4167 rpcz_store.cc:276] 0811 18:57:23.227176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.227241 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:27.205980 (+3978739us) ts_tablet_manager.cc:1145] Deleting tablet 21cc1822d4b94081ad86a6177acb3afe
0811 18:57:27.246383 (+ 40403us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.246516 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3952564}
W20260811 18:57:27.247275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 21cc1822d4b94081ad86a6177acb3afe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21cc1822d4b94081ad86a6177acb3afe already in progress: creating tablet
I20260811 18:57:27.251029 5133 raft_consensus.cc:504] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.251963 5133 leader_election.cc:288] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:27.253041 3678 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.332s (SENT)
I20260811 18:57:27.280051 4163 consensus_queue.cc:260] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.281265 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef393c341bb54baf87176db5b5254ec7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.281569 3744 raft_consensus.cc:3037] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.285290 3744 raft_consensus.cc:2454] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.286269 4163 ts_tablet_manager.cc:1428] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.051s user 0.002s sys 0.002s
I20260811 18:57:27.287370 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 546) took 4039 ms. Trace:
I20260811 18:57:27.287967 4167 rpcz_store.cc:276] 0811 18:57:23.248237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.248325 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:27.247672 (+3999347us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:27.281126 (+ 33454us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.287346 (+ 6220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3993288}
I20260811 18:57:27.288729 4167 tablet_replica.cc:330] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:27.288821 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4b6ec419ed7747c3b24dc11992518e88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b6ec419ed7747c3b24dc11992518e88 already in progress: creating tablet
I20260811 18:57:27.289067 4167 raft_consensus.cc:2229] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 LEADER]: Raft consensus shutting down.
I20260811 18:57:27.289479 4167 raft_consensus.cc:2258] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.290518 4167 ts_tablet_manager.cc:1905] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.287140 3824 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.291846 5133 raft_consensus.cc:2781] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.292058 5133 raft_consensus.cc:686] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.292330 5133 consensus_queue.cc:237] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.301827 4163 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.329123 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_863d848a099c49edae9fa42c7b7db8ec" } new_table_name: "default.Table_0df46efea56e41b79fbb3e4dd034f84e" modify_external_catalogs: true
I20260811 18:57:27.331370 4162 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.331573 4162 ts_tablet_manager.cc:1397] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
W20260811 18:57:27.333405 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a811d4488d0e4541808a5e1aa44d663b consensus state has no leader
I20260811 18:57:27.333621 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f64f05d1ee44029680c5423fd0d81b" } modify_external_catalogs: true
I20260811 18:57:27.333940 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_82ea06e136674461909baae7b6d4e8cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.334944 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82ea06e136674461909baae7b6d4e8cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.334244 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1bf717c45ad424daa97f656cabd9db8" } modify_external_catalogs: true
I20260811 18:57:27.335729 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1bf717c45ad424daa97f656cabd9db8" } modify_external_catalogs: true
I20260811 18:57:27.336709 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f64f05d1ee44029680c5423fd0d81b" } modify_external_catalogs: true
I20260811 18:57:27.337568 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ea791cf775eb426ba289b61dfbaf8b0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.337960 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8" } modify_external_catalogs: true
I20260811 18:57:27.338327 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed948e1e9ee14cffae82bcd5517709f8" } modify_external_catalogs: true
W20260811 18:57:27.338737 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea791cf775eb426ba289b61dfbaf8b0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.339053 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd454d73c8da4bf79e68487caa3dd843" } modify_external_catalogs: true
W20260811 18:57:27.339351 3824 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.114s (SENT)
I20260811 18:57:27.339345 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd454d73c8da4bf79e68487caa3dd843" } modify_external_catalogs: true
I20260811 18:57:27.339514 3824 leader_election.cc:302] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:27.339735 3824 raft_consensus.cc:2670] T 0190cbd859274fcfa09c6fab899c4843 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.341593 5133 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.341811 5133 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.342902 5091 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.343091 5091 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.343955 5091 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.345072 4167 ts_tablet_manager.cc:1918] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.5
I20260811 18:57:27.345352 4167 log.cc:1196] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:27.345228 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:27.346323 4167 ts_tablet_manager.cc:1939] T 9de9acdb33a2438eb4378a5eac844c35 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.349501 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" } modify_external_catalogs: true
I20260811 18:57:27.349859 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d88f576cc3ca4c6eadc6e071aacc15b2" } modify_external_catalogs: true
I20260811 18:57:27.288821 4169 ts_tablet_manager.cc:1918] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.350476 4169 log.cc:1196] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a732c700ff79432aaf1f0d1c1e90aabe
I20260811 18:57:27.351034 4169 ts_tablet_manager.cc:1939] T a732c700ff79432aaf1f0d1c1e90aabe P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:27.352057 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 471) took 6616 ms. Trace:
I20260811 18:57:27.359103 4169 rpcz_store.cc:276] 0811 18:57:20.736065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.736434 (+ 369us) service_pool.cc:227] Handling call
0811 18:57:27.091443 (+6355009us) ts_tablet_manager.cc:1145] Deleting tablet a732c700ff79432aaf1f0d1c1e90aabe
0811 18:57:27.091454 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.232484 (+141030us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.288707 (+ 56223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.352035 (+ 63328us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6354638}
I20260811 18:57:27.353370 5133 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.352907 4163 tablet_bootstrap.cc:654] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.361496 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 472) took 6625 ms. Trace:
I20260811 18:57:27.361493 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a732c700ff79432aaf1f0d1c1e90aabe (table default.Table_8e4cf372a70649839b2083d6740f8c99 [id=a9cc173b7d5140d58181bb4941a4d434]) successfully deleted
I20260811 18:57:27.363900 4169 rpcz_store.cc:276] 0811 18:57:20.736358 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.736861 (+ 503us) service_pool.cc:227] Handling call
0811 18:57:27.361341 (+6624480us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:27.361355 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.361474 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6623988}
W20260811 18:57:27.346760 3676 leader_election.cc:341] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.353053 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.358752 4162 raft_consensus.cc:348] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.364318 4162 raft_consensus.cc:374] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.364337 3744 raft_consensus.cc:2454] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.364442 4162 raft_consensus.cc:729] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.364775 4162 consensus_queue.cc:260] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.367753 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 547) took 4005 ms. Trace:
I20260811 18:57:27.367908 4167 rpcz_store.cc:276] 0811 18:57:23.362368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.362503 (+ 135us) service_pool.cc:227] Handling call
0811 18:57:27.288669 (+3926166us) ts_tablet_manager.cc:1145] Deleting tablet 9de9acdb33a2438eb4378a5eac844c35
0811 18:57:27.288685 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.311115 (+ 22430us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.342524 (+ 31409us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.367729 (+ 25205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3911552}
I20260811 18:57:27.368649 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 560) took 3643 ms. Trace:
I20260811 18:57:27.368813 4167 rpcz_store.cc:276] 0811 18:57:23.725273 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725327 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:27.368519 (+3643192us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:27.368531 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.368631 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3640473}
I20260811 18:57:27.369323 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]) successfully deleted
I20260811 18:57:27.369415 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 559) took 3644 ms. Trace:
I20260811 18:57:27.369480 4167 rpcz_store.cc:276] 0811 18:57:23.725115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725510 (+ 395us) service_pool.cc:227] Handling call
0811 18:57:27.369289 (+3643779us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:27.369314 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.369398 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3641196}
W20260811 18:57:27.369719 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5d495354c0994fb4a74c376078e41a26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d495354c0994fb4a74c376078e41a26 already in progress: creating tablet
I20260811 18:57:27.369931 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 558) took 3645 ms. Trace:
W20260811 18:57:27.370011 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe203387bc9c45ba82afe6b058aa5e5f already in progress: creating tablet
I20260811 18:57:27.370015 4167 rpcz_store.cc:276] 0811 18:57:23.724895 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.726121 (+ 1226us) service_pool.cc:227] Handling call
0811 18:57:27.369835 (+3643714us) ts_tablet_manager.cc:1145] Deleting tablet d593aea288e14bb1a4c54360b260cffa
0811 18:57:27.369844 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.369915 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3633256}
I20260811 18:57:27.370513 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 561) took 3635 ms. Trace:
I20260811 18:57:27.370654 4167 rpcz_store.cc:276] 0811 18:57:23.734880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.734942 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:27.370411 (+3635469us) ts_tablet_manager.cc:1145] Deleting tablet 1f60a286ac024c45bcb845227c5445c9
0811 18:57:27.370422 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.370498 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3630080}
I20260811 18:57:27.371163 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 562) took 3624 ms. Trace:
W20260811 18:57:27.371359 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d593aea288e14bb1a4c54360b260cffa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d593aea288e14bb1a4c54360b260cffa already in progress: creating tablet
I20260811 18:57:27.371333 4167 rpcz_store.cc:276] 0811 18:57:23.747449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.747729 (+ 280us) service_pool.cc:227] Handling call
0811 18:57:27.371061 (+3623332us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:27.371073 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.371145 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3616788}
W20260811 18:57:27.371685 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1f60a286ac024c45bcb845227c5445c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f60a286ac024c45bcb845227c5445c9 already in progress: creating tablet
I20260811 18:57:27.371809 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 557) took 3647 ms. Trace:
I20260811 18:57:27.371882 4167 rpcz_store.cc:276] 0811 18:57:23.724741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.725922 (+ 1181us) service_pool.cc:227] Handling call
0811 18:57:27.371711 (+3645789us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:27.371722 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.371794 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3614059}
W20260811 18:57:27.371978 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 122b7898fea44ef78f269b1505d8ddc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 122b7898fea44ef78f269b1505d8ddc4 already in progress: creating tablet
W20260811 18:57:27.372400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b3f0cdcee681460084662e01152ad9dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3f0cdcee681460084662e01152ad9dd already in progress: creating tablet
I20260811 18:57:27.372680 4162 ts_tablet_manager.cc:1428] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.041s user 0.001s sys 0.001s
I20260811 18:57:27.364857 3824 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.374969 5133 raft_consensus.cc:2781] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.375121 5133 raft_consensus.cc:482] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.375276 5133 raft_consensus.cc:3037] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.379112 5133 raft_consensus.cc:504] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.380900 5133 leader_election.cc:288] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.381639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 476) took 6390 ms. Trace:
W20260811 18:57:27.382087 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 059b325df1db4fde9d4b26b559121316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 059b325df1db4fde9d4b26b559121316 already in progress: creating tablet
I20260811 18:57:27.380669 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6f36ab6bb774257b5c2692c64eff63d" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
W20260811 18:57:27.382951 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13788f40714847ff8ce6a810ac53e398 has no consensus state
I20260811 18:57:27.383076 3744 raft_consensus.cc:3037] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.383757 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1a2751cf08eb4ce0b354486cf5678c1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.384568 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a2751cf08eb4ce0b354486cf5678c1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.386356 3744 raft_consensus.cc:2454] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.382169 4167 tablet_replica.cc:330] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:27.387329 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet f287040b879642448367abe2b1c73cd9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.387460 4167 raft_consensus.cc:2229] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:27.393335 4162 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.386943 3824 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.396147 4169 rpcz_store.cc:276] 0811 18:57:20.990897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.993112 (+ 2215us) service_pool.cc:227] Handling call
0811 18:57:27.381502 (+6388390us) ts_tablet_manager.cc:1145] Deleting tablet bf5749d4eebd42319df1b52eeba4c062
0811 18:57:27.381516 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.381620 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6337970}
I20260811 18:57:27.396518 5133 raft_consensus.cc:2781] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:27.397317 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bf5749d4eebd42319df1b52eeba4c062 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf5749d4eebd42319df1b52eeba4c062 already in progress: creating tablet
I20260811 18:57:27.397709 5133 raft_consensus.cc:686] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.398013 5133 consensus_queue.cc:237] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.404011 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 475) took 6422 ms. Trace:
I20260811 18:57:27.404363 4167 raft_consensus.cc:2258] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.405507 4167 ts_tablet_manager.cc:1905] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.405797 4169 rpcz_store.cc:276] 0811 18:57:20.981891 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:20.984828 (+ 2937us) service_pool.cc:227] Handling call
0811 18:57:27.403863 (+6419035us) ts_tablet_manager.cc:1145] Deleting tablet 3468b39bd73c4272a84dad37fcfd6f27
0811 18:57:27.403876 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.403983 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6358806}
W20260811 18:57:27.410946 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3468b39bd73c4272a84dad37fcfd6f27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3468b39bd73c4272a84dad37fcfd6f27 already in progress: creating tablet
I20260811 18:57:27.395829 4163 tablet_bootstrap.cc:492] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.411491 4163 ts_tablet_manager.cc:1397] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.110s user 0.005s sys 0.000s
I20260811 18:57:27.413087 4163 raft_consensus.cc:348] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.409641 5091 consensus_queue.cc:1035] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:27.431411 3678 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.028s (SENT)
I20260811 18:57:27.432832 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f2f1e0bdeb7a45e3b9e6041aa2ac2578" } new_table_name: "default.Table_710d4f427cd6437a908b1109b4a140a1" modify_external_catalogs: true
I20260811 18:57:27.459793 4162 tablet_bootstrap.cc:654] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:27.466661 3678 leader_election.cc:334] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.919s (SENT)
I20260811 18:57:27.466831 3678 leader_election.cc:302] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:27.466735 4163 raft_consensus.cc:374] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:27.467003 3678 raft_consensus.cc:2670] T 33979de5b48c4674a46c8da3b7151753 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:27.467099 4163 raft_consensus.cc:729] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.467427 4163 consensus_queue.cc:260] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.411939 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 478) took 6315 ms. Trace:
W20260811 18:57:27.440452 3824 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.044s (SENT)
I20260811 18:57:27.473762 3824 leader_election.cc:302] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.474330 4169 rpcz_store.cc:276] 0811 18:57:21.096333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.099622 (+ 3289us) service_pool.cc:227] Handling call
0811 18:57:27.411797 (+6312175us) ts_tablet_manager.cc:1145] Deleting tablet fdeec0c5672b4b3190d7d7cf341cb118
0811 18:57:27.411811 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.411920 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6288008}
I20260811 18:57:27.475227 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 477) took 6380 ms. Trace:
I20260811 18:57:27.475459 4169 rpcz_store.cc:276] 0811 18:57:21.094693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.100189 (+ 5496us) service_pool.cc:227] Handling call
0811 18:57:27.475085 (+6374896us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:27.475099 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.475209 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6341728}
I20260811 18:57:27.476116 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 479) took 6152 ms. Trace:
I20260811 18:57:27.476294 4169 rpcz_store.cc:276] 0811 18:57:21.323309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.341308 (+ 17999us) service_pool.cc:227] Handling call
0811 18:57:27.475984 (+6134676us) ts_tablet_manager.cc:1145] Deleting tablet 289e607f10e14a2983bc6ecc93ce9bbc
0811 18:57:27.475996 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.476101 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6133907}
I20260811 18:57:27.477494 5133 raft_consensus.cc:2726] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.479900 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 481) took 5920 ms. Trace:
I20260811 18:57:27.480049 4169 rpcz_store.cc:276] 0811 18:57:21.559061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.559158 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:27.477212 (+5918054us) ts_tablet_manager.cc:1145] Deleting tablet bf2816cc2dfe45f0815ce50a5a929324
0811 18:57:27.477223 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.479880 (+ 2657us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5917583}
I20260811 18:57:27.480603 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 480) took 5921 ms. Trace:
I20260811 18:57:27.480733 4169 rpcz_store.cc:276] 0811 18:57:21.558766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.561626 (+ 2860us) service_pool.cc:227] Handling call
0811 18:57:27.480508 (+5918882us) ts_tablet_manager.cc:1145] Deleting tablet d8aea8dc31914052832bdc83d638461f
0811 18:57:27.480518 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.480590 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5918263}
I20260811 18:57:27.481249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 482) took 5844 ms. Trace:
I20260811 18:57:27.482044 4169 rpcz_store.cc:276] 0811 18:57:21.636943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:21.637036 (+ 93us) service_pool.cc:227] Handling call
0811 18:57:27.481087 (+5844051us) ts_tablet_manager.cc:1145] Deleting tablet ffa8a4a1c0cb4834be501bdd7d468472
0811 18:57:27.481165 (+ 78us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.481233 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5826083}
I20260811 18:57:27.481779 4163 ts_tablet_manager.cc:1428] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.070s user 0.002s sys 0.000s
W20260811 18:57:27.487529 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
I20260811 18:57:27.488324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_47c0e5602f6b4e1793b88727c8bfdaee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.489042 4163 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:27.489658 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47c0e5602f6b4e1793b88727c8bfdaee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.491775 4163 tablet_bootstrap.cc:654] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.493721 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 499) took 5447 ms. Trace:
I20260811 18:57:27.493813 4169 rpcz_store.cc:276] 0811 18:57:22.046379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.046480 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:27.493542 (+5447062us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:27.493552 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.493684 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5430510}
I20260811 18:57:27.489086 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8ad9f329a9c449e4ade20157c2601ee8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.495448 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ad9f329a9c449e4ade20157c2601ee8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.496174 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fdeec0c5672b4b3190d7d7cf341cb118 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdeec0c5672b4b3190d7d7cf341cb118 already in progress: creating tablet
W20260811 18:57:27.496631 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 42405aec21e3405486b6c658438b5fb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42405aec21e3405486b6c658438b5fb2 already in progress: creating tablet
W20260811 18:57:27.496976 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 289e607f10e14a2983bc6ecc93ce9bbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 289e607f10e14a2983bc6ecc93ce9bbc already in progress: creating tablet
W20260811 18:57:27.497303 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bf2816cc2dfe45f0815ce50a5a929324 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf2816cc2dfe45f0815ce50a5a929324 already in progress: creating tablet
W20260811 18:57:27.497661 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d8aea8dc31914052832bdc83d638461f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8aea8dc31914052832bdc83d638461f already in progress: creating tablet
W20260811 18:57:27.498029 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ffa8a4a1c0cb4834be501bdd7d468472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffa8a4a1c0cb4834be501bdd7d468472 already in progress: creating tablet
W20260811 18:57:27.499025 3678 leader_election.cc:334] T 0190cbd859274fcfa09c6fab899c4843 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.960s (SENT)
I20260811 18:57:27.499531 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 419824ebd14e415f90072b8c335f1ad1 (DEFAULT_TABLE table=default.Table_a4f7f1249bb248359c032a4e20f7c9b4 [id=7087e984854645998d78661c89e4503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.499450 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 18c599d945044e0088bc66c8472d2495 (DEFAULT_TABLE table=default.Table_120c3bf5084e4fd1a967d4564e474f6c [id=c6b3d432c45c440f80a484b8220cbef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.500123 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 419824ebd14e415f90072b8c335f1ad1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.502425 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7b404e99e9414c67b3cd5aff12203a53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b404e99e9414c67b3cd5aff12203a53 already in progress: creating tablet
I20260811 18:57:27.503237 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5193d57970f2486795256698cdf51e6a (DEFAULT_TABLE table=default.Table_2962915ef36940f8b6dee3a420d74c59 [id=c69988bcd74d4ac5b0ed3b8ce97ffec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503497 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 18c599d945044e0088bc66c8472d2495 (DEFAULT_TABLE table=default.Table_120c3bf5084e4fd1a967d4564e474f6c [id=c6b3d432c45c440f80a484b8220cbef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503255 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 06519254820241f1bd4bbceb3faedca3 (DEFAULT_TABLE table=default.Table_0f271e4958af4bfbb65af29a053ac2d4 [id=2d9b9e4757454293a104889e19de1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.503953 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5193d57970f2486795256698cdf51e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504163 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06519254820241f1bd4bbceb3faedca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504087 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 06519254820241f1bd4bbceb3faedca3 (DEFAULT_TABLE table=default.Table_0f271e4958af4bfbb65af29a053ac2d4 [id=2d9b9e4757454293a104889e19de1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.504513 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18c599d945044e0088bc66c8472d2495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.504488 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5193d57970f2486795256698cdf51e6a (DEFAULT_TABLE table=default.Table_2962915ef36940f8b6dee3a420d74c59 [id=c69988bcd74d4ac5b0ed3b8ce97ffec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.514492 3555 catalog_manager.cc:5495] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:27.515898 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_9b7ce6401d2c44b9937b7f442ccbd56d [id=bc887ab2039c4ff0a9c3ff318bf243e4] alter complete (version 4)
I20260811 18:57:27.516661 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ab9ee42f727a415ab19b78c0e5bd013a" } new_table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" modify_external_catalogs: true
I20260811 18:57:27.520280 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
I20260811 18:57:27.520936 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 419824ebd14e415f90072b8c335f1ad1 (DEFAULT_TABLE table=default.Table_a4f7f1249bb248359c032a4e20f7c9b4 [id=7087e984854645998d78661c89e4503d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.521224 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_988443c8d6bd4b06b90d4e7f95a0998c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.522020 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_988443c8d6bd4b06b90d4e7f95a0998c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.524413 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:27.529649 3678 leader_election.cc:334] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.161s (SENT)
I20260811 18:57:27.534832 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a49b8b344177487fb5bd096ee9cdea2f
I20260811 18:57:27.535581 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eda358ebeebb4b949f6ee9dc1c5eb80a" } new_table_name: "default.Table_744e4342189a4b29b424eebd6ad7542f" modify_external_catalogs: true
I20260811 18:57:27.538208 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet a49b8b344177487fb5bd096ee9cdea2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.541198 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c21eb02db3de4c97878fa8badccaa6e3
I20260811 18:57:27.542238 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1af253ec487e4e30b462dc9cb3b1b3cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.543176 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1af253ec487e4e30b462dc9cb3b1b3cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.545106 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:27.546130 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 4059e5907b1e488ab9e1d04e563f96cd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.549124 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.549515 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.550599 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.550864 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a49b8b344177487fb5bd096ee9cdea2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.551124 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet c21eb02db3de4c97878fa8badccaa6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.551435 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.556875 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 9ccceeb89da34853a768df69e1298457 (DEFAULT_TABLE table=default.Table_4f6cd18a5e464333bbdc36290a29eaea [id=edb6b1a8db364ec089c07b27e22da454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.557524 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ccceeb89da34853a768df69e1298457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.559947 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 78868ff1f8024163ae1d782151100cd4 (DEFAULT_TABLE table=default.Table_9d42d349a68447e39dbdda4dea1dd35b [id=45b2ec435ce94becafdb869c7437640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.560461 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78868ff1f8024163ae1d782151100cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.560506 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 114d22f22d3d4dc09563aa1108e1f509 (DEFAULT_TABLE table=default.Table_256aa74bb2884a0d83f0d17b159b9e49 [id=4ec9375cd8b64a2ba89d8759b6a789b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.560997 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 053d923618464b0c919563b4be9bc118 (DEFAULT_TABLE table=default.Table_e80925854f7a430c93376f6448db3632 [id=d8411ab33e004d1ebe3c9aa92a1d4721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.561676 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 5d93073a69c143ff9460248848a108d4 (DEFAULT_TABLE table=default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562132 3863 tablet_service.cc:1467] Processing CreateTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b (DEFAULT_TABLE table=default.Table_8f1e69da8480425682a8348cad2ffa51 [id=79f4456b3f5442c98f7d361d2b6be7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562548 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 78868ff1f8024163ae1d782151100cd4 (DEFAULT_TABLE table=default.Table_9d42d349a68447e39dbdda4dea1dd35b [id=45b2ec435ce94becafdb869c7437640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562734 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 114d22f22d3d4dc09563aa1108e1f509 (DEFAULT_TABLE table=default.Table_256aa74bb2884a0d83f0d17b159b9e49 [id=4ec9375cd8b64a2ba89d8759b6a789b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.562937 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9ccceeb89da34853a768df69e1298457 (DEFAULT_TABLE table=default.Table_4f6cd18a5e464333bbdc36290a29eaea [id=edb6b1a8db364ec089c07b27e22da454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.563246 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114d22f22d3d4dc09563aa1108e1f509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.563409 3861 tablet_service.cc:1467] Processing CreateTablet for tablet dd80633f926942e88f54b32d9a03ea3a (DEFAULT_TABLE table=default.Table_f9d18547a0114b83bfbed5110e565661 [id=199e295bad4d4c80b2887b8e9386b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.564203 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18c599d945044e0088bc66c8472d2495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.566243 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 053d923618464b0c919563b4be9bc118 (DEFAULT_TABLE table=default.Table_e80925854f7a430c93376f6448db3632 [id=d8411ab33e004d1ebe3c9aa92a1d4721]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.566538 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 17247ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:27.566645 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06519254820241f1bd4bbceb3faedca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.566751 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053d923618464b0c919563b4be9bc118. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.569689 3678 leader_election.cc:334] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.894s (SENT)
I20260811 18:57:27.570926 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5193d57970f2486795256698cdf51e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.570832 3722 tablet_service.cc:1467] Processing CreateTablet for tablet dd80633f926942e88f54b32d9a03ea3a (DEFAULT_TABLE table=default.Table_f9d18547a0114b83bfbed5110e565661 [id=199e295bad4d4c80b2887b8e9386b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.571333 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd80633f926942e88f54b32d9a03ea3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.574668 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111),
I20260811 18:57:27.575019 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:27.575610 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114d22f22d3d4dc09563aa1108e1f509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.578735 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053d923618464b0c919563b4be9bc118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.582653 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d93073a69c143ff9460248848a108d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.586529 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1342b8cd65b4c31a47b27a79c678d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.591790 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78868ff1f8024163ae1d782151100cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.594761 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ccceeb89da34853a768df69e1298457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.598587 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd80633f926942e88f54b32d9a03ea3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.598500 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5ab42c67414f6699e48987979dfa68 (DEFAULT_TABLE table=default.Table_b5da605dec7141f8aa11be6246b92608 [id=f641203ddd0348a39947fb306dd07cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.599032 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5ab42c67414f6699e48987979dfa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.602828 3723 tablet_service.cc:1467] Processing CreateTablet for tablet e7d75d8468fc4c938fde0b2951d84562 (DEFAULT_TABLE table=default.Table_07499669fc704a31830a06152cd615cf [id=f9c87a8d5bca46369a27d34a7cdea6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.603379 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d75d8468fc4c938fde0b2951d84562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.606778 3709 tablet_service.cc:1467] Processing CreateTablet for tablet c80955bd879c4c1eb16b07ad38096259 (DEFAULT_TABLE table=default.Table_69f270a4181547b2a2cc604a814c82c7 [id=74ef6dbec821458e88361f3cea2175e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.607095 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5ab42c67414f6699e48987979dfa68 (DEFAULT_TABLE table=default.Table_b5da605dec7141f8aa11be6246b92608 [id=f641203ddd0348a39947fb306dd07cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.607448 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80955bd879c4c1eb16b07ad38096259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.608086 3862 tablet_service.cc:1467] Processing CreateTablet for tablet e7d75d8468fc4c938fde0b2951d84562 (DEFAULT_TABLE table=default.Table_07499669fc704a31830a06152cd615cf [id=f9c87a8d5bca46369a27d34a7cdea6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.608865 3858 tablet_service.cc:1467] Processing CreateTablet for tablet c80955bd879c4c1eb16b07ad38096259 (DEFAULT_TABLE table=default.Table_69f270a4181547b2a2cc604a814c82c7 [id=74ef6dbec821458e88361f3cea2175e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.609658 3872 tablet_service.cc:1467] Processing CreateTablet for tablet d0ff928a633e4cae9d49229b989216d8 (DEFAULT_TABLE table=default.Table_5eb58df7cc2042f69300d10a7999719b [id=1fedf260324e464b9e390a637d62deeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.622602 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5ab42c67414f6699e48987979dfa68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.623242 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ff928a633e4cae9d49229b989216d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.623867 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80955bd879c4c1eb16b07ad38096259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.624408 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d75d8468fc4c938fde0b2951d84562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.630573 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1f25b2427bf4b23b84761e4fb0dba55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:27.631086 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f287040b879642448367abe2b1c73cd9
W20260811 18:57:27.631528 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f25b2427bf4b23b84761e4fb0dba55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.631645 3549 catalog_manager.cc:6565] ReplaceTablet: tablet f287040b879642448367abe2b1c73cd9 deleted and replaced by tablet 482c2c39beb14880b40a600214c5efe4
I20260811 18:57:27.632066 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.632720 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:27.633397 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:27.633601 5091 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.633850 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 14e34732bda44670ae9da8facc405221 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:27.633756 5091 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.634449 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2be6762e326448d7b677a5641e2b3103
I20260811 18:57:27.634570 5091 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.635011 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db07259b8d8744d49c2c9fc89d4c5506" } new_table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" modify_external_catalogs: true
I20260811 18:57:27.635479 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.636520 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cafc7915ae7e4280a4d71e3bce4ca57c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.639606 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 419824ebd14e415f90072b8c335f1ad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.641537 5161 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.641980 5161 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.643637 5161 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.645130 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet f287040b879642448367abe2b1c73cd9 with delete_type TABLET_DATA_DELETED (replaced by tablet 482c2c39beb14880b40a600214c5efe4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.645607 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be6762e326448d7b677a5641e2b3103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.647060 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet ef393c341bb54baf87176db5b5254ec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.647225 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f287040b879642448367abe2b1c73cd9 with delete_type TABLET_DATA_DELETED (replaced by tablet 482c2c39beb14880b40a600214c5efe4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.647858 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be6762e326448d7b677a5641e2b3103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.649755 3716 tablet_service.cc:1467] Processing CreateTablet for tablet d0ff928a633e4cae9d49229b989216d8 (DEFAULT_TABLE table=default.Table_5eb58df7cc2042f69300d10a7999719b [id=1fedf260324e464b9e390a637d62deeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.654752 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.654881 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.655246 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.655746 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.656515 3718 tablet_service.cc:1467] Processing CreateTablet for tablet b4f21761b37b42dbb814bd54cca89452 (DEFAULT_TABLE table=default.Table_0790be2c84c14fb3aeb2add67667646e [id=d9452934b441409d9bf9065826e30176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.657295 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 5d93073a69c143ff9460248848a108d4 (DEFAULT_TABLE table=default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.658229 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.658998 3676 leader_election.cc:341] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.660377 3851 tablet_service.cc:1467] Processing CreateTablet for tablet b4f21761b37b42dbb814bd54cca89452 (DEFAULT_TABLE table=default.Table_0790be2c84c14fb3aeb2add67667646e [id=d9452934b441409d9bf9065826e30176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.660959 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f21761b37b42dbb814bd54cca89452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.661613 3719 tablet_service.cc:1467] Processing CreateTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b (DEFAULT_TABLE table=default.Table_8f1e69da8480425682a8348cad2ffa51 [id=79f4456b3f5442c98f7d361d2b6be7e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.662968 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 38a4783168554053aa953dba5e893b6b (DEFAULT_TABLE table=default.Table_7c395f6c4c364b7eb463f78e16675847 [id=17f081f75cf9467fa9c7b46b110403cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.665107 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6880ad27a3eb4599b407edeb2969a08f (DEFAULT_TABLE table=default.Table_41c44c966ece466eb232aeab834c5fe6 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.666021 3720 tablet_service.cc:1467] Processing CreateTablet for tablet b62ef754b8264afc8df6b4fd4bdb3fee (DEFAULT_TABLE table=default.Table_9df3de8d1ff649e8a41bff8b3de9d92f [id=f39457130ce54643a89d0928d8624350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.667583 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 261daa79df704a019f567bedb1046a15 (DEFAULT_TABLE table=default.Table_7db24c0702d141f08ea7023227018d81 [id=e814d7c224214dd3a051525c5ac34391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.670409 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 6880ad27a3eb4599b407edeb2969a08f (DEFAULT_TABLE table=default.Table_41c44c966ece466eb232aeab834c5fe6 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.670832 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880ad27a3eb4599b407edeb2969a08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.675889 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 498) took 5629 ms. Trace:
I20260811 18:57:27.676919 4162 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.677627 4162 ts_tablet_manager.cc:1397] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.284s user 0.000s sys 0.006s
I20260811 18:57:27.681614 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:27.681937 3744 raft_consensus.cc:2454] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:27.670111 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 261daa79df704a019f567bedb1046a15 (DEFAULT_TABLE table=default.Table_7db24c0702d141f08ea7023227018d81 [id=e814d7c224214dd3a051525c5ac34391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.667222 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b62ef754b8264afc8df6b4fd4bdb3fee (DEFAULT_TABLE table=default.Table_9df3de8d1ff649e8a41bff8b3de9d92f [id=f39457130ce54643a89d0928d8624350]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.684167 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a4783168554053aa953dba5e893b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.684207 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 38a4783168554053aa953dba5e893b6b (DEFAULT_TABLE table=default.Table_7c395f6c4c364b7eb463f78e16675847 [id=17f081f75cf9467fa9c7b46b110403cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.685299 3824 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.685791 5161 raft_consensus.cc:2781] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:27.685882 5161 raft_consensus.cc:482] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:27.685947 5161 raft_consensus.cc:3037] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.687215 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 60f202e2f0934241be03ce67399d4b9e (DEFAULT_TABLE table=default.Table_b50aff06b9214a069e8b74b23089e9bb [id=b840acbfd40e46e689a8bbe29b24aa9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.688657 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a4783168554053aa953dba5e893b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.694600 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1342b8cd65b4c31a47b27a79c678d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.695405 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62ef754b8264afc8df6b4fd4bdb3fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.697974 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261daa79df704a019f567bedb1046a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.705035 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9fc9077de3204866a090b8d53ea5bd1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.707137 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fc9077de3204866a090b8d53ea5bd1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.708843 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4059e5907b1e488ab9e1d04e563f96cd
I20260811 18:57:27.710568 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 4059e5907b1e488ab9e1d04e563f96cd deleted and replaced by tablet 99bb8e934e6c44ca88640715b11e6bef
W20260811 18:57:27.711737 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 302cf69ea7c14fdaac72f6a9a9aff113 consensus state has no leader
I20260811 18:57:27.713613 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.714170 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ce5b9945b46348149fb08dafb33f61b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.715090 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce5b9945b46348149fb08dafb33f61b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.715878 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.716853 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7db24c0702d141f08ea7023227018d81" } new_table_name: "default.Table_0f3752c97bf04c169b9d733d20b26e0c" modify_external_catalogs: true
I20260811 18:57:27.723501 4162 raft_consensus.cc:348] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.728647 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 15380ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:27.676672 4169 rpcz_store.cc:276] 0811 18:57:22.046042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.046676 (+ 634us) service_pool.cc:227] Handling call
0811 18:57:27.498647 (+5451971us) ts_tablet_manager.cc:1145] Deleting tablet 3f25d1cd3b114d7d9a3e0fb15d30231e
0811 18:57:27.498666 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.675858 (+177192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5389662}
I20260811 18:57:27.730245 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" } modify_external_catalogs: true
I20260811 18:57:27.730567 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d1d051735e24450cb04f8f19ea9ec4d8" } modify_external_catalogs: true
W20260811 18:57:27.731217 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3f25d1cd3b114d7d9a3e0fb15d30231e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f25d1cd3b114d7d9a3e0fb15d30231e already in progress: creating tablet
I20260811 18:57:27.732960 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0a55918c905f4a4386f6529271fb60dc" } new_table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" modify_external_catalogs: true
I20260811 18:57:27.735206 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9743ba51a07b4fca8aaf1a188a692aac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.736282 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9743ba51a07b4fca8aaf1a188a692aac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.740190 3854 tablet_service.cc:1467] Processing CreateTablet for tablet e42f184431074846820dfd1939148025 (DEFAULT_TABLE table=default.Table_558f1f69ec7547c3b029b4672f04c6e6 [id=c5608ce4470d487fbf252c1890491b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:27.745851 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" } modify_external_catalogs: true
I20260811 18:57:27.746189 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_32b9a3e653554115b17152dc38aa0439" } modify_external_catalogs: true
I20260811 18:57:27.749226 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:27.750303 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd92de7863b94ff38b2c402c27ddf26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:27.765596 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 500) took 5645 ms. Trace:
I20260811 18:57:27.765736 4169 rpcz_store.cc:276] 0811 18:57:22.120313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.121411 (+ 1098us) service_pool.cc:227] Handling call
0811 18:57:27.731606 (+5610195us) ts_tablet_manager.cc:1145] Deleting tablet 7991dabe263b44f6b7b7d1d43b96ee44
0811 18:57:27.731625 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.765568 (+ 33943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5609644}
W20260811 18:57:27.767853 4863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:27.768594 4863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:27.768962 4863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:27.769973 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7991dabe263b44f6b7b7d1d43b96ee44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7991dabe263b44f6b7b7d1d43b96ee44 already in progress: creating tablet
I20260811 18:57:27.742138 5161 raft_consensus.cc:504] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.772598 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3670f8908d54ebb82f698cdd207e48f consensus state has no leader
I20260811 18:57:27.773463 4162 raft_consensus.cc:374] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.773594 4162 raft_consensus.cc:729] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.773831 4162 consensus_queue.cc:260] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.775045 4162 ts_tablet_manager.cc:1428] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.096s user 0.000s sys 0.003s
I20260811 18:57:27.729636 5091 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.777395 5091 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.778266 5091 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.782727 4162 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.783205 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de" } modify_external_catalogs: true
I20260811 18:57:27.783645 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_971d1dda6c9c4e65a732e83f6528e2de" } modify_external_catalogs: true
W20260811 18:57:27.786643 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 261daa79df704a019f567bedb1046a15 consensus state has no leader
I20260811 18:57:27.787351 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" } modify_external_catalogs: true
I20260811 18:57:27.787633 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_04036b5eb2da4ac0907c22692a6a4452" } modify_external_catalogs: true
I20260811 18:57:27.787597 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet b6f36ab6bb774257b5c2692c64eff63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.791512 5161 leader_election.cc:288] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:27.794493 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4093564a88264ed59c945523c9ddc9b3
I20260811 18:57:27.803488 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.804867 3676 leader_election.cc:341] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.808858 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d7d986900244ddbc05dbe73a708dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.745083 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 60f202e2f0934241be03ce67399d4b9e (DEFAULT_TABLE table=default.Table_b50aff06b9214a069e8b74b23089e9bb [id=b840acbfd40e46e689a8bbe29b24aa9d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:27.753824 3678 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.830s (SENT)
W20260811 18:57:27.815521 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 61734.1us
I20260811 18:57:27.816162 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f184431074846820dfd1939148025. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:27.816223 3678 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.082s (SENT)
I20260811 18:57:27.758087 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880ad27a3eb4599b407edeb2969a08f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.818792 4162 tablet_bootstrap.cc:654] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.822821 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d7d986900244ddbc05dbe73a708dcf" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:27.823105 3743 raft_consensus.cc:3037] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.828079 5161 consensus_queue.cc:1035] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.829948 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 501) took 5701 ms. Trace:
I20260811 18:57:27.831219 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4f21761b37b42dbb814bd54cca89452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.832101 4169 rpcz_store.cc:276] 0811 18:57:22.128214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.128272 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:27.766133 (+5637861us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:27.802390 (+ 36257us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.829920 (+ 27530us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":65,"tablet-delete.queue_time_us":5637435}
I20260811 18:57:27.833480 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62ef754b8264afc8df6b4fd4bdb3fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.834383 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 261daa79df704a019f567bedb1046a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.835412 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 4093564a88264ed59c945523c9ddc9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:27.838807 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0ff928a633e4cae9d49229b989216d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.840569 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 502) took 5695 ms. Trace:
I20260811 18:57:27.840692 4169 rpcz_store.cc:276] 0811 18:57:22.145264 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.147934 (+ 2670us) service_pool.cc:227] Handling call
0811 18:57:27.835963 (+5688029us) ts_tablet_manager.cc:1145] Deleting tablet 9ec4bba841dd446aa3db751358ce7f32
0811 18:57:27.835978 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.840540 (+ 4562us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5687576}
I20260811 18:57:27.841042 4169 tablet_replica.cc:330] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:27.841277 4169 raft_consensus.cc:2229] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:27.841953 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9ec4bba841dd446aa3db751358ce7f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ec4bba841dd446aa3db751358ce7f32 already in progress: creating tablet
W20260811 18:57:27.843356 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5d93073a69c143ff9460248848a108d4 (table default.Table_1623cbb9139845d2a67dab2017c8e7ff [id=11d887e48988464c932c5844031fb451], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5d93073a69c143ff9460248848a108d4 consensus state has no leader
I20260811 18:57:27.845286 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03949bb266e742cbbe858496f37177f3" } modify_external_catalogs: true
I20260811 18:57:27.845705 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03949bb266e742cbbe858496f37177f3" } modify_external_catalogs: true
I20260811 18:57:27.847832 4162 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.847991 4162 ts_tablet_manager.cc:1397] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.005s
I20260811 18:57:27.848529 4169 raft_consensus.cc:2258] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:27.849015 4162 raft_consensus.cc:348] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.849258 4162 raft_consensus.cc:374] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.849326 4162 raft_consensus.cc:729] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.849525 4162 consensus_queue.cc:260] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.850303 4169 ts_tablet_manager.cc:1905] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:27.850803 4162 ts_tablet_manager.cc:1428] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:27.851181 3743 raft_consensus.cc:2454] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:27.852286 4162 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:27.858757 3824 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.078s (SENT)
I20260811 18:57:27.859010 3824 leader_election.cc:302] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:27.859848 5161 raft_consensus.cc:2726] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:27.863358 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e4e304a679345cd9289252278ddb281" } new_table_name: "default.Table_deb475a5050c4f3dbaece0eb1e4ac618" modify_external_catalogs: true
I20260811 18:57:27.865917 4162 tablet_bootstrap.cc:654] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.879050 4167 ts_tablet_manager.cc:1918] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:27.880165 3824 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:27.881134 5080 raft_consensus.cc:2781] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.884706 4163 tablet_bootstrap.cc:492] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:27.887353 4163 ts_tablet_manager.cc:1397] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.398s user 0.001s sys 0.005s
W20260811 18:57:27.890555 3678 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.384s (SENT)
I20260811 18:57:27.892999 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d93073a69c143ff9460248848a108d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.900417 5080 raft_consensus.cc:686] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:27.900826 5080 consensus_queue.cc:237] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.901572 4163 raft_consensus.cc:348] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.906292 4163 raft_consensus.cc:374] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.907658 4163 raft_consensus.cc:729] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:27.909238 4863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:27.911353 4863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260811 18:57:27.912150 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f202e2f0934241be03ce67399d4b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.910483 4163 consensus_queue.cc:260] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:27.913576 4863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:27.916630 4863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:27.919866 4163 ts_tablet_manager.cc:1428] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.031s user 0.003s sys 0.002s
I20260811 18:57:27.922415 4167 log.cc:1196] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:27.923076 4167 ts_tablet_manager.cc:1939] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:27.925951 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 567) took 4144 ms. Trace:
I20260811 18:57:27.926441 5091 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.926721 5091 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.927513 5091 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:27.927577 4163 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:27.928997 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:27.932124 3676 leader_election.cc:341] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:27.944914 4162 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:27.945241 4162 ts_tablet_manager.cc:1397] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.010s
I20260811 18:57:27.946885 4162 raft_consensus.cc:348] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.947436 4162 raft_consensus.cc:374] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.947706 4162 raft_consensus.cc:729] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.949133 4162 consensus_queue.cc:260] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.926599 4167 rpcz_store.cc:276] 0811 18:57:23.781728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.781793 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:27.382114 (+3600321us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:27.382127 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.457664 (+ 75537us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.878588 (+420924us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:27.925875 (+ 47287us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3595130}
I20260811 18:57:27.955050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 575) took 4055 ms. Trace:
I20260811 18:57:27.955185 4167 rpcz_store.cc:276] 0811 18:57:23.899666 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.899799 (+ 133us) service_pool.cc:227] Handling call
0811 18:57:27.954849 (+4055050us) ts_tablet_manager.cc:1145] Deleting tablet 56d24aff9fd34f3eb05fdf942cc7efc7
0811 18:57:27.954863 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955027 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3988968}
I20260811 18:57:27.955593 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 586) took 3614 ms. Trace:
I20260811 18:57:27.955683 4167 rpcz_store.cc:276] 0811 18:57:24.341367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.341434 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:27.955470 (+3614036us) ts_tablet_manager.cc:1145] Deleting tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
0811 18:57:27.955481 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955576 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3613490}
I20260811 18:57:27.955989 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 587) took 3613 ms. Trace:
I20260811 18:57:27.956094 4167 rpcz_store.cc:276] 0811 18:57:24.342289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.342371 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:27.955888 (+3613517us) ts_tablet_manager.cc:1145] Deleting tablet 64b922f2a71544e8a18533aa0a1ccb52
0811 18:57:27.955898 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.955972 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3612931}
I20260811 18:57:27.956429 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 585) took 3615 ms. Trace:
I20260811 18:57:27.956511 4167 rpcz_store.cc:276] 0811 18:57:24.341129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.358587 (+ 17458us) service_pool.cc:227] Handling call
0811 18:57:27.956320 (+3597733us) ts_tablet_manager.cc:1145] Deleting tablet 14e34732bda44670ae9da8facc405221
0811 18:57:27.956333 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.956413 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3575451}
I20260811 18:57:27.956818 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 584) took 3615 ms. Trace:
I20260811 18:57:27.956898 4167 rpcz_store.cc:276] 0811 18:57:24.340950 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.362104 (+ 21154us) service_pool.cc:227] Handling call
0811 18:57:27.956718 (+3594614us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:27.956730 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.956801 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3575701}
I20260811 18:57:27.957511 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 592) took 3181 ms. Trace:
I20260811 18:57:27.957615 4167 rpcz_store.cc:276] 0811 18:57:24.775962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.776026 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:27.957386 (+3181360us) ts_tablet_manager.cc:1145] Deleting tablet 0b40da8cb7324da9804e6fdafd8e8266
0811 18:57:27.957398 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.957490 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3180227}
I20260811 18:57:27.958002 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 593) took 3174 ms. Trace:
I20260811 18:57:27.958051 4162 ts_tablet_manager.cc:1428] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.002s
I20260811 18:57:27.986377 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:27.984453 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 4093564a88264ed59c945523c9ddc9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:27.989532 4162 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:27.991221 4162 tablet_bootstrap.cc:654] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.991117 5080 consensus_queue.cc:1035] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:27.997785 5080 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:27.998557 5080 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:27.999388 5080 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.002673 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.004063 3743 raft_consensus.cc:2454] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.004938 3824 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.005405 5133 raft_consensus.cc:2781] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.005538 5133 raft_consensus.cc:482] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.005708 5133 raft_consensus.cc:3037] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.766605 3710 tablet_service.cc:1467] Processing CreateTablet for tablet e42f184431074846820dfd1939148025 (DEFAULT_TABLE table=default.Table_558f1f69ec7547c3b029b4672f04c6e6 [id=c5608ce4470d487fbf252c1890491b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.007361 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f184431074846820dfd1939148025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.008888 5133 raft_consensus.cc:504] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.009588 5133 leader_election.cc:288] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.010672 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04eb81bf6f454fbfadd27393af15ab17" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.010969 3743 raft_consensus.cc:3037] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.013794 3743 raft_consensus.cc:2454] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.014654 3824 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.015126 5080 raft_consensus.cc:2781] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.015324 5080 raft_consensus.cc:686] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.015674 5080 consensus_queue.cc:237] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.023059 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 already in progress: creating tablet
W20260811 18:57:28.023401 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 64b922f2a71544e8a18533aa0a1ccb52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64b922f2a71544e8a18533aa0a1ccb52 already in progress: creating tablet
W20260811 18:57:28.023643 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 14e34732bda44670ae9da8facc405221 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e34732bda44670ae9da8facc405221 already in progress: creating tablet
W20260811 18:57:28.023895 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7057b8a7bae249caaed304b5c66e6871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7057b8a7bae249caaed304b5c66e6871 already in progress: creating tablet
W20260811 18:57:28.024220 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0b40da8cb7324da9804e6fdafd8e8266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b40da8cb7324da9804e6fdafd8e8266 already in progress: creating tablet
I20260811 18:57:28.025066 4162 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.025421 4162 ts_tablet_manager.cc:1397] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.005s
I20260811 18:57:27.986701 4167 rpcz_store.cc:276] 0811 18:57:24.783914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.783994 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:27.957893 (+3173899us) ts_tablet_manager.cc:1145] Deleting tablet 19d7716739f848d78b9cbbd3975f3d44
0811 18:57:27.957905 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:27.957986 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3173449}
I20260811 18:57:28.027493 4162 raft_consensus.cc:348] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.028028 4162 raft_consensus.cc:374] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.028167 4162 raft_consensus.cc:729] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.029667 4162 consensus_queue.cc:260] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.040555 4162 ts_tablet_manager.cc:1428] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260811 18:57:28.043388 4162 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.039984 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 594) took 3247 ms. Trace:
W20260811 18:57:28.050719 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa95131ce50943e3b0c6b5bdd5a30939 consensus state has no leader
I20260811 18:57:28.054049 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5296710afead477eb67f356c9afbc4a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.047101 4167 rpcz_store.cc:276] 0811 18:57:24.791459 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.791526 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:28.031472 (+3239946us) ts_tablet_manager.cc:1145] Deleting tablet 2d0b59cafb154416952aecaa418e861f
0811 18:57:28.032551 (+ 1079us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.038486 (+ 5935us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3234512}
I20260811 18:57:28.061789 4162 tablet_bootstrap.cc:654] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.067469 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 595) took 3260 ms. Trace:
I20260811 18:57:28.069347 4167 rpcz_store.cc:276] 0811 18:57:24.807343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.808450 (+ 1107us) service_pool.cc:227] Handling call
0811 18:57:28.067332 (+3258882us) ts_tablet_manager.cc:1145] Deleting tablet f25ff8a9e3a944269e9799995c73ebe0
0811 18:57:28.067348 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.067444 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3255769}
W20260811 18:57:28.082361 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5296710afead477eb67f356c9afbc4a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.089263 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3de36d9c6d4542be9ee2c873f0bfcfca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.094739 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3de36d9c6d4542be9ee2c873f0bfcfca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.091617 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 596) took 3283 ms. Trace:
W20260811 18:57:28.089655 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 19d7716739f848d78b9cbbd3975f3d44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d7716739f848d78b9cbbd3975f3d44 already in progress: creating tablet
W20260811 18:57:28.096432 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2d0b59cafb154416952aecaa418e861f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d0b59cafb154416952aecaa418e861f already in progress: creating tablet
W20260811 18:57:28.097126 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f25ff8a9e3a944269e9799995c73ebe0 because the tablet was not found. No further retry: Not found: Tablet not found: f25ff8a9e3a944269e9799995c73ebe0
I20260811 18:57:28.096150 4167 rpcz_store.cc:276] 0811 18:57:24.807533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.808090 (+ 557us) service_pool.cc:227] Handling call
0811 18:57:28.086420 (+3278330us) ts_tablet_manager.cc:1145] Deleting tablet 3b40e44a2f4940af891eddf6c3b6293c
0811 18:57:28.087134 (+ 714us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.090898 (+ 3764us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3273901}
I20260811 18:57:28.103229 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.104060 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3b40e44a2f4940af891eddf6c3b6293c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b40e44a2f4940af891eddf6c3b6293c already in progress: creating tablet
I20260811 18:57:28.112339 5091 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.112490 5091 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.111179 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 597) took 3302 ms. Trace:
I20260811 18:57:28.113220 5091 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.114197 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.115180 3676 leader_election.cc:341] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.117940 4162 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.118297 4162 ts_tablet_manager.cc:1397] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.076s user 0.010s sys 0.000s
I20260811 18:57:28.113262 4167 rpcz_store.cc:276] 0811 18:57:24.807782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.807943 (+ 161us) service_pool.cc:227] Handling call
0811 18:57:28.107494 (+3299551us) ts_tablet_manager.cc:1145] Deleting tablet 92e49846ab714dd294acdd09d49a83bd
0811 18:57:28.107516 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.110611 (+ 3095us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3290085}
I20260811 18:57:28.120453 4162 raft_consensus.cc:348] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.121004 4162 raft_consensus.cc:374] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.121213 4162 raft_consensus.cc:729] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:28.122646 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 92e49846ab714dd294acdd09d49a83bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e49846ab714dd294acdd09d49a83bd already in progress: creating tablet
I20260811 18:57:28.123067 4162 consensus_queue.cc:260] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.129915 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ae7da0868b114052ba421801b76fb33c
I20260811 18:57:28.130918 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.131693 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6284ad4471b4b7095b11fe7a8f304e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.133697 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.132688 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 598) took 3298 ms. Trace:
W20260811 18:57:28.134420 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.135720 4162 ts_tablet_manager.cc:1428] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.017s user 0.001s sys 0.003s
I20260811 18:57:28.136022 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6d988ef7e9dd43158d107998ea4d82b9
I20260811 18:57:28.136771 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c281ac8dd4a444c08084edb70426f847" } new_table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" modify_external_catalogs: true
I20260811 18:57:28.139567 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7da0868b114052ba421801b76fb33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.139850 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.139505 4162 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.134773 4167 rpcz_store.cc:276] 0811 18:57:24.833863 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833933 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:28.125902 (+3291969us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:28.126930 (+ 1028us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.132334 (+ 5404us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3290291}
W20260811 18:57:28.144157 3678 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.278s (SENT)
I20260811 18:57:28.144526 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b6c70b4163634984bd02dd9b47ffb472
I20260811 18:57:28.145512 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c70b4163634984bd02dd9b47ffb472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.146193 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e79b6e054e914fd39ca0a6282eb0ce17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.146971 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e79b6e054e914fd39ca0a6282eb0ce17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.149354 4167 tablet_replica.cc:330] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.153517 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3baa014716584cb184d8a0e4299a8542" } new_table_name: "default.Table_3b056c714ed947198f75f2c7802ae4dd" modify_external_catalogs: true
I20260811 18:57:28.161612 4167 raft_consensus.cc:2229] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.167603 4163 tablet_bootstrap.cc:654] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.169294 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
I20260811 18:57:28.172438 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f202e2f0934241be03ce67399d4b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.172438 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.178174 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b908c7d655a467bbda1f9d2fdc1b99a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.181114 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 46505ba544104458888dbc2a37115dcc consensus state has no leader
I20260811 18:57:28.181901 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_988443c8d6bd4b06b90d4e7f95a0998c" } new_table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" modify_external_catalogs: true
I20260811 18:57:28.183058 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9914a933914f4377ba70e0b57c70edff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.184628 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2523fa07587246d994eca722c7032105 consensus state has no leader
I20260811 18:57:28.185086 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7" } modify_external_catalogs: true
I20260811 18:57:28.185426 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe81c0c6efaa487bb9ee5992b0a2c9c7" } modify_external_catalogs: true
W20260811 18:57:28.193670 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b908c7d655a467bbda1f9d2fdc1b99a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.196206 4167 raft_consensus.cc:2258] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.196944 4167 ts_tablet_manager.cc:1905] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#4 0xa686e1 in std::__shared_ptr<kudu::tablet::Tablet, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() ../../../include/c++/8/bits/shared_ptr_base.h:1167:31
I20260811 18:57:28.198510 4162 tablet_bootstrap.cc:654] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.202971 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9914a933914f4377ba70e0b57c70edff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.203598 4863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:34821,127.3.72.125:38889,127.3.72.124:44143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:34821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=40953
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
#5 0x7fc7bfa75e66 in std::__shared_ptr<kudu::tablet::Tablet, (__gnu_cxx::_Lock_policy)2>::reset() ../../../include/c++/8/bits/shared_ptr_base.h:1285:9
#6 0x7fc7bfa57c66 in kudu::tablet::TabletReplica::Stop() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_replica.cc:371:13
W20260811 18:57:28.221303 3953 kernel_stack_watchdog.cc:198] Thread 4006 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21151ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7bfab9145 kudu::tablet::OpDriver::ExecuteAsync() at ??:0
@ 0x7fc7bfa60ace kudu::tablet::TabletReplica::SubmitAlterSchema() at ??:0
@ 0x7fc7c232926e kudu::tserver::TabletServiceAdminImpl::AlterSchema() at ??:0
@ 0x7fc7bc95c787 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf()::$_5::operator()() at ??:0
@ 0x7fc7bc95c4b6 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc7b9db79bf std::function<>::operator()() at ??:0
W20260811 18:57:28.224426 3824 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.155s (SENT)
I20260811 18:57:28.224609 3824 leader_election.cc:302] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:28.224807 3824 raft_consensus.cc:2670] T 828993fb34bb4c62bce2a9322d7c992a P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:28.225579 5080 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.225736 5080 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.226599 5080 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.227612 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
W20260811 18:57:28.227885 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6df5545532e346f7abcdc4cd228b84da has no consensus state
I20260811 18:57:28.227943 3743 raft_consensus.cc:2454] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.228504 3824 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.228832 5080 raft_consensus.cc:2781] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.228940 5080 raft_consensus.cc:482] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.229017 5080 raft_consensus.cc:3037] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:28.229682 3540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260811 18:57:28.231031 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260811 18:57:28.232395 5080 raft_consensus.cc:504] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.234670 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799ba9a044e145ec89d0025c03f391ea" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.234932 3743 raft_consensus.cc:3037] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.238490 5080 leader_election.cc:288] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.238834 3743 raft_consensus.cc:2454] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.239254 5091 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.239557 3824 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.239449 5091 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.239756 4863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:28.239923 5080 raft_consensus.cc:2781] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.240487 5091 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:28.240696 3678 leader_election.cc:334] T f25ff8a9e3a944269e9799995c73ebe0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.245s (SENT)
I20260811 18:57:28.241412 5080 raft_consensus.cc:686] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.241765 5080 consensus_queue.cc:237] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.242012 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.242735 3676 leader_election.cc:341] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:28.244287 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b57ed7814a1e440cba362613cbab32f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b57ed7814a1e440cba362613cbab32f8 already in progress: creating tablet
I20260811 18:57:28.245674 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet ae7da0868b114052ba421801b76fb33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.247174 4163 tablet_bootstrap.cc:492] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.247453 4163 ts_tablet_manager.cc:1397] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.321s user 0.012s sys 0.026s
I20260811 18:57:28.248908 4163 raft_consensus.cc:348] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.249332 4163 raft_consensus.cc:374] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.249464 4163 raft_consensus.cc:729] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.252921 4163 consensus_queue.cc:260] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.253376 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d93e11fb1464fcbb4999abaebc3d611" } new_table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" modify_external_catalogs: true
I20260811 18:57:28.253845 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d988ef7e9dd43158d107998ea4d82b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.254441 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c70b4163634984bd02dd9b47ffb472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.254868 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255056 4163 ts_tablet_manager.cc:1428] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260811 18:57:28.255213 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255383 4163 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.255544 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.255885 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.257071 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b6ec419ed7747c3b24dc11992518e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.257527 3824 consensus_peers.cc:487] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:28.263149 4169 ts_tablet_manager.cc:1918] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:28.263299 4169 log.cc:1196] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ace00158b744bd8a97dc1c47fc6049c
I20260811 18:57:28.263770 4169 ts_tablet_manager.cc:1939] T 0ace00158b744bd8a97dc1c47fc6049c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.267477 4162 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.267809 4162 ts_tablet_manager.cc:1397] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.130s user 0.005s sys 0.003s
I20260811 18:57:28.269423 4162 raft_consensus.cc:348] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.269847 4162 raft_consensus.cc:374] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.270962 4162 raft_consensus.cc:729] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.271306 4162 consensus_queue.cc:260] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.272346 4162 ts_tablet_manager.cc:1428] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:28.273041 4162 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.279546 4863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:57:28.280576 4163 tablet_bootstrap.cc:654] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.284029 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ccc63b46a4e346f79d1e03a82490794c consensus state has no leader
W20260811 18:57:28.284855 3678 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.153s (SENT)
I20260811 18:57:28.285089 4162 tablet_bootstrap.cc:654] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.290679 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f9cadd28493e444185170bcd4c314208"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.298672 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9cadd28493e444185170bcd4c314208 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.304731 5091 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.304948 5091 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.305758 5091 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.321318 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:28.321504 4163 tablet_bootstrap.cc:492] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.321797 4163 ts_tablet_manager.cc:1397] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.066s user 0.005s sys 0.001s
W20260811 18:57:28.322109 3676 leader_election.cc:341] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.321756 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 503) took 6058 ms. Trace:
I20260811 18:57:28.323089 4169 rpcz_store.cc:276] 0811 18:57:22.262859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.262943 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:27.840988 (+5578045us) ts_tablet_manager.cc:1145] Deleting tablet 0ace00158b744bd8a97dc1c47fc6049c
0811 18:57:27.841000 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.187746 (+346746us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.263030 (+ 75284us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.321740 (+ 58710us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5576785}
I20260811 18:57:28.324887 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ace00158b744bd8a97dc1c47fc6049c (table default.Table_102e5367d375402c9800357142894fa0 [id=a067d27ecf9c402db2631f2df5f0fea4]) successfully deleted
I20260811 18:57:28.325839 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 509) took 5991 ms. Trace:
I20260811 18:57:28.326568 4167 ts_tablet_manager.cc:1918] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:28.326711 4167 log.cc:1196] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:28.327085 4167 ts_tablet_manager.cc:1939] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.327358 4169 rpcz_store.cc:276] 0811 18:57:22.334216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.334307 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:28.325693 (+5991386us) ts_tablet_manager.cc:1145] Deleting tablet 2605007cc2454599868f44c717deb2d2
0811 18:57:28.325705 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.325817 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5990971}
I20260811 18:57:28.327643 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 599) took 3484 ms. Trace:
I20260811 18:57:28.327754 4167 rpcz_store.cc:276] 0811 18:57:24.842943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.843022 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:28.147764 (+3304742us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:28.148223 (+ 459us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.221926 (+ 73703us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.326414 (+104488us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.327621 (+ 1207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3291176}
W20260811 18:57:28.328122 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2605007cc2454599868f44c717deb2d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2605007cc2454599868f44c717deb2d2 already in progress: creating tablet
I20260811 18:57:28.328536 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ef393c341bb54baf87176db5b5254ec7 (table default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]) successfully deleted
I20260811 18:57:28.329020 4169 tablet_replica.cc:330] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.329509 4162 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.329329 4169 raft_consensus.cc:2229] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.329710 4162 ts_tablet_manager.cc:1397] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.006s
I20260811 18:57:28.329810 4169 raft_consensus.cc:2258] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.330811 4162 raft_consensus.cc:348] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.331117 4162 raft_consensus.cc:374] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.331243 4162 raft_consensus.cc:729] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.331501 4162 consensus_queue.cc:260] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.332142 4162 ts_tablet_manager.cc:1428] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:28.333302 4162 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.326083 4163 raft_consensus.cc:348] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.334695 4163 raft_consensus.cc:374] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.334839 4163 raft_consensus.cc:729] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.335142 4163 consensus_queue.cc:260] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.336979 4169 ts_tablet_manager.cc:1905] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:28.343223 3678 leader_election.cc:334] T 828993fb34bb4c62bce2a9322d7c992a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.114s (SENT)
I20260811 18:57:28.346649 3552 catalog_manager.cc:5495] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:28.349098 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:28.353323 4162 tablet_bootstrap.cc:654] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.354802 4167 tablet_replica.cc:330] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.355028 4167 raft_consensus.cc:2229] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.355273 4167 raft_consensus.cc:2258] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.357435 4167 ts_tablet_manager.cc:1905] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.359006 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7309030d82da426ab87ff64346efd25b" } modify_external_catalogs: true
I20260811 18:57:28.359385 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7309030d82da426ab87ff64346efd25b" } modify_external_catalogs: true
I20260811 18:57:28.362066 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_82ea06e136674461909baae7b6d4e8cd" } modify_external_catalogs: true
I20260811 18:57:28.362282 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_82ea06e136674461909baae7b6d4e8cd" } modify_external_catalogs: true
I20260811 18:57:28.363981 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_70493900fc9346389c1d4b71c611b962" } modify_external_catalogs: true
I20260811 18:57:28.364266 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_70493900fc9346389c1d4b71c611b962" } modify_external_catalogs: true
I20260811 18:57:28.367291 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 6df5545532e346f7abcdc4cd228b84da (DEFAULT_TABLE table=default.Table_ff02853da3f1406c90f961567adf92bf [id=79599e38b25046ae9ad11292ceeddcf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.367233 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4c88a7260fbc4472bbbd3244c0cc4a85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.367805 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df5545532e346f7abcdc4cd228b84da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.368176 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.372709 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_378dd90b6af541b8ab95d9f19681a03f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.373028 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 3317f9d8e2354df38c6342fc48d82f6f (DEFAULT_TABLE table=default.Table_1a2751cf08eb4ce0b354486cf5678c1c [id=10901c364ced4e9fbcbc9611a3ceb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.373458 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3317f9d8e2354df38c6342fc48d82f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.374862 3719 tablet_service.cc:1467] Processing CreateTablet for tablet e1ad3860d03f46baadd5ede854f1a909 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.375409 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ad3860d03f46baadd5ede854f1a909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.376343 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 547f6c33d6414bcfa3eeff20cb9a0302 (DEFAULT_TABLE table=default.Table_8ad9f329a9c449e4ade20157c2601ee8 [id=8f080597ed9e4f358b7539596a864252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352384 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 6df5545532e346f7abcdc4cd228b84da (DEFAULT_TABLE table=default.Table_ff02853da3f1406c90f961567adf92bf [id=79599e38b25046ae9ad11292ceeddcf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352488 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 547f6c33d6414bcfa3eeff20cb9a0302 (DEFAULT_TABLE table=default.Table_8ad9f329a9c449e4ade20157c2601ee8 [id=8f080597ed9e4f358b7539596a864252]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.352789 3854 tablet_service.cc:1467] Processing CreateTablet for tablet e1ad3860d03f46baadd5ede854f1a909 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.365613 5080 consensus_queue.cc:1035] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:28.371824 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3317f9d8e2354df38c6342fc48d82f6f (DEFAULT_TABLE table=default.Table_1a2751cf08eb4ce0b354486cf5678c1c [id=10901c364ced4e9fbcbc9611a3ceb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.378180 4163 ts_tablet_manager.cc:1428] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.056s user 0.001s sys 0.001s
I20260811 18:57:28.388569 4163 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.397537 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547f6c33d6414bcfa3eeff20cb9a0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.399201 5091 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.399375 5091 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.406760 5091 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.407796 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.408625 3676 leader_election.cc:341] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:28.423512 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_378dd90b6af541b8ab95d9f19681a03f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.434542 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_558f1f69ec7547c3b029b4672f04c6e6" } new_table_name: "default.Table_267cfa966911410c8d062f77ed7d2453" modify_external_catalogs: true
I20260811 18:57:28.442569 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d8146f47d9ae48889ac7efc954d4a6b7
I20260811 18:57:28.442904 3862 tablet_service.cc:1467] Processing CreateTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e (DEFAULT_TABLE table=default.Table_82ea06e136674461909baae7b6d4e8cd [id=a9b8af67e69f42229cc4340500fc95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.443511 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 799ba9a044e145ec89d0025c03f391ea with delete_type TABLET_DATA_DELETED (replaced by tablet e1af33fd98c6486f975368ec90493db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.439945 5207 tablet.cc:1722] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:28.444865 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:28.449043 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.384s (SENT) [suppressed 13 similar messages]
I20260811 18:57:28.446102 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_710d4f427cd6437a908b1109b4a140a1" } new_table_name: "default.Table_1c4b1f6ef3514273a9cb50060b3a72d3" modify_external_catalogs: true
I20260811 18:57:28.457540 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90" } modify_external_catalogs: true
I20260811 18:57:28.457957 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_14b83cad2c2f4713b46f4d1fa3782f90" } modify_external_catalogs: true
I20260811 18:57:28.459290 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.459857 3714 tablet_service.cc:1467] Processing CreateTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e (DEFAULT_TABLE table=default.Table_82ea06e136674461909baae7b6d4e8cd [id=a9b8af67e69f42229cc4340500fc95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.460021 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet d8146f47d9ae48889ac7efc954d4a6b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.460031 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef4494689f0147118d362ed74d1bf78c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.461292 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38c172a9fef4a3ab8ee188335a7c97e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.462059 3678 leader_election.cc:334] T 405773849fb94800be61ad8a7a6055a4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.302s (SENT)
W20260811 18:57:28.462443 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef4494689f0147118d362ed74d1bf78c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.471124 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a38c172a9fef4a3ab8ee188335a7c97e
I20260811 18:57:28.472219 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.474035 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.479075 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
I20260811 18:57:28.480299 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c3df98432314ff2b4cbd7004701b258"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.480710 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:28.481345 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c3df98432314ff2b4cbd7004701b258 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.485268 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.485966 5209 tablet.cc:1722] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:28.487170 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_832e411b94e24eb49b8798f66fa24774"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.491329 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_832e411b94e24eb49b8798f66fa24774 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.491611 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet a38c172a9fef4a3ab8ee188335a7c97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.493290 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.495833 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cf6c8d542ae2489ba466ffadd518db12
W20260811 18:57:28.499253 3678 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.153s (SENT)
I20260811 18:57:28.501099 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.504653 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fdae8856d30a46259e91345b60181c2b" } new_table_name: "default.Table_f5f50e8f17e04760a61e10c54d1aa657" modify_external_catalogs: true
I20260811 18:57:28.508139 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.508503 4162 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.512262 4162 ts_tablet_manager.cc:1397] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.179s user 0.008s sys 0.011s
I20260811 18:57:28.518306 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_862e6dcdf4f748738d4f651170e5611f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.523718 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_862e6dcdf4f748738d4f651170e5611f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.519164 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6064366e4ea94146903cb48b739102e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.524611 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6064366e4ea94146903cb48b739102e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.527882 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38c172a9fef4a3ab8ee188335a7c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528254 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 547f6c33d6414bcfa3eeff20cb9a0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528568 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3317f9d8e2354df38c6342fc48d82f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.528985 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6df5545532e346f7abcdc4cd228b84da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.530498 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ad3860d03f46baadd5ede854f1a909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.531510 4163 tablet_bootstrap.cc:654] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:28.538556 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:28.533421 4162 raft_consensus.cc:348] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.539525 4162 raft_consensus.cc:374] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.539601 4162 raft_consensus.cc:729] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.542259 3714 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0e443ab84424dac307b745b496986 (DEFAULT_TABLE table=default.Table_a35f3caa50e849abac2fc4d5711ea9fb [id=faa0f0b707b34742a67085c9c3f4fe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.543273 3858 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0e443ab84424dac307b745b496986 (DEFAULT_TABLE table=default.Table_a35f3caa50e849abac2fc4d5711ea9fb [id=faa0f0b707b34742a67085c9c3f4fe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.556143 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0e443ab84424dac307b745b496986. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.561476 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:28.563577 5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:28.570363 4863 server_base.cc:1034] running on GCE node
W20260811 18:57:28.572398 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e42f184431074846820dfd1939148025 consensus state has no leader
I20260811 18:57:28.572923 4863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:28.573153 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a79b27c0bd524af985c2e6167624551a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.573987 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a79b27c0bd524af985c2e6167624551a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.573985 4162 consensus_queue.cc:260] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:28.575515 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dce6143309948d49e229dc9890d904f consensus state has no leader
I20260811 18:57:28.579861 4163 tablet_bootstrap.cc:492] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.580400 4163 ts_tablet_manager.cc:1397] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.192s user 0.003s sys 0.006s
W20260811 18:57:28.586228 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 44045.4us
I20260811 18:57:28.589077 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 (DEFAULT_TABLE table=default.Table_068dc550d67d413c916d1384c3cb811c [id=377abe00a3d6469ab4285270de14cbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.589644 4863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:28.590655 4162 ts_tablet_manager.cc:1428] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.075s user 0.004s sys 0.000s
I20260811 18:57:28.591161 4162 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.600121 4163 raft_consensus.cc:348] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.600318 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 (DEFAULT_TABLE table=default.Table_068dc550d67d413c916d1384c3cb811c [id=377abe00a3d6469ab4285270de14cbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.600942 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d721b0e55384d8a8efd88a0e47b1504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.602548 4163 raft_consensus.cc:374] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.602712 4163 raft_consensus.cc:729] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.602999 4163 consensus_queue.cc:260] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.605851 4163 ts_tablet_manager.cc:1428] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260811 18:57:28.603519 4169 ts_tablet_manager.cc:1918] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.607009 4169 log.cc:1196] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e51a08e6dbf04e3fa58689ee0cd45db3
I20260811 18:57:28.608242 4169 ts_tablet_manager.cc:1939] T e51a08e6dbf04e3fa58689ee0cd45db3 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.610416 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f13a14bd64704798bf6de4fc05201a31
I20260811 18:57:28.621986 4863 hybrid_clock.cc:648] HybridClock initialized: now 1786474648621714 us; error 78 us; skew 500 ppm
I20260811 18:57:28.622665 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet f13a14bd64704798bf6de4fc05201a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.622824 4863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:28.626865 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d721b0e55384d8a8efd88a0e47b1504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.630270 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet f13a14bd64704798bf6de4fc05201a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.630837 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0e443ab84424dac307b745b496986. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.634922 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:28.635035 4163 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.635473 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.635658 4162 tablet_bootstrap.cc:654] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.636273 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.637912 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0a164081b1349d9bc89f20d1ea6425d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.639496 4167 ts_tablet_manager.cc:1918] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.639680 4167 log.cc:1196] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ff1be026ca404eada53f60f257ea367c
I20260811 18:57:28.640103 4167 ts_tablet_manager.cc:1939] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.640707 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 600) took 3797 ms. Trace:
I20260811 18:57:28.640810 4167 rpcz_store.cc:276] 0811 18:57:24.843433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.843499 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:28.354601 (+3511102us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:28.354756 (+ 155us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.555561 (+200805us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.639357 (+ 83796us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.640684 (+ 1327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3498185}
I20260811 18:57:28.641321 4163 tablet_bootstrap.cc:654] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.642303 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 511) took 6304 ms. Trace:
I20260811 18:57:28.650771 4169 rpcz_store.cc:276] 0811 18:57:22.337691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.337768 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:28.328896 (+5991128us) ts_tablet_manager.cc:1145] Deleting tablet e51a08e6dbf04e3fa58689ee0cd45db3
0811 18:57:28.328913 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.551583 (+222670us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.603364 (+ 51781us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.642279 (+ 38915us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5990699}
I20260811 18:57:28.651444 4169 tablet_replica.cc:330] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.651706 4169 raft_consensus.cc:2229] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.652014 4169 raft_consensus.cc:2258] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.652740 4169 ts_tablet_manager.cc:1905] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.654876 4162 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.655042 4162 ts_tablet_manager.cc:1397] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.002s
I20260811 18:57:28.657001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 601) took 3802 ms. Trace:
I20260811 18:57:28.657122 4167 rpcz_store.cc:276] 0811 18:57:24.854225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.854284 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:28.656857 (+3802573us) ts_tablet_manager.cc:1145] Deleting tablet 0190cbd859274fcfa09c6fab899c4843
0811 18:57:28.656872 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.656975 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3800180}
I20260811 18:57:28.658166 4162 raft_consensus.cc:348] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.660053 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34c1bb6042d54870bb236a2204ce71ed" } new_table_name: "default.Table_56b89b2b2c4b45768b2866c74d9f1390" modify_external_catalogs: true
I20260811 18:57:28.660315 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 13788f40714847ff8ce6a810ac53e398 (DEFAULT_TABLE table=default.Table_0df46efea56e41b79fbb3e4dd034f84e [id=77dfe9e4685a4c02b9b440096e180d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.660845 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13788f40714847ff8ce6a810ac53e398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.661358 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 630) took 3311 ms. Trace:
I20260811 18:57:28.662556 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e51a08e6dbf04e3fa58689ee0cd45db3 (table default.Table_acb61e38f2a0442688865698892cc5c9 [id=3c7daa88956747719fb78e9860e7039f]) successfully deleted
I20260811 18:57:28.663112 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fcb655a52f4449839a4919d317bc6cfe" } new_table_name: "default.Table_e5769978bc1148a7a53c0df338266d1e" modify_external_catalogs: true
I20260811 18:57:28.663717 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 (DEFAULT_TABLE table=default.Table_c2053e3676584227a3ea13c4f912e315 [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.664088 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4451cc4d7c5940cdb41c021ae486a5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.646986 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ff1be026ca404eada53f60f257ea367c (table default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]) successfully deleted
I20260811 18:57:28.666389 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 482c2c39beb14880b40a600214c5efe4 (DEFAULT_TABLE table=default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.666783 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482c2c39beb14880b40a600214c5efe4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.667204 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0190cbd859274fcfa09c6fab899c4843 because the tablet was not found. No further retry: Not found: Tablet not found: 0190cbd859274fcfa09c6fab899c4843
I20260811 18:57:28.668124 3859 tablet_service.cc:1467] Processing CreateTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e (DEFAULT_TABLE table=default.Table_1bea4820fb5f4e65acb2a1a36332ac2d [id=55e75293fccb4dcfb18d24be0b92d4de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.668557 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb54beb34a654fbdb9c7632901bb0b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.673729 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1e36da0b2c374304a201a3972d4762f2 (DEFAULT_TABLE table=default.Table_3453632ca8274e459b096036508fbb8e [id=b9048b08f3a54c4787533497eb63ca95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.658412 4162 raft_consensus.cc:374] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.674633 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 99bb8e934e6c44ca88640715b11e6bef (DEFAULT_TABLE table=default.Table_fcb655a52f4449839a4919d317bc6cfe [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.675024 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99bb8e934e6c44ca88640715b11e6bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.667714 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 1e36da0b2c374304a201a3972d4762f2 (DEFAULT_TABLE table=default.Table_3453632ca8274e459b096036508fbb8e [id=b9048b08f3a54c4787533497eb63ca95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.667013 3724 tablet_service.cc:1467] Processing CreateTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e (DEFAULT_TABLE table=default.Table_1bea4820fb5f4e65acb2a1a36332ac2d [id=55e75293fccb4dcfb18d24be0b92d4de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.676054 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e36da0b2c374304a201a3972d4762f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.676496 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb54beb34a654fbdb9c7632901bb0b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.676410 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" } modify_external_catalogs: true
I20260811 18:57:28.676993 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495c9c518a6d4462b9983c0f74a02098" } modify_external_catalogs: true
I20260811 18:57:28.667749 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 482c2c39beb14880b40a600214c5efe4 (DEFAULT_TABLE table=default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.678879 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482c2c39beb14880b40a600214c5efe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.681768 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e36da0b2c374304a201a3972d4762f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.682739 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4096c2e6bc426b97dde27e85fec544 (DEFAULT_TABLE table=default.Table_e1f25b2427bf4b23b84761e4fb0dba55 [id=892d56d389c44374afdb0d90eeb5ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.683095 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4096c2e6bc426b97dde27e85fec544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.668084 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 99bb8e934e6c44ca88640715b11e6bef (DEFAULT_TABLE table=default.Table_fcb655a52f4449839a4919d317bc6cfe [id=07bc4a05b4fb49a6883635270832afe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.690050 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99bb8e934e6c44ca88640715b11e6bef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:28.693780 3678 leader_election.cc:334] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.172s (SENT)
I20260811 18:57:28.693895 3678 leader_election.cc:302] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:28.694067 3678 raft_consensus.cc:2670] T f8eb041f3d704762868a048faec0e0fc P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:28.668061 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 (DEFAULT_TABLE table=default.Table_c2053e3676584227a3ea13c4f912e315 [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.694869 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4451cc4d7c5940cdb41c021ae486a5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.672060 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 13788f40714847ff8ce6a810ac53e398 (DEFAULT_TABLE table=default.Table_0df46efea56e41b79fbb3e4dd034f84e [id=77dfe9e4685a4c02b9b440096e180d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.696524 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13788f40714847ff8ce6a810ac53e398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.698038 5161 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.698203 5161 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.674434 4162 raft_consensus.cc:729] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.699034 5161 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.704144 4163 tablet_bootstrap.cc:492] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.704352 4163 ts_tablet_manager.cc:1397] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.006s
I20260811 18:57:28.705265 4163 raft_consensus.cc:348] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.705488 4163 raft_consensus.cc:374] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.705565 4163 raft_consensus.cc:729] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.667412 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 4b4096c2e6bc426b97dde27e85fec544 (DEFAULT_TABLE table=default.Table_e1f25b2427bf4b23b84761e4fb0dba55 [id=892d56d389c44374afdb0d90eeb5ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.705837 4163 consensus_queue.cc:260] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.706427 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b4096c2e6bc426b97dde27e85fec544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.707847 4163 ts_tablet_manager.cc:1428] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:28.708132 4163 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.709568 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 75941ddbffc24f9bb8ba6ed2e6682eae (DEFAULT_TABLE table=default.Table_1af253ec487e4e30b462dc9cb3b1b3cb [id=129d3800251c4ce78db2dc60da27a88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.710009 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75941ddbffc24f9bb8ba6ed2e6682eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.710235 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6dd21c795924a47a8a2435b5de1afb7" } modify_external_catalogs: true
I20260811 18:57:28.710649 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6dd21c795924a47a8a2435b5de1afb7" } modify_external_catalogs: true
I20260811 18:57:28.712396 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0" } modify_external_catalogs: true
I20260811 18:57:28.712584 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_47326d9f3fac4cb7b76516843cf7bfd0" } modify_external_catalogs: true
I20260811 18:57:28.705183 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.714542 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
I20260811 18:57:28.714597 3743 raft_consensus.cc:2454] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.715004 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 deleted and replaced by tablet 8319453f68a54da9813747152ef08756
W20260811 18:57:28.719133 3678 leader_election.cc:334] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.520s (SENT)
I20260811 18:57:28.719770 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 with delete_type TABLET_DATA_DELETED (replaced by tablet 8319453f68a54da9813747152ef08756) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.720270 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" } modify_external_catalogs: true
I20260811 18:57:28.720803 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 75941ddbffc24f9bb8ba6ed2e6682eae (DEFAULT_TABLE table=default.Table_1af253ec487e4e30b462dc9cb3b1b3cb [id=129d3800251c4ce78db2dc60da27a88e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.720880 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 with delete_type TABLET_DATA_DELETED (replaced by tablet 8319453f68a54da9813747152ef08756) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.721339 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75941ddbffc24f9bb8ba6ed2e6682eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.720674 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d84e9ae317424a768d54ab23dc68d514" } modify_external_catalogs: true
I20260811 18:57:28.725446 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.726120 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44c67a0eb7485fb8dcde09416cd331 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:28.726619 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b485cd94aa7c4642a0b3d0021cef63c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.727041 3824 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.727322 5161 raft_consensus.cc:2781] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.727401 5161 raft_consensus.cc:482] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.727460 5161 raft_consensus.cc:3037] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.727677 4863 webserver.cc:460] Webserver started at http://127.3.72.126:40953/ using document root <none> and password file <none>
I20260811 18:57:28.728945 4863 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:28.729277 4863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:28.729573 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2be35bff506498281ab88f9cbb55de8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.730836 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2be35bff506498281ab88f9cbb55de8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.730522 5161 raft_consensus.cc:504] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.731411 5161 leader_election.cc:288] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.737181 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad424c8843ff434ba0449a86849bb5a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.737457 3743 raft_consensus.cc:3037] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.740108 4163 tablet_bootstrap.cc:654] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.740437 3743 raft_consensus.cc:2454] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.741312 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44c67a0eb7485fb8dcde09416cd331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.743644 5161 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.744172 5161 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.744920 5161 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.745724 4169 ts_tablet_manager.cc:1918] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.745997 4169 log.cc:1196] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/125757b8c1a6448b841c63339f392339
I20260811 18:57:28.755154 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6ec419ed7747c3b24dc11992518e88" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.755666 3743 raft_consensus.cc:2454] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.662009 4167 rpcz_store.cc:276] 0811 18:57:25.349548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.349661 (+ 113us) service_pool.cc:227] Handling call
0811 18:57:28.661222 (+3311561us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:28.661235 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.661341 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3311268}
W20260811 18:57:28.756996 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 221d0d8d69f142a29ec7065a82b3f769 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 221d0d8d69f142a29ec7065a82b3f769 already in progress: creating tablet
I20260811 18:57:28.757097 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 631) took 3407 ms. Trace:
I20260811 18:57:28.757242 4167 rpcz_store.cc:276] 0811 18:57:25.349709 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.350144 (+ 435us) service_pool.cc:227] Handling call
0811 18:57:28.756871 (+3406727us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:28.756898 (+ 27us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.757066 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3402604}
W20260811 18:57:28.757669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 68a7051bc539468aafba30fbaf0669d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68a7051bc539468aafba30fbaf0669d4 already in progress: creating tablet
I20260811 18:57:28.698938 4162 consensus_queue.cc:260] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.757957 4169 ts_tablet_manager.cc:1939] T 125757b8c1a6448b841c63339f392339 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.762791 4162 ts_tablet_manager.cc:1428] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.107s user 0.002s sys 0.000s
I20260811 18:57:28.761881 4163 tablet_bootstrap.cc:492] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.763283 4163 ts_tablet_manager.cc:1397] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.003s
W20260811 18:57:28.763541 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1af33fd98c6486f975368ec90493db7 consensus state has no leader
I20260811 18:57:28.763967 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 629) took 3414 ms. Trace:
I20260811 18:57:28.764055 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 8e120a45539f49bb82ab0fb95eac36f9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.764205 4163 raft_consensus.cc:348] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.764511 4163 raft_consensus.cc:374] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.764578 4163 raft_consensus.cc:729] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.764050 4167 rpcz_store.cc:276] 0811 18:57:25.349275 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.349815 (+ 540us) service_pool.cc:227] Handling call
0811 18:57:28.763842 (+3414027us) ts_tablet_manager.cc:1145] Deleting tablet 380c1f53b3ac4509a09b25a2800028af
0811 18:57:28.763853 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.763949 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3400270}
I20260811 18:57:28.764791 4163 consensus_queue.cc:260] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.765358 4163 ts_tablet_manager.cc:1428] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:28.762823 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 510) took 6425 ms. Trace:
I20260811 18:57:28.765466 4169 rpcz_store.cc:276] 0811 18:57:22.337499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.338669 (+ 1170us) service_pool.cc:227] Handling call
0811 18:57:28.651380 (+6312711us) ts_tablet_manager.cc:1145] Deleting tablet 125757b8c1a6448b841c63339f392339
0811 18:57:28.651395 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.726769 (+ 75374us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.745621 (+ 18852us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.762798 (+ 17177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6312276}
W20260811 18:57:28.766481 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 380c1f53b3ac4509a09b25a2800028af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 380c1f53b3ac4509a09b25a2800028af already in progress: creating tablet
I20260811 18:57:28.766597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 512) took 6428 ms. Trace:
I20260811 18:57:28.766678 4169 rpcz_store.cc:276] 0811 18:57:22.338611 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.339144 (+ 533us) service_pool.cc:227] Handling call
0811 18:57:28.766095 (+6426951us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:28.766375 (+ 280us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.766577 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6426592}
I20260811 18:57:28.767895 4163 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.756309 3824 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.768627 5161 raft_consensus.cc:2781] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.768889 5161 raft_consensus.cc:686] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.769186 5161 consensus_queue.cc:237] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.770511 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99bb8e934e6c44ca88640715b11e6bef consensus state has no leader
I20260811 18:57:28.770927 3824 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.771147 5182 raft_consensus.cc:2781] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.771209 5182 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.771261 5182 raft_consensus.cc:3037] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.774120 5182 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.775373 5182 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.767496 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 513) took 6427 ms. Trace:
I20260811 18:57:28.775663 4169 rpcz_store.cc:276] 0811 18:57:22.339811 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.339866 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:28.767282 (+6427416us) ts_tablet_manager.cc:1145] Deleting tablet e14f8717bcd94d6d88295fd8be130ca5
0811 18:57:28.767292 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.767478 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6427012}
I20260811 18:57:28.776190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 517) took 6381 ms. Trace:
I20260811 18:57:28.776353 4169 rpcz_store.cc:276] 0811 18:57:22.394591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.394654 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:28.776011 (+6381357us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:28.776024 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.776169 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6379644}
I20260811 18:57:28.776858 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 518) took 6351 ms. Trace:
I20260811 18:57:28.777002 4169 rpcz_store.cc:276] 0811 18:57:22.424934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.424985 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:28.776738 (+6351753us) ts_tablet_manager.cc:1145] Deleting tablet c6e9da8dfb4a4c6c9275c092fae7d910
0811 18:57:28.776751 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.776837 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6328926}
I20260811 18:57:28.777352 4169 tablet_replica.cc:330] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.777562 4162 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:28.777611 4169 raft_consensus.cc:2229] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.777933 4169 raft_consensus.cc:2258] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.778620 4169 ts_tablet_manager.cc:1905] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.779382 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 632) took 3416 ms. Trace:
I20260811 18:57:28.779639 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b6ec419ed7747c3b24dc11992518e88" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.779930 3743 raft_consensus.cc:3037] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:28.781221 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3263da75f0d84528b7c8cc55dcaeaa46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3263da75f0d84528b7c8cc55dcaeaa46 already in progress: creating tablet
I20260811 18:57:28.783058 3743 raft_consensus.cc:2454] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.783555 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ce5b9945b46348149fb08dafb33f61b2" } new_table_name: "default.Table_5b05303fcd96430bb6ab896fbaa5097a" modify_external_catalogs: true
I20260811 18:57:28.786015 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0df46efea56e41b79fbb3e4dd034f84e" } new_table_name: "default.Table_62d056bd90b1458fb8d4452078546e14" modify_external_catalogs: true
W20260811 18:57:28.786667 3678 leader_election.cc:334] T e7259cf84f5c4c89a1b4510f743c2bc5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.499s (SENT)
I20260811 18:57:28.788444 4863 fs_manager.cc:709] Time spent opening directory manager: real 0.054s user 0.002s sys 0.003s
W20260811 18:57:28.789991 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a847581412494e34b2c76e605ff11772 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a847581412494e34b2c76e605ff11772 already in progress: creating tablet
W20260811 18:57:28.790575 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c6e9da8dfb4a4c6c9275c092fae7d910 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6e9da8dfb4a4c6c9275c092fae7d910 already in progress: creating tablet
#7 0x7fc7c23e62e6 in kudu::tserver::TSTabletManager::DeleteTablet(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, kudu::tserver::TabletServerErrorPB_Code*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1194:12
#8 0x7fc7c2418404 in kudu::tserver::DeleteTabletRunnable::Run() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1101:36
W20260811 18:57:28.792582 3678 leader_election.cc:334] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.300s (SENT)
I20260811 18:57:28.792719 3555 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c] alter complete (version 1)
I20260811 18:57:28.792989 4162 tablet_bootstrap.cc:654] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.779489 4167 rpcz_store.cc:276] 0811 18:57:25.363395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.367150 (+ 3755us) service_pool.cc:227] Handling call
0811 18:57:28.778401 (+3411251us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:28.778419 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.779362 (+ 943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3406203}
I20260811 18:57:28.791149 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b8f2a97c0f46ea8c56b3de2b97e60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.791711 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet ad424c8843ff434ba0449a86849bb5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.791983 3824 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.795425 5080 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.796821 4163 tablet_bootstrap.cc:654] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.797015 5182 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.797600 5182 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.797516 5080 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.798142 5080 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.798131 5182 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.798457 5182 raft_consensus.cc:2781] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.798559 5182 raft_consensus.cc:686] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.798803 5182 consensus_queue.cc:237] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.801456 3678 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.387s (SENT)
I20260811 18:57:28.803088 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.803505 3743 raft_consensus.cc:2454] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:28.804580 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:28.804925 3743 raft_consensus.cc:2454] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:28.805343 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 45fde25bcb40493aafb083e161aa6932 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45fde25bcb40493aafb083e161aa6932 already in progress: creating tablet
I20260811 18:57:28.805449 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.805456 3824 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.805619 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 633) took 3167 ms. Trace:
I20260811 18:57:28.805747 4167 rpcz_store.cc:276] 0811 18:57:25.637953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.638210 (+ 257us) service_pool.cc:227] Handling call
0811 18:57:28.805515 (+3167305us) ts_tablet_manager.cc:1145] Deleting tablet 33d995d72f8142afaeb57a2c29910fa3
0811 18:57:28.805528 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.805601 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3167025}
I20260811 18:57:28.806327 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44c67a0eb7485fb8dcde09416cd331 (DEFAULT_TABLE table=default.Table_ebb36a4f14a0491692aec7b72295b80c [id=3c3b71fd2a9646588c77b15f600b7857]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:28.807291 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33d995d72f8142afaeb57a2c29910fa3 because the tablet was not found. No further retry: Not found: Tablet not found: 33d995d72f8142afaeb57a2c29910fa3
I20260811 18:57:28.808219 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f89f09e89fbd4af7961d8db040ce3f49 (DEFAULT_TABLE table=default.Table_ea791cf775eb426ba289b61dfbaf8b0f [id=a8159550bbb94c04a3ebc52c05610daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.808866 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44c67a0eb7485fb8dcde09416cd331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.809684 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89f09e89fbd4af7961d8db040ce3f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.810861 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:28.811075 5182 raft_consensus.cc:2781] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.811254 5182 raft_consensus.cc:482] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.811385 5182 raft_consensus.cc:3037] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.814401 5182 raft_consensus.cc:504] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.815196 5182 leader_election.cc:288] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.815541 3824 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
#9 0x7fc7c23f9870 in kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&)::$_16::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1121:67
I20260811 18:57:28.817701 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b8f2a97c0f46ea8c56b3de2b97e60c" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.818015 3743 raft_consensus.cc:3037] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.809237 4167 tablet_replica.cc:330] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:28.818585 4167 raft_consensus.cc:2229] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:28.818908 4167 raft_consensus.cc:2258] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.819531 4167 ts_tablet_manager.cc:1905] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.821574 3743 raft_consensus.cc:2454] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.822129 5182 raft_consensus.cc:2781] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:28.822324 5182 raft_consensus.cc:482] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:28.822503 5182 raft_consensus.cc:3037] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.823189 5205 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.823320 5205 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.823907 5205 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.824807 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.828135 3676 leader_election.cc:341] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.828768 3854 tablet_service.cc:1467] Processing CreateTablet for tablet f89f09e89fbd4af7961d8db040ce3f49 (DEFAULT_TABLE table=default.Table_ea791cf775eb426ba289b61dfbaf8b0f [id=a8159550bbb94c04a3ebc52c05610daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.829241 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89f09e89fbd4af7961d8db040ce3f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.831030 3724 tablet_service.cc:1467] Processing CreateTablet for tablet ae75d6652c704e1995966c6dd4c06307 (DEFAULT_TABLE table=default.Table_706c784347b146089c032d699f6a9abe [id=2565a2c170c24cd592a1360a4491287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.831503 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae75d6652c704e1995966c6dd4c06307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:28.836259 5091 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.836433 5091 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:28.837069 5091 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:28.838523 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:28.840711 3676 leader_election.cc:341] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.842167 3854 tablet_service.cc:1467] Processing CreateTablet for tablet ae75d6652c704e1995966c6dd4c06307 (DEFAULT_TABLE table=default.Table_706c784347b146089c032d699f6a9abe [id=2565a2c170c24cd592a1360a4491287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:28.842778 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae75d6652c704e1995966c6dd4c06307. 1 dirs total, 0 dirs full, 0 dirs failed
#10 0x7fc7c23f95fd in std::_Function_handler<void (), kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&)::$_16>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:28.844864 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 515d5b5afbd74ec5ab90a503aef2f4fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
#11 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:28.847630 4162 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.847816 4162 ts_tablet_manager.cc:1397] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.085s user 0.005s sys 0.000s
W20260811 18:57:28.850020 3953 kernel_stack_watchdog.cc:198] Thread 4036 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21977ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be15abaf kudu::log::Log::AppendThread::Wake() at ??:0
@ 0x7fc7be1679ec kudu::log::Log::AsyncAppend() at ??:0
@ 0x7fc7be1683c5 kudu::log::Log::AsyncAppendReplicates() at ??:0
@ 0x7fc7be73585d kudu::consensus::LogCache::AppendOperations() at ??:0
@ 0x7fc7be6d866b kudu::consensus::PeerMessageQueue::AppendOperations() at ??:0
@ 0x7fc7be798393 kudu::consensus::RaftConsensus::UpdateReplica() at ??:0
I20260811 18:57:28.851157 3824 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.851430 5080 raft_consensus.cc:2781] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.851558 5080 raft_consensus.cc:686] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:28.851548 4162 raft_consensus.cc:348] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.851851 4162 raft_consensus.cc:374] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.851917 4162 raft_consensus.cc:729] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.851828 5080 consensus_queue.cc:237] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:28.855451 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (table default.Table_f5f50e8f17e04760a61e10c54d1aa657 [id=de9b6daef0ed416aa79b0dc0f4b02c2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:28.859315 5161 consensus_queue.cc:1035] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:28.860073 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_598311ac8416435286cfa4a6301380bd" } modify_external_catalogs: true
I20260811 18:57:28.860401 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_598311ac8416435286cfa4a6301380bd" } modify_external_catalogs: true
I20260811 18:57:28.871012 4163 tablet_bootstrap.cc:492] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.871227 4163 ts_tablet_manager.cc:1397] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.103s user 0.001s sys 0.008s
I20260811 18:57:28.872603 4163 raft_consensus.cc:348] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.872888 4163 raft_consensus.cc:374] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.872953 4163 raft_consensus.cc:729] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.878298 4162 consensus_queue.cc:260] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.886754 4162 ts_tablet_manager.cc:1428] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.036s user 0.000s sys 0.002s
I20260811 18:57:28.887105 4162 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:28.888146 3678 leader_election.cc:334] T 33d995d72f8142afaeb57a2c29910fa3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.260s (SENT)
I20260811 18:57:28.872977 5182 raft_consensus.cc:504] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.896965 5182 leader_election.cc:288] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:28.899442 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9efd2f19d74a68a8536fd057377738" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:28.899820 3743 raft_consensus.cc:3037] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:28.900085 4163 consensus_queue.cc:260] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.904639 4163 ts_tablet_manager.cc:1428] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.033s user 0.001s sys 0.002s
I20260811 18:57:28.904824 3743 raft_consensus.cc:2454] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:28.907464 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b203df4124364737839bf4c1ae072988
I20260811 18:57:28.908617 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.909372 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.910748 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:28.911253 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7df549360f014d629938e3e94659af8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:28.911815 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e2635923a5784be1839e861c2277e900"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.912487 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2635923a5784be1839e861c2277e900 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:28.912730 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7df549360f014d629938e3e94659af8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.914157 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b4dc2df4c6464d71bb13fe25e0046758
I20260811 18:57:28.914942 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_256aa74bb2884a0d83f0d17b159b9e49" } new_table_name: "default.Table_8518b7d355d14d14983f2c89aad83f5e" modify_external_catalogs: true
I20260811 18:57:28.915558 4163 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:28.915810 5230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:28.916896 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.917783 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:28.919355 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8d6cbc9a8ed4b798188572c6fc1ff52 has no consensus state
I20260811 18:57:28.922254 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2474ae972e9242b69f488b7a2fdbc481
I20260811 18:57:28.922989 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923457 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923862 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.923887 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8e120a45539f49bb82ab0fb95eac36f9
I20260811 18:57:28.924427 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 8e120a45539f49bb82ab0fb95eac36f9 deleted and replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1
I20260811 18:57:28.924455 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:28.883148 4169 ts_tablet_manager.cc:1918] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.924871 4169 log.cc:1196] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/494939a4eb5a45d6a123d174ad513bcb
I20260811 18:57:28.925436 4169 ts_tablet_manager.cc:1939] T 494939a4eb5a45d6a123d174ad513bcb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:28.926457 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 521) took 6457 ms. Trace:
I20260811 18:57:28.926585 4169 rpcz_store.cc:276] 0811 18:57:22.469353 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.469444 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:28.777312 (+6307868us) ts_tablet_manager.cc:1145] Deleting tablet 494939a4eb5a45d6a123d174ad513bcb
0811 18:57:28.777321 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.809008 (+ 31687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.883043 (+ 74035us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.926437 (+ 43394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6277776}
I20260811 18:57:28.927062 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 534) took 6235 ms. Trace:
I20260811 18:57:28.927377 4169 rpcz_store.cc:276] 0811 18:57:22.746564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.746706 (+ 142us) service_pool.cc:227] Handling call
0811 18:57:28.926914 (+6180208us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:28.926927 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.927045 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6120374}
I20260811 18:57:28.927748 4169 tablet_replica.cc:330] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:28.928675 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 494939a4eb5a45d6a123d174ad513bcb (table default.Table_815d74faaf044b2f9000790283e989c9 [id=8442ea619ff14702bac1136eb42a84fc]) successfully deleted
I20260811 18:57:28.928959 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 with delete_type TABLET_DATA_DELETED (replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:28.929080 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 452ee450778e4d33adfa7e2745b5d970 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 452ee450778e4d33adfa7e2745b5d970 already in progress: creating tablet
I20260811 18:57:28.930408 4163 tablet_bootstrap.cc:654] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.932960 4863 fs_manager.cc:725] Time spent opening block manager: real 0.068s user 0.002s sys 0.000s
I20260811 18:57:28.933257 4863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "7cb005c681cf42d4ac278ec73a28e4a9"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-zpfg"
I20260811 18:57:28.936452 4863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:28.937070 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet c77882ff8d974057a61f037f1943f58f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:28.912886 4162 tablet_bootstrap.cc:654] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:28.913304 4167 ts_tablet_manager.cc:1918] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:28.922574 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba6246e6d29418cbca5f751b34f4ef1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939458 4167 log.cc:1196] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:28.924157 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 2474ae972e9242b69f488b7a2fdbc481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.924297 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.924727 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.925670 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e120a45539f49bb82ab0fb95eac36f9 with delete_type TABLET_DATA_DELETED (replaced by tablet e7b6f913d82c4ea9830e6e680de3dda1) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939951 4167 ts_tablet_manager.cc:1939] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:28.929749 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:28.939879 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3982bc659a664e949beb16b472873d9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.941110 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3982bc659a664e949beb16b472873d9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.941767 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 634) took 3303 ms. Trace:
I20260811 18:57:28.941874 4167 rpcz_store.cc:276] 0811 18:57:25.638128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.638541 (+ 413us) service_pool.cc:227] Handling call
0811 18:57:28.809182 (+3170641us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:28.809194 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.855148 (+ 45954us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.911722 (+ 56574us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.941741 (+ 30019us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170440}
I20260811 18:57:28.942179 4163 tablet_bootstrap.cc:492] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:28.942584 5205 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:28.942606 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 460e03cc14444c83b6c08ca194ca5a8d (table default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]) successfully deleted
I20260811 18:57:28.942730 5205 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.943325 5205 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:28.943527 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f7a8c696f8df4bec92e19adef1eff6d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:28.944335 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7a8c696f8df4bec92e19adef1eff6d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:28.944514 3824 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:28.944515 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:28.945557 5229 raft_consensus.cc:2781] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:28.945770 5229 raft_consensus.cc:686] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
W20260811 18:57:28.945801 3676 leader_election.cc:341] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:28.946512 5229 consensus_queue.cc:237] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.947901 4162 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.948071 4162 ts_tablet_manager.cc:1397] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.003s
I20260811 18:57:28.948932 4162 raft_consensus.cc:348] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.949183 4162 raft_consensus.cc:374] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.949281 4162 raft_consensus.cc:729] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.949515 4162 consensus_queue.cc:260] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.950217 4169 raft_consensus.cc:2229] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:28.951763 4169 raft_consensus.cc:2258] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:28.952643 4169 ts_tablet_manager.cc:1905] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:28.942320 4163 ts_tablet_manager.cc:1397] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.004s
I20260811 18:57:28.955943 4163 raft_consensus.cc:348] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#12 0x7fc7b32c1344 in kudu::ThreadPool::DispatchThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:776:7
#13 0x7fc7b32c491b in kudu::ThreadPool::CreateThread()::$_2::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:849:48
#14 0x7fc7b32c4798 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
previously allocated by thread T135 (tablet-open [wo) here:
I20260811 18:57:28.970868 4162 ts_tablet_manager.cc:1428] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
#0 0x8c3cc8 in operator new(unsigned long) /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99:3
#1 0x7fc7bf9f51a5 in kudu::tablet::TabletBootstrap::OpenTablet(bool*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:693:29
#2 0x7fc7bf9f35ef in kudu::tablet::TabletBootstrap::RunBootstrap(std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:644:3
#3 0x7fc7bf9f1fd3 in kudu::tablet::TabletBootstrap::Bootstrap(std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:579:29
#4 0x7fc7bf9f18ac in kudu::tablet::BootstrapTablet(scoped_refptr<kudu::tablet::TabletMetadata>, kudu::consensus::RaftConfigPB, kudu::clock::Clock*, std::shared_ptr<kudu::MemTracker>, scoped_refptr<kudu::rpc::ResultTracker>, kudu::MetricRegistry*, kudu::FileCache*, scoped_refptr<kudu::tablet::TabletReplica>, scoped_refptr<kudu::log::LogAnchorRegistry>, std::shared_ptr<kudu::tablet::Tablet>*, scoped_refptr<kudu::log::Log>*, kudu::consensus::ConsensusBootstrapInfo*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet_bootstrap.cc:521:3
I20260811 18:57:28.977516 4162 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
#5 0x7fc7c23e19a6 in kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr<kudu::tablet::TabletReplica> const&, scoped_refptr<kudu::tserver::TransitionInProgressDeleter> const&, std::atomic<int>*, std::atomic<int>*, kudu::Timer*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1406:9
#6 0x7fc7c23f814a in kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:703:3
I20260811 18:57:28.981016 4162 tablet_bootstrap.cc:654] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#7 0x7fc7c23f7e8d in std::_Function_handler<void (), kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#8 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#9 0x7fc7b32c1344 in kudu::ThreadPool::DispatchThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:776:7
#10 0x7fc7b32c491b in kudu::ThreadPool::CreateThread()::$_2::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:849:48
#11 0x7fc7b32c4798 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#12 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#13 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#14 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Thread T54 (rpc worker-4005) created by T0 here:
W20260811 18:57:28.995182 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13788f40714847ff8ce6a810ac53e398 consensus state has no leader
I20260811 18:57:28.998127 4162 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:28.998306 4162 ts_tablet_manager.cc:1397] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20260811 18:57:28.999526 4162 raft_consensus.cc:348] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:28.999764 4162 raft_consensus.cc:374] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:28.999819 4162 raft_consensus.cc:729] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:28.978933 4169 ts_tablet_manager.cc:1918] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260811 18:57:28.991380 3824 leader_election.cc:334] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.047s (SENT)
I20260811 18:57:29.001914 3824 leader_election.cc:302] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:29.002250 3824 raft_consensus.cc:2670] T 36724a83bf9844b9a27a88ca36a9bf50 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:29.002987 3824 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.612s (SENT)
I20260811 18:57:29.003110 3824 leader_election.cc:302] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:29.003259 3824 raft_consensus.cc:2670] T 3744b3d608754de68c4c57b072d1dc15 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:29.005879 4169 log.cc:1196] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e5fd9fce095e4a769dbb38c7903359cc
I20260811 18:57:29.006523 4169 ts_tablet_manager.cc:1939] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.007225 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 535) took 6202 ms. Trace:
I20260811 18:57:29.007426 4169 rpcz_store.cc:276] 0811 18:57:22.804771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.804843 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:28.927704 (+6122861us) ts_tablet_manager.cc:1145] Deleting tablet e5fd9fce095e4a769dbb38c7903359cc
0811 18:57:28.927716 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:28.960512 (+ 32796us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:28.978811 (+ 18299us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.007203 (+ 28392us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6048702}
I20260811 18:57:29.007872 4169 tablet_replica.cc:330] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.008383 4169 raft_consensus.cc:2229] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.008345 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e5fd9fce095e4a769dbb38c7903359cc (table default.Table_2664a3dc787a4a4d98a334656bc584f3 [id=b3ed273027a946aab2041419fc3c11e2]) successfully deleted
I20260811 18:57:29.008805 4169 raft_consensus.cc:2258] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.009579 4169 ts_tablet_manager.cc:1905] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.010089 4162 consensus_queue.cc:260] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.009789 5229 consensus_queue.cc:1035] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:29.013443 4162 ts_tablet_manager.cc:1428] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260811 18:57:29.013652 4163 raft_consensus.cc:374] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.013829 4163 raft_consensus.cc:729] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.018232 4162 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.027741 4162 tablet_bootstrap.cc:654] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.014163 4163 consensus_queue.cc:260] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.033759 4163 ts_tablet_manager.cc:1428] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.079s user 0.000s sys 0.002s
I20260811 18:57:29.048545 4163 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.049328 4162 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.049485 4162 ts_tablet_manager.cc:1397] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260811 18:57:29.050237 4162 raft_consensus.cc:348] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.050488 4162 raft_consensus.cc:374] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.050545 4162 raft_consensus.cc:729] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.050781 4162 consensus_queue.cc:260] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.052045 4162 ts_tablet_manager.cc:1428] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:29.052783 3824 consensus_peers.cc:487] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff1be026ca404eada53f60f257ea367c. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:29.054263 4162 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.055924 4162 tablet_bootstrap.cc:654] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.059952 4163 tablet_bootstrap.cc:654] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:29.062971 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114d22f22d3d4dc09563aa1108e1f509 consensus state has no leader
I20260811 18:57:29.064560 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c77882ff8d974057a61f037f1943f58f
I20260811 18:57:29.066404 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet c77882ff8d974057a61f037f1943f58f with delete_type TABLET_DATA_DELETED (replaced by tablet c0f68d33e1b14144b34c17e3c7201408) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.067569 3554 catalog_manager.cc:6565] ReplaceTablet: tablet c77882ff8d974057a61f037f1943f58f deleted and replaced by tablet c0f68d33e1b14144b34c17e3c7201408
I20260811 18:57:29.067538 4169 ts_tablet_manager.cc:1918] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.067675 4169 log.cc:1196] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/94943cfbe41a47b3a36c81957a3adaeb
I20260811 18:57:29.068087 4169 ts_tablet_manager.cc:1939] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.068794 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet b57ed7814a1e440cba362613cbab32f8 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:29.068696 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 102259us
I20260811 18:57:29.069821 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b012268c924347d8a87ea52350010dc3" } new_table_name: "default.Table_a055754403774bc0a94fda2f957c70d5" modify_external_catalogs: true
I20260811 18:57:29.072072 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a30ff94f4f34c7dbdfa8dc94b2b4a48" } new_table_name: "default.Table_0db19ab70f764e00b17ee2e0f4742b8b" modify_external_catalogs: true
I20260811 18:57:29.072484 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet c77882ff8d974057a61f037f1943f58f with delete_type TABLET_DATA_DELETED (replaced by tablet c0f68d33e1b14144b34c17e3c7201408) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.073624 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 536) took 6214 ms. Trace:
I20260811 18:57:29.073750 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b8dd0414c4784bbea975b8d3f1491cd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.074934 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8dd0414c4784bbea975b8d3f1491cd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.076874 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.077689 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f14f93014a8d464ba0da0bab0ceb9ddc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.077867 5080 consensus_queue.cc:1035] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:29.080885 4162 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.081032 4162 ts_tablet_manager.cc:1397] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260811 18:57:29.083009 4162 raft_consensus.cc:348] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.083235 4162 raft_consensus.cc:374] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.083285 4162 raft_consensus.cc:729] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.083496 4162 consensus_queue.cc:260] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.084473 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 635) took 3331 ms. Trace:
I20260811 18:57:29.084569 4167 rpcz_store.cc:276] 0811 18:57:25.753465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.753547 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:29.074165 (+3320618us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:29.074180 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.084450 (+ 10270us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3320213}
I20260811 18:57:29.085142 4162 ts_tablet_manager.cc:1428] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260811 18:57:29.085278 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8304ac7709ca4392baec4cfa78ef5b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8304ac7709ca4392baec4cfa78ef5b41 already in progress: creating tablet
I20260811 18:57:29.085723 4162 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.087690 4162 tablet_bootstrap.cc:654] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.090449 4167 tablet_replica.cc:330] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.090919 4167 raft_consensus.cc:2229] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.091229 4167 raft_consensus.cc:2258] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.092183 4167 ts_tablet_manager.cc:1905] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.109354 4162 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.109520 4162 ts_tablet_manager.cc:1397] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260811 18:57:29.110265 4162 raft_consensus.cc:348] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.110529 4162 raft_consensus.cc:374] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.110590 4162 raft_consensus.cc:729] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.110785 4162 consensus_queue.cc:260] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.112239 4162 ts_tablet_manager.cc:1428] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:29.117802 4162 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.121546 4162 tablet_bootstrap.cc:654] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.123848 4169 rpcz_store.cc:276] 0811 18:57:22.859061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.859942 (+ 881us) service_pool.cc:227] Handling call
0811 18:57:29.007817 (+6147875us) ts_tablet_manager.cc:1145] Deleting tablet 94943cfbe41a47b3a36c81957a3adaeb
0811 18:57:29.007830 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.018191 (+ 10361us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.067367 (+ 49176us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.073601 (+ 6234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6114238}
I20260811 18:57:29.125335 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 94943cfbe41a47b3a36c81957a3adaeb (table default.Table_cb0316f2e1db4353b9919edf3d8e1c90 [id=4e8ccdf70ea64f41a3f26f73e4f6aed7]) successfully deleted
I20260811 18:57:29.125391 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 538) took 6235 ms. Trace:
I20260811 18:57:29.125494 4169 rpcz_store.cc:276] 0811 18:57:22.906010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:22.906277 (+ 267us) service_pool.cc:227] Handling call
0811 18:57:29.125175 (+6218898us) ts_tablet_manager.cc:1145] Deleting tablet b20872d80c9545e5822bfc82d496ce97
0811 18:57:29.125191 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.125366 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6181656}
W20260811 18:57:29.129118 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b20872d80c9545e5822bfc82d496ce97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b20872d80c9545e5822bfc82d496ce97 already in progress: creating tablet
I20260811 18:57:29.129277 4167 ts_tablet_manager.cc:1918] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:29.129276 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 541) took 6009 ms. Trace:
I20260811 18:57:29.129390 4167 log.cc:1196] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a443fe209556489abb47f4faa5d696e5
I20260811 18:57:29.129374 4169 rpcz_store.cc:276] 0811 18:57:23.119734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.119799 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.129092 (+6009293us) ts_tablet_manager.cc:1145] Deleting tablet 21cc1822d4b94081ad86a6177acb3afe
0811 18:57:29.129118 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.129251 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6008748}
I20260811 18:57:29.129779 4167 ts_tablet_manager.cc:1939] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:29.132292 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 21cc1822d4b94081ad86a6177acb3afe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21cc1822d4b94081ad86a6177acb3afe already in progress: creating tablet
I20260811 18:57:29.132417 4169 tablet_replica.cc:330] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.137142 4169 raft_consensus.cc:2229] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.138283 4169 pending_rounds.cc:62] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:29.139191 4169 pending_rounds.cc:68] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400161482723328 op_type: NO_OP noop_request { }
I20260811 18:57:29.140074 4169 raft_consensus.cc:2866] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:29.140559 4169 raft_consensus.cc:2258] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.142393 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddc87769613d45caa57f2df0032dc979"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.143412 4169 ts_tablet_manager.cc:1905] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.143558 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddc87769613d45caa57f2df0032dc979 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.147686 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" } modify_external_catalogs: true
I20260811 18:57:29.148083 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f7e22bcb9094f448ce9d82d4678ad1a" } modify_external_catalogs: true
I20260811 18:57:29.149155 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_068dc550d67d413c916d1384c3cb811c" } modify_external_catalogs: true
I20260811 18:57:29.149509 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_068dc550d67d413c916d1384c3cb811c" } modify_external_catalogs: true
I20260811 18:57:29.150250 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 636) took 3381 ms. Trace:
I20260811 18:57:29.150370 4167 rpcz_store.cc:276] 0811 18:57:25.769033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.769221 (+ 188us) service_pool.cc:227] Handling call
0811 18:57:29.085947 (+3316726us) ts_tablet_manager.cc:1145] Deleting tablet a443fe209556489abb47f4faa5d696e5
0811 18:57:29.085958 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.113159 (+ 27201us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.129158 (+ 15999us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.150223 (+ 21065us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3316432}
W20260811 18:57:29.150534 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c142d8ef40947e29f3c94e1e8e4c517 consensus state has no leader
I20260811 18:57:29.150740 4167 tablet_replica.cc:330] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.150885 5205 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.150939 4167 raft_consensus.cc:2229] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.151013 5205 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.151216 4167 raft_consensus.cc:2258] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.151233 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_647ae82e63464976abb7f26c9526a349" } modify_external_catalogs: true
I20260811 18:57:29.151607 5205 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.151546 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_647ae82e63464976abb7f26c9526a349" } modify_external_catalogs: true
I20260811 18:57:29.151747 4167 ts_tablet_manager.cc:1905] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.152302 4162 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.152446 4162 ts_tablet_manager.cc:1397] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.001s
I20260811 18:57:29.152426 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.152523 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a443fe209556489abb47f4faa5d696e5 (table default.Table_bf28eaf7393d4d508363fbc8dd91686b [id=1a0803466858425cadc33f51f0933a95]) successfully deleted
W20260811 18:57:29.153091 3676 leader_election.cc:341] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.153841 4162 raft_consensus.cc:348] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.154115 4162 raft_consensus.cc:374] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.154191 4162 raft_consensus.cc:729] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.154506 4162 consensus_queue.cc:260] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.157343 4162 ts_tablet_manager.cc:1428] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:29.162781 4162 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.169283 4162 tablet_bootstrap.cc:654] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.176014 4169 ts_tablet_manager.cc:1918] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.187009 4863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:29.189262 4863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:29.189965 4863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:29.190677 4167 ts_tablet_manager.cc:1918] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.190829 4167 log.cc:1196] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:29.191069 4169 log.cc:1196] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:29.191303 4167 ts_tablet_manager.cc:1939] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.192239 4162 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.192473 4162 ts_tablet_manager.cc:1397] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.005s
I20260811 18:57:29.193801 4162 raft_consensus.cc:348] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.194177 4162 raft_consensus.cc:374] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.194319 4162 raft_consensus.cc:729] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.194698 4162 consensus_queue.cc:260] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.195641 4162 ts_tablet_manager.cc:1428] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:29.196269 4162 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.198815 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 637) took 3429 ms. Trace:
I20260811 18:57:29.198932 4167 rpcz_store.cc:276] 0811 18:57:25.769165 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.769582 (+ 417us) service_pool.cc:227] Handling call
0811 18:57:29.150682 (+3381100us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:29.150696 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.170337 (+ 19641us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.190547 (+ 20210us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.198792 (+ 8245us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3380923}
I20260811 18:57:29.199338 4167 tablet_replica.cc:330] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:29.199450 3678 leader_election.cc:334] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.326s (SENT)
I20260811 18:57:29.199571 4167 raft_consensus.cc:2229] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.199749 4167 raft_consensus.cc:2258] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.199786 5205 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.199887 5205 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.200342 4167 ts_tablet_manager.cc:1905] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.200498 5205 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.200703 4162 tablet_bootstrap.cc:654] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.201236 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.201424 4169 ts_tablet_manager.cc:1939] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.202028 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 542) took 6079 ms. Trace:
W20260811 18:57:29.202116 3676 leader_election.cc:341] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.202132 4169 rpcz_store.cc:276] 0811 18:57:23.122710 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.122770 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:29.132355 (+6009585us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:29.132370 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.171893 (+ 39523us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.175872 (+ 3979us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.202003 (+ 26131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6009135}
I20260811 18:57:29.202782 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f61d3822fc2d4b569f223238a6464e0a (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:29.219609 4169 tablet_replica.cc:330] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:29.219951 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3dd4ec30acde4585a1c3451927217cb3 consensus state has no leader
I20260811 18:57:29.220009 4169 raft_consensus.cc:2229] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.220305 4169 raft_consensus.cc:2258] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.220783 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.221144 4169 ts_tablet_manager.cc:1905] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.221642 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.223269 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e425950244124732b380187898a2b68f" } new_table_name: "default.Table_1e5db5512ab043148049e8476163a5a8" modify_external_catalogs: true
I20260811 18:57:29.227041 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c" } modify_external_catalogs: true
I20260811 18:57:29.227342 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bd92de7863b94ff38b2c402c27ddf26c" } modify_external_catalogs: true
I20260811 18:57:29.228309 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b3ee7e916a0c4d2180027e84bc055f54" } new_table_name: "default.Table_5627fd661e4441d8a677d35bc0f16cf0" modify_external_catalogs: true
I20260811 18:57:29.229394 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 4b6ec419ed7747c3b24dc11992518e88 (table default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]) successfully deleted
I20260811 18:57:29.237694 4167 ts_tablet_manager.cc:1918] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.237923 4167 log.cc:1196] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:29.243268 4163 tablet_bootstrap.cc:492] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:29.243554 4163 ts_tablet_manager.cc:1397] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.195s user 0.005s sys 0.017s
I20260811 18:57:29.245180 4163 raft_consensus.cc:348] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.245536 4163 raft_consensus.cc:374] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.245652 4163 raft_consensus.cc:729] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.246059 4163 consensus_queue.cc:260] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.247314 4167 ts_tablet_manager.cc:1939] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.248404 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 638) took 3448 ms. Trace:
I20260811 18:57:29.248570 4167 rpcz_store.cc:276] 0811 18:57:25.799648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.799719 (+ 71us) service_pool.cc:227] Handling call
0811 18:57:29.199284 (+3399565us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:29.199297 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.221998 (+ 22701us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.237522 (+ 15524us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.248375 (+ 10853us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3399170}
I20260811 18:57:29.249244 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:29.249939 4163 ts_tablet_manager.cc:1428] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:29.254923 4167 tablet_replica.cc:330] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.255221 4167 raft_consensus.cc:2229] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:29.255597 4167 raft_consensus.cc:2258] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.256278 4167 ts_tablet_manager.cc:1905] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:29.267633 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50af4e9b20a74ed983b26281c3c6eda1 consensus state has no leader
W20260811 18:57:29.268165 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:29.269268 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 6880ad27a3eb4599b407edeb2969a08f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:29.273221 4163 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.284490 5205 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.284687 5205 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.285526 5205 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.286316 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.287106 3676 leader_election.cc:341] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.302026 4167 ts_tablet_manager.cc:1918] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:29.304296 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_16d7946e87be4700a338a7744c866d5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.305267 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16d7946e87be4700a338a7744c866d5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.307447 4167 log.cc:1196] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:29.307982 4167 ts_tablet_manager.cc:1939] T 01d564ad80064912a8ce8e31c1b6c4d2 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.308498 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a677dcb5a43648a49944a18f38fb74fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.309195 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a677dcb5a43648a49944a18f38fb74fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.310608 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 639) took 3494 ms. Trace:
I20260811 18:57:29.310714 4167 rpcz_store.cc:276] 0811 18:57:25.816103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.816243 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:29.254831 (+3438588us) ts_tablet_manager.cc:1145] Deleting tablet 01d564ad80064912a8ce8e31c1b6c4d2
0811 18:57:29.254847 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.278671 (+ 23824us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.301867 (+ 23196us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.310583 (+ 8716us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3438328}
I20260811 18:57:29.311528 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]) successfully deleted
I20260811 18:57:29.313643 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 640) took 3381 ms. Trace:
I20260811 18:57:29.313755 4167 rpcz_store.cc:276] 0811 18:57:25.932436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.932489 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:29.313456 (+3380967us) ts_tablet_manager.cc:1145] Deleting tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
0811 18:57:29.313469 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.313620 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3380761}
W20260811 18:57:29.314352 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa already in progress: creating tablet
I20260811 18:57:29.317706 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d79d187445ee43d6864f03f905400405" } new_table_name: "default.Table_c499c7d38b65414692ba6f4d7307a7bf" modify_external_catalogs: true
I20260811 18:57:29.318878 4167 tablet_replica.cc:330] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.319128 4167 raft_consensus.cc:2229] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.321127 4167 raft_consensus.cc:2258] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.323468 5205 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.323655 5205 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.324635 5205 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.325335 4167 ts_tablet_manager.cc:1905] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.327394 4162 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.327538 4162 ts_tablet_manager.cc:1397] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.007s
I20260811 18:57:29.330045 3553 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:29.331624 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_32f181a8b481409187abd0dc0a1765ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.332587 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32f181a8b481409187abd0dc0a1765ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.335740 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_944ef43f4d664e43b89a55f61dffcfb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.336534 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_944ef43f4d664e43b89a55f61dffcfb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.337774 3824 consensus_peers.cc:487] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67e63ee9b9fd4b12ae791b9d5bc2e648. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:29.343546 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.346602 3676 leader_election.cc:341] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.330745 4162 raft_consensus.cc:348] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.348469 4162 raft_consensus.cc:374] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.349081 4162 raft_consensus.cc:729] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.349844 4163 tablet_bootstrap.cc:654] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.349654 4162 consensus_queue.cc:260] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.358845 4162 ts_tablet_manager.cc:1428] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.031s user 0.003s sys 0.000s
I20260811 18:57:29.365170 3854 tablet_service.cc:1467] Processing CreateTablet for tablet a3466b2364d64b0cad93b68732e0dae2 (DEFAULT_TABLE table=default.Table_a79b27c0bd524af985c2e6167624551a [id=b46bfa34023d4fdf8d4864dec08ce39b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.365712 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3466b2364d64b0cad93b68732e0dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.372587 4162 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:29.374068 4169 ts_tablet_manager.cc:1918] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.5
I20260811 18:57:29.375492 3708 tablet_service.cc:1467] Processing CreateTablet for tablet a3466b2364d64b0cad93b68732e0dae2 (DEFAULT_TABLE table=default.Table_a79b27c0bd524af985c2e6167624551a [id=b46bfa34023d4fdf8d4864dec08ce39b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.375959 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3466b2364d64b0cad93b68732e0dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.383203 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1ddb10e5ee5b441c875dd0b345201949 (DEFAULT_TABLE table=default.Table_6064366e4ea94146903cb48b739102e9 [id=94bd67a82648467ab2ffb80d4e1b9ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.383852 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ddb10e5ee5b441c875dd0b345201949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.386766 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 5638cefdec734ed58f3de696740ba891 (DEFAULT_TABLE table=default.Table_832e411b94e24eb49b8798f66fa24774 [id=8b596fd1e4af496680f105839144bf79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.387250 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 08f4d8bbe23b463680c153ae4ff4ebcf (DEFAULT_TABLE table=default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a [id=7de6fc7b04824669845b02a164b3a819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.397184 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5638cefdec734ed58f3de696740ba891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.403235 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f4d8bbe23b463680c153ae4ff4ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.404748 5205 raft_consensus.cc:482] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.404934 5205 raft_consensus.cc:504] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.406004 5205 leader_election.cc:288] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:29.407555 4167 ts_tablet_manager.cc:1918] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.407722 4167 log.cc:1196] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:29.408268 4167 ts_tablet_manager.cc:1939] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.408767 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 641) took 3471 ms. Trace:
I20260811 18:57:29.408902 4167 rpcz_store.cc:276] 0811 18:57:25.936853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.937016 (+ 163us) service_pool.cc:227] Handling call
0811 18:57:29.318823 (+3381807us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:29.318836 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.329231 (+ 10395us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.404557 (+ 75326us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.408746 (+ 4189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3381559}
I20260811 18:57:29.409778 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b6f36ab6bb774257b5c2692c64eff63d (table default.Table_e0c0cbe2c2f846c2b85cda84670f448c [id=22529e4acc82410a93bca07dfda7d1b9]) successfully deleted
I20260811 18:57:29.415068 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 642) took 3460 ms. Trace:
I20260811 18:57:29.415233 4167 rpcz_store.cc:276] 0811 18:57:25.954790 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.954855 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.414915 (+3460060us) ts_tablet_manager.cc:1145] Deleting tablet 84e152c74f33463e83843b92c0783803
0811 18:57:29.414929 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.415046 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3459857}
W20260811 18:57:29.415788 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 84e152c74f33463e83843b92c0783803 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e152c74f33463e83843b92c0783803 already in progress: creating tablet
I20260811 18:57:29.417181 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 643) took 3461 ms. Trace:
I20260811 18:57:29.417279 4167 rpcz_store.cc:276] 0811 18:57:25.956113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.956171 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:29.417030 (+3460859us) ts_tablet_manager.cc:1145] Deleting tablet 649cb98f553a418b911e307fdaf49917
0811 18:57:29.417052 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.417150 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1792,"tablet-delete.queue_time_us":3460531}
W20260811 18:57:29.417788 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 649cb98f553a418b911e307fdaf49917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 649cb98f553a418b911e307fdaf49917 already in progress: creating tablet
I20260811 18:57:29.420476 4169 log.cc:1196] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9de9acdb33a2438eb4378a5eac844c35
I20260811 18:57:29.420522 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515d5b5afbd74ec5ab90a503aef2f4fb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:29.420850 3887 raft_consensus.cc:2454] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:29.421278 4169 ts_tablet_manager.cc:1939] T 9de9acdb33a2438eb4378a5eac844c35 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.421417 3676 leader_election.cc:302] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:29.421881 5205 raft_consensus.cc:2781] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:29.421979 5205 raft_consensus.cc:482] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:29.422065 5205 raft_consensus.cc:3037] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:29.425648 5205 raft_consensus.cc:504] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.427031 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515d5b5afbd74ec5ab90a503aef2f4fb" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:29.427371 3887 raft_consensus.cc:3037] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:29.428320 5205 leader_election.cc:288] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:29.430250 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 4149626f48354ba0923cad28b95be048 (DEFAULT_TABLE table=default.Table_5296710afead477eb67f356c9afbc4a4 [id=54cc236881e246a18f2af7ae6df6b983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.430730 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 23c85b2315254e4c8d9d69fec4cc9aa2 (DEFAULT_TABLE table=default.Table_e79b6e054e914fd39ca0a6282eb0ce17 [id=8d92dfcc02424160a65fca84e2417a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.431049 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 72f11748cf984942ba99bda4ad6605ab (DEFAULT_TABLE table=default.Table_c6284ad4471b4b7095b11fe7a8f304e2 [id=53f3832432d242058789ef564beeae06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.429807 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 468c71415a984ea7ab04c21fa9612070 (DEFAULT_TABLE table=default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 [id=4a0a8c18d3114ce4a1c4e69e733d372b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.433403 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 543) took 6179 ms. Trace:
I20260811 18:57:29.433533 4169 rpcz_store.cc:276] 0811 18:57:23.253508 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.255514 (+ 2006us) service_pool.cc:227] Handling call
0811 18:57:29.219545 (+5964031us) ts_tablet_manager.cc:1145] Deleting tablet 9de9acdb33a2438eb4378a5eac844c35
0811 18:57:29.219559 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.335174 (+115615us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.373885 (+ 38711us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.433374 (+ 59489us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5924771}
I20260811 18:57:29.434015 4169 tablet_replica.cc:330] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:29.434311 4169 raft_consensus.cc:2229] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.434549 4169 raft_consensus.cc:2258] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.435293 4169 ts_tablet_manager.cc:1905] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.438956 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 644) took 3474 ms. Trace:
I20260811 18:57:29.439127 4167 rpcz_store.cc:276] 0811 18:57:25.964049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.964374 (+ 325us) service_pool.cc:227] Handling call
0811 18:57:29.419726 (+3455352us) ts_tablet_manager.cc:1145] Deleting tablet 86d690ec09524248bc147ad7408e2ee6
0811 18:57:29.419751 (+ 25us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.438914 (+ 19163us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3455131}
I20260811 18:57:29.440249 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c85b2315254e4c8d9d69fec4cc9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.440640 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468c71415a984ea7ab04c21fa9612070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.446676 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f11748cf984942ba99bda4ad6605ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.450572 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4149626f48354ba0923cad28b95be048. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.457033 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06c8776bb47c4cb891ef13a2a1e36781 consensus state has no leader
I20260811 18:57:29.459256 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 47a782b8470b4d14ac89b3b7d463a125
I20260811 18:57:29.459568 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 645) took 3495 ms. Trace:
I20260811 18:57:29.459734 4167 rpcz_store.cc:276] 0811 18:57:25.964321 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.964617 (+ 296us) service_pool.cc:227] Handling call
0811 18:57:29.459312 (+3494695us) ts_tablet_manager.cc:1145] Deleting tablet 6d28c486e408451a990ae3905ef3c2ff
0811 18:57:29.459335 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.459500 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3494225}
I20260811 18:57:29.460476 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6880ad27a3eb4599b407edeb2969a08f
I20260811 18:57:29.460817 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]). Expected version 3 got 0
I20260811 18:57:29.461102 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:29.461414 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.462782 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1c74abe9dcf642e9b4d40df6ff3639ef
I20260811 18:57:29.463042 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8fb6e34f004c466381fe6ab5b780568f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.464366 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 646) took 3498 ms. Trace:
I20260811 18:57:29.465549 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1d721b0e55384d8a8efd88a0e47b1504
I20260811 18:57:29.466154 3551 catalog_manager.cc:5495] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:29.466889 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:29.467068 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_5781967ec2ee41a5ab982d9d7394c274 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 2 got 0
I20260811 18:57:29.464485 4167 rpcz_store.cc:276] 0811 18:57:25.965826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.965881 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.464178 (+3498297us) ts_tablet_manager.cc:1145] Deleting tablet b7b48a35acf1486bbb63680ca5da31dd
0811 18:57:29.464195 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.464336 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3498113}
I20260811 18:57:29.469450 4167 tablet_replica.cc:330] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:29.469542 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5781967ec2ee41a5ab982d9d7394c274" } new_table_name: "default.Table_e03268eb26a144ba8b53cbd960628036" modify_external_catalogs: true
I20260811 18:57:29.469754 4167 raft_consensus.cc:2229] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:29.469828 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 6880ad27a3eb4599b407edeb2969a08f deleted and replaced by tablet 101994cc708a4fadb46fc8c5915a5587
I20260811 18:57:29.469997 4167 raft_consensus.cc:2258] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:29.482177 4167 ts_tablet_manager.cc:1905] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:29.482357 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_862e6dcdf4f748738d4f651170e5611f" } new_table_name: "default.Table_2940279efa3f41ec980785fb086dd12d" modify_external_catalogs: true
I20260811 18:57:29.487917 3887 raft_consensus.cc:2454] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:29.489830 3676 leader_election.cc:302] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:29.490226 5091 raft_consensus.cc:2781] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:29.490387 5091 raft_consensus.cc:686] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:29.490615 5091 consensus_queue.cc:237] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:29.492188 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_13786cfeafbc4312bf84f93b204add4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.493139 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13786cfeafbc4312bf84f93b204add4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.494406 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fb6e34f004c466381fe6ab5b780568f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.494874 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbf1868109474dcea3712c10bb4d4f3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.499748 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6d28c486e408451a990ae3905ef3c2ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d28c486e408451a990ae3905ef3c2ff already in progress: creating tablet
I20260811 18:57:29.500090 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cef66a1c435645d6822188ec4776526b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.503145 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cef66a1c435645d6822188ec4776526b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.500452 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b38a0019b5541c495088e96174c8de4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.506925 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b38a0019b5541c495088e96174c8de4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.508513 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880ad27a3eb4599b407edeb2969a08f with delete_type TABLET_DATA_DELETED (replaced by tablet 101994cc708a4fadb46fc8c5915a5587) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.508543 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.509114 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ce3acbad48bb431fad5b6904dec8e5ae (DEFAULT_TABLE table=default.Table_9743ba51a07b4fca8aaf1a188a692aac [id=4e7d8b7d88074d04b97ad14a02626671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.509681 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.510070 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.510396 3714 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.516793 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 73e15954a1604189af634a727cb48690 (DEFAULT_TABLE table=default.Table_3de36d9c6d4542be9ee2c873f0bfcfca [id=d446e82dcf024d6e972911e406ca588a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.519390 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b752cb750e9447b092ecac296b5f836e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:29.520460 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3acbad48bb431fad5b6904dec8e5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.521515 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e15954a1604189af634a727cb48690. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.524968 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b752cb750e9447b092ecac296b5f836e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.526919 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b7b48a35acf1486bbb63680ca5da31dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b48a35acf1486bbb63680ca5da31dd already in progress: creating tablet
I20260811 18:57:29.529348 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.529106 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0495c5b770b34d54afb1379a6e36204c (DEFAULT_TABLE table=default.Table_cafc7915ae7e4280a4d71e3bce4ca57c [id=bcffeabc4e924226a6c508ea5540bb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.529678 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.530052 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.530107 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.531629 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb3e997b4454598bdce102e92296887 (DEFAULT_TABLE table=default.Table_f9cadd28493e444185170bcd4c314208 [id=d0d78361eb3d4f0e8a6fa132a05a34e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.532089 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 2ee5acbaa9f54d4ba523b603641e4a97 (DEFAULT_TABLE table=default.Table_9914a933914f4377ba70e0b57c70edff [id=6d6cbea588c54de4a0966727cc2efaf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.532588 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 8319453f68a54da9813747152ef08756 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.533335 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb3e997b4454598bdce102e92296887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.534607 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8319453f68a54da9813747152ef08756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.538118 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ee5acbaa9f54d4ba523b603641e4a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.544219 5080 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.544389 5080 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.545250 5080 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:29.551940 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 385fef74b0c1421ea86c23bfee38c681 (DEFAULT_TABLE table=default.Table_ef4494689f0147118d362ed74d1bf78c [id=bd0e71434abf458795a3e2f8b6251847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.555764 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385fef74b0c1421ea86c23bfee38c681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.557940 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0495c5b770b34d54afb1379a6e36204c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.564144 4169 ts_tablet_manager.cc:1918] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.565019 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 90fde17971ea49aaad212a5836661b08 (DEFAULT_TABLE table=default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 [id=e354251945a14c9fb3846267e17bcb74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.570399 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fde17971ea49aaad212a5836661b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.572531 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f3670f8908d54ebb82f698cdd207e48f
I20260811 18:57:29.580194 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9dd303f12b584056a04e385cc80faffe" } new_table_name: "default.Table_9bb28cb6f9204966a43b9a2304b164ac" modify_external_catalogs: true
I20260811 18:57:29.578935 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 985dac332bff463781ae93c36f45fdda (DEFAULT_TABLE table=default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a [id=cc40efcc988e48c7b37a10f5f1217956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.583912 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet f3670f8908d54ebb82f698cdd207e48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.584261 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:29.585078 3706 tablet_service.cc:1467] Processing CreateTablet for tablet c8d6cbc9a8ed4b798188572c6fc1ff52 (DEFAULT_TABLE table=default.Table_5b05303fcd96430bb6ab896fbaa5097a [id=e81a772f5829416f974f5f6a19cc09b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.592680 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d6cbc9a8ed4b798188572c6fc1ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.585606 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 (DEFAULT_TABLE table=default.Table_0c3df98432314ff2b4cbd7004701b258 [id=6e4e47c552c943aca96b05e0959d681d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.594803 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9b279c6c7344f1931b81e10248b1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.586066 3720 tablet_service.cc:1467] Processing CreateTablet for tablet d32212c817c94fad8379462b027e7d3d (DEFAULT_TABLE table=default.Table_f0a164081b1349d9bc89f20d1ea6425d [id=e76087cfeb3c4a338de2a131620bc473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.598771 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32212c817c94fad8379462b027e7d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.586637 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a (DEFAULT_TABLE table=default.Table_378dd90b6af541b8ab95d9f19681a03f [id=d43a5bc491c048718e2504b3d22d2890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.602721 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:29.605551 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 476da7e2681540a89824b58fb7237df9 consensus state has no leader
I20260811 18:57:29.606225 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c86ddee4faa543c79b02e248ac801fb3" } new_table_name: "default.Table_c71b8760db044bad851b4bcd819bf75f" modify_external_catalogs: true
I20260811 18:57:29.606315 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5aa83c98b49c466cb01cdf92154ad384"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.607218 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5aa83c98b49c466cb01cdf92154ad384 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.615837 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e32f59bdd0a54fcb9c76d7d9132d966c consensus state has no leader
I20260811 18:57:29.626786 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 86a57b040ab44812a2001705493f2bcd (DEFAULT_TABLE table=default.Table_8b908c7d655a467bbda1f9d2fdc1b99a [id=f795c514f7144933b455d56dbd038cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.627409 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a57b040ab44812a2001705493f2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.628238 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 33569b8849444137bb5469995bee24c4 (DEFAULT_TABLE table=default.Table_9fc9077de3204866a090b8d53ea5bd1d [id=a147807e34814e75a700c83b4ebeab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.628683 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33569b8849444137bb5469995bee24c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.631644 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 (DEFAULT_TABLE table=default.Table_bd92de7863b94ff38b2c402c27ddf26c [id=2a5e3486b8ce482ca74dc96e4dc13862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.632112 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a782b8470b4d14ac89b3b7d463a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.633491 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985dac332bff463781ae93c36f45fdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.638779 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 476da7e2681540a89824b58fb7237df9 (DEFAULT_TABLE table=default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.639318 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476da7e2681540a89824b58fb7237df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.639287 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5b0e7b307dc84d88aff7e744fe4ed97b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.640709 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b0e7b307dc84d88aff7e744fe4ed97b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:29.644479 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 consensus state has no leader
I20260811 18:57:29.648159 4162 tablet_bootstrap.cc:700] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent opening tablet: real 0.204s user 0.001s sys 0.000s
I20260811 18:57:29.648626 4162 tablet_bootstrap.cc:654] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:29.650923 3678 leader_election.cc:334] T 494939a4eb5a45d6a123d174ad513bcb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.208s (SENT)
W20260811 18:57:29.647910 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 273375us
I20260811 18:57:29.652019 4169 log.cc:1196] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/460e03cc14444c83b6c08ca194ca5a8d
I20260811 18:57:29.652478 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9de9acdb33a2438eb4378a5eac844c35 (table default.Table_08b93745a5f44ce8bc61f090e881f952 [id=263ab2d6963f45debd14127cc7d122c8]) successfully deleted
I20260811 18:57:29.653558 4169 ts_tablet_manager.cc:1939] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:29.654115 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 544) took 6303 ms. Trace:
I20260811 18:57:29.648422 4163 tablet_bootstrap.cc:492] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:29.654536 4163 ts_tablet_manager.cc:1397] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.381s user 0.003s sys 0.011s
I20260811 18:57:29.654553 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 1ddb10e5ee5b441c875dd0b345201949 (DEFAULT_TABLE table=default.Table_6064366e4ea94146903cb48b739102e9 [id=94bd67a82648467ab2ffb80d4e1b9ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.655700 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5638cefdec734ed58f3de696740ba891 (DEFAULT_TABLE table=default.Table_832e411b94e24eb49b8798f66fa24774 [id=8b596fd1e4af496680f105839144bf79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.656060 4163 raft_consensus.cc:348] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.656436 4163 raft_consensus.cc:374] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.656549 4163 raft_consensus.cc:729] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.656912 4163 consensus_queue.cc:260] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.657253 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 72f11748cf984942ba99bda4ad6605ab (DEFAULT_TABLE table=default.Table_c6284ad4471b4b7095b11fe7a8f304e2 [id=53f3832432d242058789ef564beeae06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.656951 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 08f4d8bbe23b463680c153ae4ff4ebcf (DEFAULT_TABLE table=default.Table_d5334e4d6b9d409bbab0faf6a82ecb2a [id=7de6fc7b04824669845b02a164b3a819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.657156 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 468c71415a984ea7ab04c21fa9612070 (DEFAULT_TABLE table=default.Table_4c88a7260fbc4472bbbd3244c0cc4a85 [id=4a0a8c18d3114ce4a1c4e69e733d372b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.659138 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 23c85b2315254e4c8d9d69fec4cc9aa2 (DEFAULT_TABLE table=default.Table_e79b6e054e914fd39ca0a6282eb0ce17 [id=8d92dfcc02424160a65fca84e2417a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.659816 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 4149626f48354ba0923cad28b95be048 (DEFAULT_TABLE table=default.Table_5296710afead477eb67f356c9afbc4a4 [id=54cc236881e246a18f2af7ae6df6b983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.660360 3866 tablet_service.cc:1467] Processing CreateTablet for tablet ce3acbad48bb431fad5b6904dec8e5ae (DEFAULT_TABLE table=default.Table_9743ba51a07b4fca8aaf1a188a692aac [id=4e7d8b7d88074d04b97ad14a02626671]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.660696 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d721b0e55384d8a8efd88a0e47b1504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.660923 5091 raft_consensus.cc:482] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.661541 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.662652 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.670969 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ddb10e5ee5b441c875dd0b345201949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.671283 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4149626f48354ba0923cad28b95be048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.681013 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c74abe9dcf642e9b4d40df6ff3639ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.661082 5091 raft_consensus.cc:504] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.687891 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880ad27a3eb4599b407edeb2969a08f with delete_type TABLET_DATA_DELETED (replaced by tablet 101994cc708a4fadb46fc8c5915a5587) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:29.699586 5091 leader_election.cc:288] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.700282 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43ad6993b63e4bbd98c8cc52f1a53451" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.700955 3676 leader_election.cc:341] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:29.696720 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 45808.9us
W20260811 18:57:29.701537 3678 leader_election.cc:334] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.139s (SENT)
I20260811 18:57:29.706465 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:29.706760 3743 raft_consensus.cc:2454] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:29.711547 4163 ts_tablet_manager.cc:1428] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.057s user 0.002s sys 0.001s
W20260811 18:57:29.711714 3953 kernel_stack_watchdog.cc:198] Thread 3968 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 20007ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:29.714231 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 73e15954a1604189af634a727cb48690 (DEFAULT_TABLE table=default.Table_3de36d9c6d4542be9ee2c873f0bfcfca [id=d446e82dcf024d6e972911e406ca588a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.714675 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d3652d29236a4b12b7abd44659c70311" } new_table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" modify_external_catalogs: true
I20260811 18:57:29.718755 4169 rpcz_store.cc:276] 0811 18:57:23.350625 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.360208 (+ 9583us) service_pool.cc:227] Handling call
0811 18:57:29.433962 (+6073754us) ts_tablet_manager.cc:1145] Deleting tablet 460e03cc14444c83b6c08ca194ca5a8d
0811 18:57:29.433976 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.471748 (+ 37772us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.564008 (+ 92260us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.654093 (+ 90085us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6054256}
I20260811 18:57:29.720160 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 554) took 5977 ms. Trace:
I20260811 18:57:29.720482 4169 rpcz_store.cc:276] 0811 18:57:23.742800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.742855 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.719773 (+5976918us) ts_tablet_manager.cc:1145] Deleting tablet d593aea288e14bb1a4c54360b260cffa
0811 18:57:29.719823 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.720098 (+ 275us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5966264}
I20260811 18:57:29.729497 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a72e17b47b774a0fae13895039a39086"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.730486 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a72e17b47b774a0fae13895039a39086 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.725404 4163 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:29.727046 5080 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.735731 5080 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.736683 5080 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:29.745965 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b" } modify_external_catalogs: true
I20260811 18:57:29.746459 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6ed856dda814eb0bd8eb76c2f83c70b" } modify_external_catalogs: true
I20260811 18:57:29.747372 5080 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.747555 5080 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.748396 5080 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:29.751276 3678 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.001s (SENT)
I20260811 18:57:29.755898 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 555) took 6009 ms. Trace:
I20260811 18:57:29.761965 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3fbd6111333f4635a274f38a20902738"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:29.762864 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fbd6111333f4635a274f38a20902738 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:29.762734 5091 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.764148 5091 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.765005 5091 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.766825 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 0495c5b770b34d54afb1379a6e36204c (DEFAULT_TABLE table=default.Table_cafc7915ae7e4280a4d71e3bce4ca57c [id=bcffeabc4e924226a6c508ea5540bb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.765877 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.771065 3676 leader_election.cc:341] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.758487 4167 ts_tablet_manager.cc:1918] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:29.782910 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be655558b83041fcace57520a599b8a0" } modify_external_catalogs: true
I20260811 18:57:29.783416 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be655558b83041fcace57520a599b8a0" } modify_external_catalogs: true
I20260811 18:57:29.761108 4169 rpcz_store.cc:276] 0811 18:57:23.745355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.747573 (+ 2218us) service_pool.cc:227] Handling call
0811 18:57:29.741921 (+5994348us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:29.742475 (+ 554us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.754332 (+ 11857us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5986722}
I20260811 18:57:29.796736 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61b2dc398e804cdabeda17315814c849" } modify_external_catalogs: true
I20260811 18:57:29.797062 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61b2dc398e804cdabeda17315814c849" } modify_external_catalogs: true
I20260811 18:57:29.793112 4167 log.cc:1196] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:29.814182 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" } modify_external_catalogs: true
I20260811 18:57:29.814507 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_45b8aa8da0974fe59719c2d87bb95773" } modify_external_catalogs: true
I20260811 18:57:29.817816 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 90fde17971ea49aaad212a5836661b08 (DEFAULT_TABLE table=default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8 [id=e354251945a14c9fb3846267e17bcb74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.812882 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 556) took 6061 ms. Trace:
I20260811 18:57:29.822553 4169 rpcz_store.cc:276] 0811 18:57:23.748658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.754405 (+ 5747us) service_pool.cc:227] Handling call
0811 18:57:29.795216 (+6040811us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:29.801942 (+ 6726us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.809967 (+ 8025us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6031578}
I20260811 18:57:29.832041 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" } modify_external_catalogs: true
I20260811 18:57:29.832350 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6cc21ee738d3430ead80d93044f49334" } modify_external_catalogs: true
I20260811 18:57:29.835212 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 553) took 6093 ms. Trace:
I20260811 18:57:29.840025 5091 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.840075 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7c395f6c4c364b7eb463f78e16675847" } new_table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" modify_external_catalogs: true
I20260811 18:57:29.840236 5091 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.841109 5091 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:29.842191 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:29.855809 3676 leader_election.cc:341] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:29.855434 4169 rpcz_store.cc:276] 0811 18:57:23.742061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.742183 (+ 122us) service_pool.cc:227] Handling call
0811 18:57:29.835006 (+6092823us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:29.835028 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.835187 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6071613}
I20260811 18:57:29.860316 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 557) took 6095 ms. Trace:
W20260811 18:57:29.860913 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ea579c33f8b14e9fa7fa389ce7f45931 (table default.Table_498524390fc04da5b952cdc86fe5decd [id=77b12e379e234e6f8285ebe4c3c46aba], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ea579c33f8b14e9fa7fa389ce7f45931 consensus state has no leader
I20260811 18:57:29.867239 4162 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:29.867563 4162 ts_tablet_manager.cc:1397] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.495s user 0.002s sys 0.007s
I20260811 18:57:29.869386 4162 raft_consensus.cc:348] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.870061 4162 raft_consensus.cc:374] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.870267 4162 raft_consensus.cc:729] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:29.878826 3678 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.147s (SENT)
I20260811 18:57:29.878965 3678 leader_election.cc:302] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:29.879189 3678 raft_consensus.cc:2670] T 94943cfbe41a47b3a36c81957a3adaeb P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:29.860494 4169 rpcz_store.cc:276] 0811 18:57:23.764614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.765352 (+ 738us) service_pool.cc:227] Handling call
0811 18:57:29.860046 (+6094694us) ts_tablet_manager.cc:1145] Deleting tablet 1f60a286ac024c45bcb845227c5445c9
0811 18:57:29.860068 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.860283 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6094231}
I20260811 18:57:29.870756 4162 consensus_queue.cc:260] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.891386 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb3e997b4454598bdce102e92296887 (DEFAULT_TABLE table=default.Table_f9cadd28493e444185170bcd4c314208 [id=d0d78361eb3d4f0e8a6fa132a05a34e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:29.889430 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 558) took 6124 ms. Trace:
I20260811 18:57:29.892469 4169 rpcz_store.cc:276] 0811 18:57:23.764904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.779759 (+ 14855us) service_pool.cc:227] Handling call
0811 18:57:29.885381 (+6105622us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:29.885831 (+ 450us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.889350 (+ 3519us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6087016}
I20260811 18:57:29.928757 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 2ee5acbaa9f54d4ba523b603641e4a97 (DEFAULT_TABLE table=default.Table_9914a933914f4377ba70e0b57c70edff [id=6d6cbea588c54de4a0966727cc2efaf6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:29.932799 3678 leader_election.cc:334] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.874s (SENT)
I20260811 18:57:29.935135 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3c807c78cf1249abbf878e819573553d" } new_table_name: "default.Table_c5b1c8014bf048a09ea36fcf6713e7ba" modify_external_catalogs: true
I20260811 18:57:29.964570 5091 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:29.964888 5091 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:29.965847 5091 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:29.966332 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.125:38889, user_credentials={real_user=slave}} blocked reactor thread for 49488us
I20260811 18:57:29.919539 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 570) took 5960 ms. Trace:
I20260811 18:57:29.967588 4167 ts_tablet_manager.cc:1939] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:29.988152 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 653) took 3937 ms. Trace:
I20260811 18:57:29.988610 4167 rpcz_store.cc:276] 0811 18:57:26.050908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.050973 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:29.469372 (+3418399us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:29.469389 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.526678 (+ 57289us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.715691 (+189013us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:29.988081 (+272390us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3418094}
I20260811 18:57:29.989939 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 799ba9a044e145ec89d0025c03f391ea (table default.Table_56b89b2b2c4b45768b2866c74d9f1390 [id=7736472828a3499c8ad8026e164b9759]) successfully deleted
I20260811 18:57:29.991480 4162 ts_tablet_manager.cc:1428] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.124s user 0.003s sys 0.004s
I20260811 18:57:29.997716 4162 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:30.008242 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 985dac332bff463781ae93c36f45fdda (DEFAULT_TABLE table=default.Table_fab0e8f9f0fd4a2aaf2fd0ec53ec845a [id=cc40efcc988e48c7b37a10f5f1217956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.010785 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c8d6cbc9a8ed4b798188572c6fc1ff52 (DEFAULT_TABLE table=default.Table_5b05303fcd96430bb6ab896fbaa5097a [id=e81a772f5829416f974f5f6a19cc09b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.012347 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 (DEFAULT_TABLE table=default.Table_0c3df98432314ff2b4cbd7004701b258 [id=6e4e47c552c943aca96b05e0959d681d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.012535 3861 tablet_service.cc:1467] Processing CreateTablet for tablet d32212c817c94fad8379462b027e7d3d (DEFAULT_TABLE table=default.Table_f0a164081b1349d9bc89f20d1ea6425d [id=e76087cfeb3c4a338de2a131620bc473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.016387 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 385fef74b0c1421ea86c23bfee38c681 (DEFAULT_TABLE table=default.Table_ef4494689f0147118d362ed74d1bf78c [id=bd0e71434abf458795a3e2f8b6251847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.017846 5091 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.018061 5091 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:30.019086 5091 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:30.025679 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.026410 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet f3670f8908d54ebb82f698cdd207e48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.027010 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a (DEFAULT_TABLE table=default.Table_378dd90b6af541b8ab95d9f19681a03f [id=d43a5bc491c048718e2504b3d22d2890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.027531 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.026270 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 8319453f68a54da9813747152ef08756 (DEFAULT_TABLE table=default.Table_b069c186694049998175e3d83241b318 [id=973acfb14e994b1e947f3a89d7ed0ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.029109 4169 rpcz_store.cc:276] 0811 18:57:23.946951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:23.949599 (+ 2648us) service_pool.cc:227] Handling call
0811 18:57:29.907215 (+5957616us) ts_tablet_manager.cc:1145] Deleting tablet 56d24aff9fd34f3eb05fdf942cc7efc7
0811 18:57:29.907236 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:29.907390 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5931437}
I20260811 18:57:30.033056 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 575) took 5457 ms. Trace:
I20260811 18:57:30.034410 4169 rpcz_store.cc:276] 0811 18:57:24.575897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.575958 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:30.031888 (+5455930us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:30.032037 (+ 149us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.032906 (+ 869us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5447097}
W20260811 18:57:30.039167 3676 leader_election.cc:341] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:30.040539 3676 leader_election.cc:341] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:30.070916 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.103582 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 576) took 5519 ms. Trace:
I20260811 18:57:30.112121 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 658) took 4023 ms. Trace:
W20260811 18:57:30.116474 3540 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.126:34821, user_credentials={real_user=slave}} blocked reactor thread for 133579us
W20260811 18:57:30.121603 3678 leader_election.cc:334] T 125757b8c1a6448b841c63339f392339 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.192s (SENT)
W20260811 18:57:30.125445 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cdb92902c2584751af77d3117f52cb4e consensus state has no leader
I20260811 18:57:30.104871 4169 rpcz_store.cc:276] 0811 18:57:24.583861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.584016 (+ 155us) service_pool.cc:227] Handling call
0811 18:57:30.037192 (+5453176us) ts_tablet_manager.cc:1145] Deleting tablet 14e34732bda44670ae9da8facc405221
0811 18:57:30.037356 (+ 164us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.103091 (+ 65735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5447722}
I20260811 18:57:30.134223 4167 rpcz_store.cc:276] 0811 18:57:26.088903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.088958 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:29.989726 (+3900768us) ts_tablet_manager.cc:1145] Deleting tablet ea4ad482bde44d6bab5ec809054aaebd
0811 18:57:29.989746 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.111975 (+122229us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3900507}
I20260811 18:57:30.134999 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 671) took 3946 ms. Trace:
W20260811 18:57:30.135236 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ea4ad482bde44d6bab5ec809054aaebd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea4ad482bde44d6bab5ec809054aaebd already in progress: creating tablet
I20260811 18:57:30.135286 4167 rpcz_store.cc:276] 0811 18:57:26.188789 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.188845 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:30.134791 (+3945946us) ts_tablet_manager.cc:1145] Deleting tablet b16cb4c1796147fbb10109f52071505e
0811 18:57:30.134812 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.134970 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3945680}
I20260811 18:57:30.135972 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 670) took 3947 ms. Trace:
W20260811 18:57:30.136068 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b16cb4c1796147fbb10109f52071505e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16cb4c1796147fbb10109f52071505e already in progress: creating tablet
I20260811 18:57:30.136116 4167 rpcz_store.cc:276] 0811 18:57:26.188663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.188715 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:30.135803 (+3947088us) ts_tablet_manager.cc:1145] Deleting tablet 54fa32f8fa1144c4bd9dcd00a550001a
0811 18:57:30.135819 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.135946 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3946303}
I20260811 18:57:30.136723 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 669) took 3948 ms. Trace:
W20260811 18:57:30.136868 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 54fa32f8fa1144c4bd9dcd00a550001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54fa32f8fa1144c4bd9dcd00a550001a already in progress: creating tablet
I20260811 18:57:30.136830 4167 rpcz_store.cc:276] 0811 18:57:26.188460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.217735 (+ 29275us) service_pool.cc:227] Handling call
0811 18:57:30.136592 (+3918857us) ts_tablet_manager.cc:1145] Deleting tablet 1d81e93739f44d9e99b6ebc647e1d0e4
0811 18:57:30.136604 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.136702 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3918472}
I20260811 18:57:30.137244 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 672) took 3875 ms. Trace:
I20260811 18:57:30.137343 4167 rpcz_store.cc:276] 0811 18:57:26.261412 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.261485 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:30.137089 (+3875604us) ts_tablet_manager.cc:1145] Deleting tablet 1811e5f62a714f8388460dfcb1717e81
0811 18:57:30.137102 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.137222 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3875132}
W20260811 18:57:30.137439 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d81e93739f44d9e99b6ebc647e1d0e4 already in progress: creating tablet
I20260811 18:57:30.137651 4167 tablet_replica.cc:330] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:30.138051 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1811e5f62a714f8388460dfcb1717e81 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1811e5f62a714f8388460dfcb1717e81 already in progress: creating tablet
W20260811 18:57:30.138945 3678 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.060s (SENT)
I20260811 18:57:30.155288 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 98ad733726594d2a8087f857f155dbe2
I20260811 18:57:30.157716 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 85140a3800f34a23b716703fd4e73d17
W20260811 18:57:30.161248 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38a4783168554053aa953dba5e893b6b consensus state has no leader
I20260811 18:57:30.161727 3707 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ad733726594d2a8087f857f155dbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.164031 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 85140a3800f34a23b716703fd4e73d17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.165536 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_827736b8218e48e086bf05354914d450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.166308 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_827736b8218e48e086bf05354914d450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.167706 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_de8a2d61a3cc411a97ad8e8df11c7d89" } new_table_name: "default.Table_972a064a812a441bb4068503de32341e" modify_external_catalogs: true
I20260811 18:57:30.168324 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1cdea1db192a43f4b210579b0813e33a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.168776 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 577) took 5583 ms. Trace:
I20260811 18:57:30.168900 4169 rpcz_store.cc:276] 0811 18:57:24.585283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.585377 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:30.168505 (+5583128us) ts_tablet_manager.cc:1145] Deleting tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8
0811 18:57:30.168545 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.168710 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5538336}
I20260811 18:57:30.169421 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 578) took 5572 ms. Trace:
I20260811 18:57:30.169507 4169 rpcz_store.cc:276] 0811 18:57:24.596730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.597567 (+ 837us) service_pool.cc:227] Handling call
0811 18:57:30.169288 (+5571721us) ts_tablet_manager.cc:1145] Deleting tablet 64b922f2a71544e8a18533aa0a1ccb52
0811 18:57:30.169303 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.169404 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5564294}
I20260811 18:57:30.169785 4169 tablet_replica.cc:330] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:30.170014 4169 raft_consensus.cc:2229] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.170243 4169 raft_consensus.cc:2258] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.171022 4169 ts_tablet_manager.cc:1905] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:30.171741 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cdea1db192a43f4b210579b0813e33a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.174623 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93b2871d4c454120a6b609ae91e096e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.175379 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b2a92f77f0074806a1b56a31c303dd69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.178017 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2076434270fa4c9c99b4d12f1aa924c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.179960 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_80a7a76a26ab4fc9919bfef0a312f217"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.182968 5286 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.183401 5286 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.184445 5286 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:30.186088 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2a92f77f0074806a1b56a31c303dd69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.190790 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93b2871d4c454120a6b609ae91e096e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.193056 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2076434270fa4c9c99b4d12f1aa924c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.194139 3678 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.256s (SENT)
I20260811 18:57:30.194507 3678 leader_election.cc:302] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:30.194983 5091 raft_consensus.cc:2726] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:30.195044 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80a7a76a26ab4fc9919bfef0a312f217 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.196615 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 460e03cc14444c83b6c08ca194ca5a8d (table default.Table_9bb5acd56644427f9aec05c1ab8e46ee [id=7fd10d057c3f4cd2bc3ecbd056d7bc46]) successfully deleted
I20260811 18:57:30.211531 4162 tablet_bootstrap.cc:654] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.215657 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d593aea288e14bb1a4c54360b260cffa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d593aea288e14bb1a4c54360b260cffa already in progress: creating tablet
W20260811 18:57:30.216209 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe203387bc9c45ba82afe6b058aa5e5f already in progress: creating tablet
W20260811 18:57:30.216761 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5d495354c0994fb4a74c376078e41a26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d495354c0994fb4a74c376078e41a26 already in progress: creating tablet
W20260811 18:57:30.217067 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b3f0cdcee681460084662e01152ad9dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3f0cdcee681460084662e01152ad9dd already in progress: creating tablet
W20260811 18:57:30.217363 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1f60a286ac024c45bcb845227c5445c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f60a286ac024c45bcb845227c5445c9 already in progress: creating tablet
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
W20260811 18:57:30.238575 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 122b7898fea44ef78f269b1505d8ddc4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 122b7898fea44ef78f269b1505d8ddc4 already in progress: creating tablet
W20260811 18:57:30.254560 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7057b8a7bae249caaed304b5c66e6871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7057b8a7bae249caaed304b5c66e6871 already in progress: creating tablet
W20260811 18:57:30.254866 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 14e34732bda44670ae9da8facc405221 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e34732bda44670ae9da8facc405221 already in progress: creating tablet
I20260811 18:57:30.256049 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3acbad48bb431fad5b6904dec8e5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.256312 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.258119 4163 tablet_bootstrap.cc:654] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:30.258389 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d6cbc9a8ed4b798188572c6fc1ff52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.259349 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c85b2315254e4c8d9d69fec4cc9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.262478 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9b279c6c7344f1931b81e10248b1b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.262744 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8319453f68a54da9813747152ef08756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.263476 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0495c5b770b34d54afb1379a6e36204c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.265573 3850 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 143) took 1038 ms. Trace:
I20260811 18:57:30.265712 3850 rpcz_store.cc:276] 0811 18:57:29.227368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.227435 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:29.348708 (+121273us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x61700010570c. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d15aded6e 00007f4d158e815e 00007f4d181e2a88 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378
0811 18:57:30.265506 (+916798us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":124492544}
I20260811 18:57:30.265862 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385fef74b0c1421ea86c23bfee38c681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.267484 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d32212c817c94fad8379462b027e7d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.272993 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 985dac332bff463781ae93c36f45fdda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.273303 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 4687b95728fc485fa0e842bc447eb83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.273507 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 867259e37fe44c33a35823ec2d896754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.277541 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb3e997b4454598bdce102e92296887. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.282092 3678 leader_election.cc:334] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.005s (SENT)
I20260811 18:57:30.283160 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.283494 3745 raft_consensus.cc:2454] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.286158 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.286599 3743 raft_consensus.cc:2454] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.292387 4167 raft_consensus.cc:2229] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.279855 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 86a57b040ab44812a2001705493f2bcd (DEFAULT_TABLE table=default.Table_8b908c7d655a467bbda1f9d2fdc1b99a [id=f795c514f7144933b455d56dbd038cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.280558 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 47a782b8470b4d14ac89b3b7d463a125 (DEFAULT_TABLE table=default.Table_bd92de7863b94ff38b2c402c27ddf26c [id=2a5e3486b8ce482ca74dc96e4dc13862]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.280962 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 476da7e2681540a89824b58fb7237df9 (DEFAULT_TABLE table=default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.294387 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86a57b040ab44812a2001705493f2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.294649 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ee5acbaa9f54d4ba523b603641e4a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.287230 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 33569b8849444137bb5469995bee24c4 (DEFAULT_TABLE table=default.Table_9fc9077de3204866a090b8d53ea5bd1d [id=a147807e34814e75a700c83b4ebeab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.295094 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e15954a1604189af634a727cb48690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.288834 4163 tablet_bootstrap.cc:492] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.298549 4163 ts_tablet_manager.cc:1397] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.573s user 0.010s sys 0.019s
I20260811 18:57:30.300107 4167 raft_consensus.cc:2258] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.300426 4163 raft_consensus.cc:348] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.300762 4163 raft_consensus.cc:374] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.300884 4163 raft_consensus.cc:729] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.301173 4163 consensus_queue.cc:260] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.301864 4163 ts_tablet_manager.cc:1428] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:30.303845 4163 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:30.305035 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468c71415a984ea7ab04c21fa9612070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.306203 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f4d8bbe23b463680c153ae4ff4ebcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.310520 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a782b8470b4d14ac89b3b7d463a125. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.313548 4163 tablet_bootstrap.cc:654] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.314297 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:30.315526 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 7d747f03ad234805a4ea4ef60840d426 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:30.316820 4167 ts_tablet_manager.cc:1905] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.333595 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ebca8dd587b144d7909761c052084855"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.334878 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebca8dd587b144d7909761c052084855 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.337563 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.338543 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f13efa6fe524f5bbd7154aa2f991d38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
I20260811 18:57:30.340458 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e9433d18ac7b46958942fc1f4b458354" } new_table_name: "default.Table_772fdfa6598e4b2ca2aa3a4b28746b24" modify_external_catalogs: true
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
I20260811 18:57:30.359477 4162 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:30.359825 4162 ts_tablet_manager.cc:1397] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.362s user 0.005s sys 0.003s
I20260811 18:57:30.361155 4162 raft_consensus.cc:348] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
#3 0x7fc7b9db98c6 in kudu::rpc::ServicePool::Init(int) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
I20260811 18:57:30.313784 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72f11748cf984942ba99bda4ad6605ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.369020 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476da7e2681540a89824b58fb7237df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.369230 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5638cefdec734ed58f3de696740ba891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.370510 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33569b8849444137bb5469995bee24c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.371112 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:30.375202 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.376227 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd5ee099cbc94fd285f1b71eea0cde80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.378481 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fde17971ea49aaad212a5836661b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.380700 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_41c44c966ece466eb232aeab834c5fe6" } new_table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" modify_external_catalogs: true
I20260811 18:57:30.384260 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aaf507832a664e6ebcdafffc433bbac7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.385291 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aaf507832a664e6ebcdafffc433bbac7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.391484 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8" } modify_external_catalogs: true
I20260811 18:57:30.391758 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f8f11c0bc2ec415d9b990fbbd81efcd8" } modify_external_catalogs: true
I20260811 18:57:30.392405 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9de9363efd764bd4ac323f86fd70658d" } new_table_name: "default.Table_4a429d101b1e46efaafa42a86e2f64f7" modify_external_catalogs: true
I20260811 18:57:30.394177 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d5ebf213ce401984be01bac1813640" } modify_external_catalogs: true
I20260811 18:57:30.394472 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_65d5ebf213ce401984be01bac1813640" } modify_external_catalogs: true
W20260811 18:57:30.323444 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 68766.1us
I20260811 18:57:30.331493 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b199f4c9e14b468cb26cbd61fc8586 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cce97bfdb1d41e685e091c8c5cc6946) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:30.398145 3824 consensus_peers.cc:487] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6f36ab6bb774257b5c2692c64eff63d. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:30.398491 3824 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.398882 5287 raft_consensus.cc:2781] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.398996 5287 raft_consensus.cc:482] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.399081 5287 raft_consensus.cc:3037] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.400111 5286 consensus_queue.cc:1035] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.400969 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_59f9d96102f349498c7f854e48e3d7c5" } new_table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" modify_external_catalogs: true
I20260811 18:57:30.412134 3824 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.412569 5286 raft_consensus.cc:2781] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.412741 5286 raft_consensus.cc:482] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.412901 5286 raft_consensus.cc:3037] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.423269 5080 consensus_queue.cc:1035] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.423274 3824 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.427526 5080 raft_consensus.cc:2781] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.427644 5080 raft_consensus.cc:482] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.427726 5080 raft_consensus.cc:3037] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.361436 4162 raft_consensus.cc:374] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.427966 4162 raft_consensus.cc:729] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:30.428658 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 already in progress: creating tablet
W20260811 18:57:30.429105 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 64b922f2a71544e8a18533aa0a1ccb52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64b922f2a71544e8a18533aa0a1ccb52 already in progress: creating tablet
I20260811 18:57:30.430090 4162 consensus_queue.cc:260] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:30.439858 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8ca801c79fc499ab50cfe76cdb03b00 consensus state has no leader
W20260811 18:57:30.440755 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 101994cc708a4fadb46fc8c5915a5587 has no consensus state
W20260811 18:57:30.461721 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 05cba6b392b642e59321570aac3541d0 consensus state has no leader
I20260811 18:57:30.447237 5313 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:30.449030 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:30.462291 3743 raft_consensus.cc:2454] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:30.451498 3740 raft_consensus.cc:1261] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:30.468328 5287 raft_consensus.cc:504] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.469534 5287 leader_election.cc:288] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.473742 5286 raft_consensus.cc:504] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.475044 5286 leader_election.cc:288] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.480683 4162 ts_tablet_manager.cc:1428] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.121s user 0.000s sys 0.002s
I20260811 18:57:30.487015 4163 tablet_bootstrap.cc:492] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.493283 4162 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:30.494767 4163 ts_tablet_manager.cc:1397] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.001s
I20260811 18:57:30.501034 4163 raft_consensus.cc:348] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.513754 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_949ccbca1e8b44b79578377f67f2422c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.515077 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_949ccbca1e8b44b79578377f67f2422c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.516530 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7a8c696f8df4bec92e19adef1eff6d2" } new_table_name: "default.Table_495f858f5eec496680f982bccd44415d" modify_external_catalogs: true
I20260811 18:57:30.518797 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6204a77bcccf427db16e32981a5621fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.519569 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6204a77bcccf427db16e32981a5621fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.520659 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d584e1cc7d8b45be912d8610e391f4f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.521369 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d584e1cc7d8b45be912d8610e391f4f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.530563 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c7f500a5444bae9a015a5e903c0622" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.530802 3740 raft_consensus.cc:3037] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.531605 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4687b95728fc485fa0e842bc447eb83f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.531777 3743 raft_consensus.cc:3037] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:30.532172 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 531ba05901734d6b93bc2be506e8ec5b has no consensus state
I20260811 18:57:30.534096 4863 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:34821
I20260811 18:57:30.534230 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aadfbb276f80467d884601ea6c92a401"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.534749 4162 tablet_bootstrap.cc:654] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:30.535024 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aadfbb276f80467d884601ea6c92a401 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.537289 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_634c196a72864956aedd6ceb1757173a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.538069 4863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
W20260811 18:57:30.538472 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_634c196a72864956aedd6ceb1757173a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.538527 5326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:34821 every 8 connection(s)
I20260811 18:57:30.541786 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 4863
I20260811 18:57:30.545457 3743 raft_consensus.cc:2454] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.547147 3740 raft_consensus.cc:2454] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.504824 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 85140a3800f34a23b716703fd4e73d17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.505030 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet 02808605ee6c4634af47fc3fd57e9cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.552420 4163 raft_consensus.cc:374] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.552731 4163 raft_consensus.cc:729] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.552628 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_94caf53b154f4e39b9ea7bf02dcb11c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.553092 4163 consensus_queue.cc:260] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:30.553634 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.505541 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ad733726594d2a8087f857f155dbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.520499 5323 tablet.cc:1722] T 55c1a7d44717415ca76a2d0b7e94394e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:30.568702 5080 raft_consensus.cc:504] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.569618 5080 leader_election.cc:288] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:30.571234 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 83952.7us
I20260811 18:57:30.528115 4169 ts_tablet_manager.cc:1918] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:30.591851 4169 log.cc:1196] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f61d3822fc2d4b569f223238a6464e0a
I20260811 18:57:30.592345 4169 ts_tablet_manager.cc:1939] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:30.593183 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 579) took 5788 ms. Trace:
I20260811 18:57:30.593290 4169 rpcz_store.cc:276] 0811 18:57:24.804985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.805042 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:30.169734 (+5364692us) ts_tablet_manager.cc:1145] Deleting tablet f61d3822fc2d4b569f223238a6464e0a
0811 18:57:30.169746 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.409096 (+239350us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.527967 (+118871us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.593164 (+ 65197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5364239}
I20260811 18:57:30.594077 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 584) took 5762 ms. Trace:
I20260811 18:57:30.594321 4169 rpcz_store.cc:276] 0811 18:57:24.831132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.831186 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:30.593911 (+5762725us) ts_tablet_manager.cc:1145] Deleting tablet 0b40da8cb7324da9804e6fdafd8e8266
0811 18:57:30.593926 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.594056 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5762369}
I20260811 18:57:30.594938 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 587) took 5762 ms. Trace:
I20260811 18:57:30.595118 4169 rpcz_store.cc:276] 0811 18:57:24.832436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.832487 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:30.594792 (+5762305us) ts_tablet_manager.cc:1145] Deleting tablet 3b40e44a2f4940af891eddf6c3b6293c
0811 18:57:30.594807 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.594909 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5761883}
W20260811 18:57:30.529500 3824 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.333s (SENT)
I20260811 18:57:30.595451 3824 leader_election.cc:302] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:30.595676 3824 raft_consensus.cc:2670] T e5fd9fce095e4a769dbb38c7903359cc P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:30.596496 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 67025.5us
W20260811 18:57:30.596796 3824 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.372s (SENT)
W20260811 18:57:30.597201 3824 consensus_peers.cc:487] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 799ba9a044e145ec89d0025c03f391ea. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:30.597849 3824 consensus_peers.cc:487] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:30.598071 3824 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.598565 5080 raft_consensus.cc:2781] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:30.598812 5080 raft_consensus.cc:482] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:30.598901 5080 raft_consensus.cc:3037] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.601619 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24c4140a00244c8d8446d91705a8d4a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.602475 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24c4140a00244c8d8446d91705a8d4a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.602860 5080 raft_consensus.cc:504] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.604123 5080 leader_election.cc:288] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:30.606833 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3a0329aed6694593bc21571dd8f59670"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.607847 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a0329aed6694593bc21571dd8f59670 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.610673 4163 ts_tablet_manager.cc:1428] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.112s user 0.003s sys 0.000s
I20260811 18:57:30.636139 3551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c5b1c8014bf048a09ea36fcf6713e7ba [id=1f86b4e3748c4cb7b34b2a24524dd1ed] alter complete (version 2)
I20260811 18:57:30.636861 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7ed0b81c09744e7af02ac80fb40508c" } new_table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" modify_external_catalogs: true
W20260811 18:57:30.642915 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 44857.7us
I20260811 18:57:30.643867 3824 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.644215 3824 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.644948 5286 raft_consensus.cc:2781] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.644891 5080 consensus_queue.cc:1035] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.645459 5283 raft_consensus.cc:2781] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.646100 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4133a1bbf6474492bbf3683ca9608a52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.647014 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4133a1bbf6474492bbf3683ca9608a52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.648999 5091 consensus_queue.cc:1035] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:30.654376 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 585) took 5822 ms. Trace:
I20260811 18:57:30.655505 4167 ts_tablet_manager.cc:1918] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:30.655664 4167 log.cc:1196] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:30.656047 4167 ts_tablet_manager.cc:1939] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:30.654503 4169 rpcz_store.cc:276] 0811 18:57:24.831808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.831874 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:30.654202 (+5822328us) ts_tablet_manager.cc:1145] Deleting tablet 19d7716739f848d78b9cbbd3975f3d44
0811 18:57:30.654215 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.654328 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5813280}
I20260811 18:57:30.663329 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 586) took 5831 ms. Trace:
I20260811 18:57:30.663465 4169 rpcz_store.cc:276] 0811 18:57:24.832203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.832261 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:30.663174 (+5830913us) ts_tablet_manager.cc:1145] Deleting tablet 2d0b59cafb154416952aecaa418e861f
0811 18:57:30.663190 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.663306 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5821627}
I20260811 18:57:30.665560 5286 raft_consensus.cc:686] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.665927 5286 consensus_queue.cc:237] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.674015 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f61d3822fc2d4b569f223238a6464e0a (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
W20260811 18:57:30.674415 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0b40da8cb7324da9804e6fdafd8e8266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b40da8cb7324da9804e6fdafd8e8266 already in progress: creating tablet
I20260811 18:57:30.674428 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "867259e37fe44c33a35823ec2d896754" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.674824 3745 raft_consensus.cc:3037] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.676694 5283 raft_consensus.cc:686] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.677111 5283 consensus_queue.cc:237] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.678210 3745 raft_consensus.cc:2454] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.681149 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 588) took 5848 ms. Trace:
I20260811 18:57:30.681283 4169 rpcz_store.cc:276] 0811 18:57:24.833160 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833213 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:30.680987 (+5847774us) ts_tablet_manager.cc:1145] Deleting tablet 92e49846ab714dd294acdd09d49a83bd
0811 18:57:30.681002 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.681122 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5838817}
I20260811 18:57:30.681802 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 589) took 5848 ms. Trace:
I20260811 18:57:30.681888 4169 rpcz_store.cc:276] 0811 18:57:24.833609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:24.833678 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:30.681678 (+5848000us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:30.681689 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.681784 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5839090}
I20260811 18:57:30.682305 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 619) took 5255 ms. Trace:
I20260811 18:57:30.682411 4169 rpcz_store.cc:276] 0811 18:57:25.426729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.427036 (+ 307us) service_pool.cc:227] Handling call
0811 18:57:30.682205 (+5255169us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:30.682215 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.682288 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5254928}
I20260811 18:57:30.682845 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 620) took 5256 ms. Trace:
I20260811 18:57:30.683250 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 673) took 4383 ms. Trace:
I20260811 18:57:30.683352 4167 rpcz_store.cc:276] 0811 18:57:26.299963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.300031 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:30.137606 (+3837575us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:30.137615 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.501865 (+364250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.551693 (+ 49828us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.683222 (+131529us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31396,"tablet-delete.queue_time_us":3812313}
I20260811 18:57:30.683902 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 674) took 4279 ms. Trace:
I20260811 18:57:30.684048 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ad424c8843ff434ba0449a86849bb5a0 (table default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]) successfully deleted
I20260811 18:57:30.684000 4167 rpcz_store.cc:276] 0811 18:57:26.404865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.404948 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:30.683772 (+4278824us) ts_tablet_manager.cc:1145] Deleting tablet 335fc15ea75d483990e6ae5a5872682f
0811 18:57:30.683787 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.683882 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4278527}
I20260811 18:57:30.684496 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 675) took 4228 ms. Trace:
I20260811 18:57:30.684577 4167 rpcz_store.cc:276] 0811 18:57:26.456387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.456459 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:30.684383 (+4227924us) ts_tablet_manager.cc:1145] Deleting tablet 705605cd929940f8a252b6e95abbeb95
0811 18:57:30.684395 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.684478 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4227656}
W20260811 18:57:30.684686 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 335fc15ea75d483990e6ae5a5872682f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 335fc15ea75d483990e6ae5a5872682f already in progress: creating tablet
I20260811 18:57:30.685057 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 676) took 4210 ms. Trace:
I20260811 18:57:30.685140 4167 rpcz_store.cc:276] 0811 18:57:26.474187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.474255 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:30.684798 (+4210543us) ts_tablet_manager.cc:1145] Deleting tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
0811 18:57:30.684807 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.685040 (+ 233us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4210261}
W20260811 18:57:30.685272 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 705605cd929940f8a252b6e95abbeb95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 705605cd929940f8a252b6e95abbeb95 already in progress: creating tablet
I20260811 18:57:30.685421 4167 tablet_replica.cc:330] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:30.685874 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a already in progress: creating tablet
I20260811 18:57:30.686985 4162 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:30.691761 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f7c3f328d7e49db85f19e13aa462cd6" } new_table_name: "default.Table_efc46b70936f4bd292f775b723c1fbff" modify_external_catalogs: true
I20260811 18:57:30.691606 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:30.695197 4162 ts_tablet_manager.cc:1397] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.202s user 0.005s sys 0.002s
I20260811 18:57:30.692539 5091 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:30.696205 5091 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.696962 5327 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:30.697240 5091 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:30.693215 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b199f4c9e14b468cb26cbd61fc8586" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:30.697809 3740 raft_consensus.cc:3037] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:30.701054 3740 raft_consensus.cc:2454] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:30.701922 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.702780 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.696171 4162 raft_consensus.cc:348] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.708405 4162 raft_consensus.cc:374] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.708542 4162 raft_consensus.cc:729] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.708822 4162 consensus_queue.cc:260] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.709364 4162 ts_tablet_manager.cc:1428] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20260811 18:57:30.709710 4162 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:30.719036 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3b40e44a2f4940af891eddf6c3b6293c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b40e44a2f4940af891eddf6c3b6293c already in progress: creating tablet
W20260811 18:57:30.719448 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 19d7716739f848d78b9cbbd3975f3d44 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19d7716739f848d78b9cbbd3975f3d44 already in progress: creating tablet
I20260811 18:57:30.721562 3713 tablet_service.cc:1467] Processing CreateTablet for tablet d21973bc8e514a089c2beb24a686060f (DEFAULT_TABLE table=default.Table_5aa83c98b49c466cb01cdf92154ad384 [id=01ddedd33be14c9ab35e1578361dd733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.722076 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21973bc8e514a089c2beb24a686060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.722831 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b (DEFAULT_TABLE table=default.Table_8fb6e34f004c466381fe6ab5b780568f [id=80d3089bffc748fc93707657e47ba89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.723628 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.725689 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf54e295be64486a90918a8d2eb8b33 (DEFAULT_TABLE table=default.Table_4b38a0019b5541c495088e96174c8de4 [id=05dc3f3a80494c168dcc8e02a62e2bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.726184 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf54e295be64486a90918a8d2eb8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.726639 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 75ceba7bfc8240268b16c789f07813d1 (DEFAULT_TABLE table=default.Table_13786cfeafbc4312bf84f93b204add4f [id=a11b457b0ad74279949798ccab096915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.727077 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ceba7bfc8240268b16c789f07813d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.682928 4169 rpcz_store.cc:276] 0811 18:57:25.426813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.427324 (+ 511us) service_pool.cc:227] Handling call
0811 18:57:30.682748 (+5255424us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:30.682758 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.682829 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5255218}
I20260811 18:57:30.732316 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 618) took 5305 ms. Trace:
I20260811 18:57:30.732424 4169 rpcz_store.cc:276] 0811 18:57:25.426635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.431643 (+ 5008us) service_pool.cc:227] Handling call
0811 18:57:30.732133 (+5300490us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:30.732147 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.732293 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5300275}
I20260811 18:57:30.732748 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 617) took 5320 ms. Trace:
I20260811 18:57:30.732854 4169 rpcz_store.cc:276] 0811 18:57:25.412202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.421317 (+ 9115us) service_pool.cc:227] Handling call
0811 18:57:30.732644 (+5311327us) ts_tablet_manager.cc:1145] Deleting tablet 380c1f53b3ac4509a09b25a2800028af
0811 18:57:30.732653 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.732731 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5275503}
I20260811 18:57:30.701756 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:30.733219 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 621) took 4980 ms. Trace:
I20260811 18:57:30.733285 4169 rpcz_store.cc:276] 0811 18:57:25.753038 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.753148 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:30.733122 (+4979974us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:30.733132 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.733205 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4979672}
I20260811 18:57:30.733546 4169 tablet_replica.cc:330] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:30.733703 3676 leader_election.cc:341] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:30.688597 4163 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:30.720563 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:30.737651 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:30.737932 4169 raft_consensus.cc:2229] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.738210 4169 raft_consensus.cc:2258] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.738484 4167 raft_consensus.cc:2229] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:30.739100 3848 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 146) took 1402 ms. Trace:
W20260811 18:57:30.740372 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2d0b59cafb154416952aecaa418e861f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d0b59cafb154416952aecaa418e861f already in progress: creating tablet
W20260811 18:57:30.740852 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 92e49846ab714dd294acdd09d49a83bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92e49846ab714dd294acdd09d49a83bd already in progress: creating tablet
W20260811 18:57:30.741642 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b57ed7814a1e440cba362613cbab32f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b57ed7814a1e440cba362613cbab32f8 already in progress: creating tablet
W20260811 18:57:30.741987 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 68a7051bc539468aafba30fbaf0669d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68a7051bc539468aafba30fbaf0669d4 already in progress: creating tablet
I20260811 18:57:30.742534 4169 ts_tablet_manager.cc:1905] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.739923 3848 rpcz_store.cc:276] 0811 18:57:29.336872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.336966 (+ 94us) service_pool.cc:227] Handling call
0811 18:57:30.475499 (+1138533us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x6150000bb424. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.739023 (+263524us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":150257408}
I20260811 18:57:30.740422 4167 raft_consensus.cc:2258] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:30.754613 4163 tablet_bootstrap.cc:654] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:30.752346 3824 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.759037 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap starting.
I20260811 18:57:30.759720 3824 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:30.760021 5286 raft_consensus.cc:2781] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.761066 5286 raft_consensus.cc:686] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.761358 5286 consensus_queue.cc:237] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.762157 5323 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:30.761543 5283 raft_consensus.cc:2781] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:30.762620 5283 raft_consensus.cc:686] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:30.762882 5283 consensus_queue.cc:237] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:30.767268 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 45fde25bcb40493aafb083e161aa6932 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45fde25bcb40493aafb083e161aa6932 already in progress: creating tablet
W20260811 18:57:30.767740 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 221d0d8d69f142a29ec7065a82b3f769 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 221d0d8d69f142a29ec7065a82b3f769 already in progress: creating tablet
W20260811 18:57:30.768015 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 380c1f53b3ac4509a09b25a2800028af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 380c1f53b3ac4509a09b25a2800028af already in progress: creating tablet
W20260811 18:57:30.768285 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8304ac7709ca4392baec4cfa78ef5b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8304ac7709ca4392baec4cfa78ef5b41 already in progress: creating tablet
I20260811 18:57:30.771534 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 14799a6f61a946008acee8a09d050c1e (DEFAULT_TABLE table=default.Table_b752cb750e9447b092ecac296b5f836e [id=839eec1f02c342f3925e487ab8277f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.774039 4167 ts_tablet_manager.cc:1905] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:30.774657 3854 tablet_service.cc:1467] Processing CreateTablet for tablet d21973bc8e514a089c2beb24a686060f (DEFAULT_TABLE table=default.Table_5aa83c98b49c466cb01cdf92154ad384 [id=01ddedd33be14c9ab35e1578361dd733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.775139 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21973bc8e514a089c2beb24a686060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.778621 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec5b2214e0c439ea122d11aa6e6f396 (DEFAULT_TABLE table=default.Table_a677dcb5a43648a49944a18f38fb74fa [id=f3978719f37e4b43847eda64b65f6cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.779191 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec5b2214e0c439ea122d11aa6e6f396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.784471 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14799a6f61a946008acee8a09d050c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.793242 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 03ad1334d0b14ca8adb813e85a429f9e (DEFAULT_TABLE table=default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 [id=76b97a0511a84aa5b2106c833a908799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.798580 3719 tablet_service.cc:1467] Processing CreateTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 (DEFAULT_TABLE table=default.Table_f14f93014a8d464ba0da0bab0ceb9ddc [id=9ea613e25082465f89a34c9da09d3855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.799021 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb1c9787d3f4fd181f86deca28ad160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.802371 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ad1334d0b14ca8adb813e85a429f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.799872 5313 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
#4 0x7fc7bc1faa67 in kudu::RpcServer::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:238:3
I20260811 18:57:30.813551 4169 ts_tablet_manager.cc:1918] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:30.813715 4169 log.cc:1196] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/01d564ad80064912a8ce8e31c1b6c4d2
I20260811 18:57:30.818184 4169 ts_tablet_manager.cc:1939] T 01d564ad80064912a8ce8e31c1b6c4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:30.819489 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:30.831231 3707 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7300b005e34e4c95de73e8d62faa64 (DEFAULT_TABLE table=default.Table_3982bc659a664e949beb16b472873d9a [id=da2587935be0402cad1db22e34c471b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.832329 3714 tablet_service.cc:1467] Processing CreateTablet for tablet e5f47ccc193d48efa15d60a06b148e00 (DEFAULT_TABLE table=default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a [id=3c2bb15d58c34694b03ffc72080c28f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.833302 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fa5a3d1ada554d778d507382c8836133 (DEFAULT_TABLE table=default.Table_e2635923a5784be1839e861c2277e900 [id=f6576859698f477bae1ad63c86d99eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.833601 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc (DEFAULT_TABLE table=default.Table_b485cd94aa7c4642a0b3d0021cef63c4 [id=256649498ffa421b9ef8837c0c25137e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.834403 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 6f804703e20c4214b8b2a31efeaef59e (DEFAULT_TABLE table=default.Table_dbf1868109474dcea3712c10bb4d4f3f [id=f5435385b235444594731609da68bd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.835493 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 73c519528f75423195b6bdf6683fd91b (DEFAULT_TABLE table=default.Table_7df549360f014d629938e3e94659af8a [id=0915677ac44e4a1dad598b19c54633b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.835436 3720 tablet_service.cc:1467] Processing CreateTablet for tablet ab04517e943e410cb65ae11d4abf5286 (DEFAULT_TABLE table=default.Table_32f181a8b481409187abd0dc0a1765ba [id=f9ceba7031af4bd8a4c270368e018851]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.836794 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1800e9e2b064a6fb546b9d33cefa68b consensus state has no leader
I20260811 18:57:30.839717 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 22abc6f8e9ef4c32ab69af31bdaad872
I20260811 18:57:30.840495 3553 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1022) took 1100 ms. Trace:
I20260811 18:57:30.840548 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b (DEFAULT_TABLE table=default.Table_8fb6e34f004c466381fe6ab5b780568f [id=80d3089bffc748fc93707657e47ba89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.840757 3553 rpcz_store.cc:276] 0811 18:57:29.744151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.745486 (+ 1335us) service_pool.cc:227] Handling call
0811 18:57:29.746741 (+ 1255us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.746987 (+ 246us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.747120 (+ 133us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.695640 (+948520us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.839401 (+143761us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.839483 (+ 82us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.839524 (+ 41us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.840350 (+ 826us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.704173 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.704324 (+ 151us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.704332 (+ 8us) write_op.cc:435] Acquired schema lock
0811 18:57:30.704334 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.704455 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.704470 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.704477 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.704615 (+ 138us) tablet.cc:699] Row locks acquired
0811 18:57:30.704619 (+ 4us) write_op.cc:260] PREPARE: finished
0811 18:57:30.704660 (+ 41us) write_op.cc:270] Start()
0811 18:57:30.704702 (+ 42us) write_op.cc:276] Timestamp: P: 1786474650704609 usec, L: 0
0811 18:57:30.704706 (+ 4us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.704907 (+ 201us) log.cc:842] Serialized 725 byte log entry
0811 18:57:30.832468 (+127561us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.834427 (+ 1959us) write_op.cc:301] APPLY: starting
0811 18:57:30.834443 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.834485 (+ 42us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.834592 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.834703 (+ 111us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.834741 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.834751 (+ 10us) write_op.cc:312] APPLY: finished
0811 18:57:30.834820 (+ 69us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.834866 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.834884 (+ 18us) write_op.cc:454] Released schema lock
0811 18:57:30.834891 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.834952 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.834958 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1913,"apply.run_cpu_time_us":586,"apply.run_wall_time_us":584,"num_ops":2,"prepare.queue_time_us":7667,"prepare.run_cpu_time_us":833,"prepare.run_wall_time_us":1317,"replication_time_us":127743,"threads_started":1}]]}
I20260811 18:57:30.842008 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 75ceba7bfc8240268b16c789f07813d1 (DEFAULT_TABLE table=default.Table_13786cfeafbc4312bf84f93b204add4f [id=a11b457b0ad74279949798ccab096915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.843935 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 452ee450778e4d33adfa7e2745b5d970 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:30.848248 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3030c31bc2b24ee296f29b5d5aa402ff
I20260811 18:57:30.848860 3546 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1026) took 1040 ms. Trace:
I20260811 18:57:30.849078 3546 rpcz_store.cc:276] 0811 18:57:29.813255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.813936 (+ 681us) service_pool.cc:227] Handling call
0811 18:57:29.814659 (+ 723us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.814862 (+ 203us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.814953 (+ 91us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.696924 (+881971us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.847930 (+151006us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.848063 (+ 133us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.848120 (+ 57us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.848682 (+ 562us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.705537 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.705626 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.705632 (+ 6us) write_op.cc:435] Acquired schema lock
0811 18:57:30.705634 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.705736 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.705748 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.705753 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.705884 (+ 131us) tablet.cc:699] Row locks acquired
0811 18:57:30.705886 (+ 2us) write_op.cc:260] PREPARE: finished
0811 18:57:30.705923 (+ 37us) write_op.cc:270] Start()
0811 18:57:30.705967 (+ 44us) write_op.cc:276] Timestamp: P: 1786474650705871 usec, L: 0
0811 18:57:30.705972 (+ 5us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.706180 (+ 208us) log.cc:842] Serialized 756 byte log entry
0811 18:57:30.845203 (+139023us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.845570 (+ 367us) write_op.cc:301] APPLY: starting
0811 18:57:30.845585 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.845628 (+ 43us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.845667 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.845735 (+ 68us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.845768 (+ 33us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.845775 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:30.845849 (+ 74us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.845906 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.845926 (+ 20us) write_op.cc:454] Released schema lock
0811 18:57:30.845934 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.845983 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.845987 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":464,"num_ops":2,"prepare.queue_time_us":7893,"prepare.run_cpu_time_us":719,"prepare.run_wall_time_us":1121,"replication_time_us":139202}]]}
I20260811 18:57:30.850142 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f7b6cd7f4ae4ecda49283b677c18e9a" } new_table_name: "default.Table_8708191242b744d0996310157990b329" modify_external_catalogs: true
I20260811 18:57:30.852799 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf54e295be64486a90918a8d2eb8b33 (DEFAULT_TABLE table=default.Table_4b38a0019b5541c495088e96174c8de4 [id=05dc3f3a80494c168dcc8e02a62e2bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.853914 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 14799a6f61a946008acee8a09d050c1e (DEFAULT_TABLE table=default.Table_b752cb750e9447b092ecac296b5f836e [id=839eec1f02c342f3925e487ab8277f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.855845 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6009c9f6bbb0493ebe125120a1a47f3f
I20260811 18:57:30.856660 3550 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:38308 (request call id 1027) took 1033 ms. Trace:
I20260811 18:57:30.856992 3550 rpcz_store.cc:276] 0811 18:57:29.827539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.831654 (+ 4115us) service_pool.cc:227] Handling call
0811 18:57:29.832490 (+ 836us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0811 18:57:29.832685 (+ 195us) catalog_manager.cc:2752] Modifying in-memory table state
0811 18:57:29.832764 (+ 79us) catalog_manager.cc:2761] Locking tablets
0811 18:57:30.698384 (+865620us) catalog_manager.cc:2775] Removing table and tablets from system table
0811 18:57:30.850426 (+152042us) catalog_manager.cc:2793] Removing table from by-name map
0811 18:57:30.855617 (+ 5191us) catalog_manager.cc:2809] Committing in-memory state
0811 18:57:30.855659 (+ 42us) catalog_manager.cc:2813] Aborting table tasks
0811 18:57:30.856523 (+ 864us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:30.706694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0811 18:57:30.706783 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0811 18:57:30.706788 (+ 5us) write_op.cc:435] Acquired schema lock
0811 18:57:30.706790 (+ 2us) tablet.cc:660] Decoding operations
0811 18:57:30.706920 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0811 18:57:30.706932 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0811 18:57:30.706938 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0811 18:57:30.707047 (+ 109us) tablet.cc:699] Row locks acquired
0811 18:57:30.707052 (+ 5us) write_op.cc:260] PREPARE: finished
0811 18:57:30.707104 (+ 52us) write_op.cc:270] Start()
0811 18:57:30.707141 (+ 37us) write_op.cc:276] Timestamp: P: 1786474650707039 usec, L: 0
0811 18:57:30.707147 (+ 6us) op_driver.cc:331] REPLICATION: starting
0811 18:57:30.707347 (+ 200us) log.cc:842] Serialized 756 byte log entry
0811 18:57:30.845319 (+137972us) op_driver.cc:444] REPLICATION: finished
0811 18:57:30.846063 (+ 744us) write_op.cc:301] APPLY: starting
0811 18:57:30.846072 (+ 9us) tablet.cc:1365] starting BulkCheckPresence
0811 18:57:30.846109 (+ 37us) tablet.cc:1368] finished BulkCheckPresence
0811 18:57:30.846129 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0811 18:57:30.846170 (+ 41us) tablet.cc:1381] finished ApplyRowOperation cycle
0811 18:57:30.846200 (+ 30us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0811 18:57:30.846207 (+ 7us) write_op.cc:312] APPLY: finished
0811 18:57:30.846252 (+ 45us) log.cc:842] Serialized 29 byte log entry
0811 18:57:30.846286 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.846299 (+ 13us) write_op.cc:454] Released schema lock
0811 18:57:30.846304 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0811 18:57:30.846370 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0811 18:57:30.846374 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":715,"apply.run_cpu_time_us":350,"apply.run_wall_time_us":350,"num_ops":2,"prepare.queue_time_us":7417,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":1114,"replication_time_us":138155}]]}
I20260811 18:57:30.857928 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec5b2214e0c439ea122d11aa6e6f396 (DEFAULT_TABLE table=default.Table_a677dcb5a43648a49944a18f38fb74fa [id=f3978719f37e4b43847eda64b65f6cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.858716 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 03ad1334d0b14ca8adb813e85a429f9e (DEFAULT_TABLE table=default.Table_e8ea44ff9ff34e76b5f970a2b9568ed3 [id=76b97a0511a84aa5b2106c833a908799]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.858649 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_03b64ac0b80f46288fcf2b34397efcaa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.859426 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03b64ac0b80f46288fcf2b34397efcaa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.859367 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 (DEFAULT_TABLE table=default.Table_f14f93014a8d464ba0da0bab0ceb9ddc [id=9ea613e25082465f89a34c9da09d3855]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.859977 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 531ba05901734d6b93bc2be506e8ec5b (DEFAULT_TABLE table=default.Table_495f858f5eec496680f982bccd44415d [id=40aeb51344cf4308ab20fa39bffbfecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.860613 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7300b005e34e4c95de73e8d62faa64 (DEFAULT_TABLE table=default.Table_3982bc659a664e949beb16b472873d9a [id=da2587935be0402cad1db22e34c471b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.862227 3857 tablet_service.cc:1467] Processing CreateTablet for tablet e5f47ccc193d48efa15d60a06b148e00 (DEFAULT_TABLE table=default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a [id=3c2bb15d58c34694b03ffc72080c28f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.862710 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 90fde17971ea49aaad212a5836661b08
I20260811 18:57:30.863659 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 73c519528f75423195b6bdf6683fd91b (DEFAULT_TABLE table=default.Table_7df549360f014d629938e3e94659af8a [id=0915677ac44e4a1dad598b19c54633b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.863811 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aadee0d920fc4be5aa7f965bc9db93e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.864621 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aadee0d920fc4be5aa7f965bc9db93e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.866170 4163 tablet_bootstrap.cc:492] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:30.866614 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7d747f03ad234805a4ea4ef60840d426
I20260811 18:57:30.866309 4163 ts_tablet_manager.cc:1397] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.178s user 0.007s sys 0.004s
I20260811 18:57:30.867231 3549 catalog_manager.cc:6565] ReplaceTablet: tablet 7d747f03ad234805a4ea4ef60840d426 deleted and replaced by tablet 97d5c52e361c4f08904475acf62dae2b
I20260811 18:57:30.869204 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b79839172cff41c3a8235675d44c21fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.870071 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b79839172cff41c3a8235675d44c21fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.870931 3870 tablet_service.cc:1467] Processing CreateTablet for tablet fa5a3d1ada554d778d507382c8836133 (DEFAULT_TABLE table=default.Table_e2635923a5784be1839e861c2277e900 [id=f6576859698f477bae1ad63c86d99eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.872613 4163 raft_consensus.cc:348] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.872942 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ddf22cbb415c485ebc1407d9104b715d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.873750 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddf22cbb415c485ebc1407d9104b715d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.875420 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e (table default.Table_f5f50e8f17e04760a61e10c54d1aa657 [id=de9b6daef0ed416aa79b0dc0f4b02c2d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cc3bf1f2d39f4a1cb3bd071f45a0a39e consensus state has no leader
I20260811 18:57:30.877728 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d4fb4116244d9e8c08ce623bea3a2e" } new_table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" modify_external_catalogs: true
I20260811 18:57:30.873858 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc (DEFAULT_TABLE table=default.Table_b485cd94aa7c4642a0b3d0021cef63c4 [id=256649498ffa421b9ef8837c0c25137e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.875149 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6f804703e20c4214b8b2a31efeaef59e (DEFAULT_TABLE table=default.Table_dbf1868109474dcea3712c10bb4d4f3f [id=f5435385b235444594731609da68bd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.875748 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ab04517e943e410cb65ae11d4abf5286 (DEFAULT_TABLE table=default.Table_32f181a8b481409187abd0dc0a1765ba [id=f9ceba7031af4bd8a4c270368e018851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.876636 3861 tablet_service.cc:1467] Processing CreateTablet for tablet 1474691d8bec4698981ff71ef23ff857 (DEFAULT_TABLE table=default.Table_b2be35bff506498281ab88f9cbb55de8 [id=32ffcc8e6240470683988a9c96d21573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.877334 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b702aef88e124443888002d8ccd47b2f (DEFAULT_TABLE table=default.Table_cef66a1c435645d6822188ec4776526b [id=73c0679bc54b446a927b0f8a96cc4afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.877889 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 37d6d857025e4541bd9f480d0a3f4d45 (DEFAULT_TABLE table=default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd [id=e3136d15218f463ab30849b81869fc5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.878039 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 30d44d69b76545b8879bdfc0e6b7008e (DEFAULT_TABLE table=default.Table_ddc87769613d45caa57f2df0032dc979 [id=c2eaa36cde1c45c096e0d584a4448d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.878861 4163 raft_consensus.cc:374] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:30.880714 3547 catalog_manager.cc:5495] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:30.881280 3862 tablet_service.cc:1467] Processing CreateTablet for tablet afa57ed239ac4afebe517cc48d879645 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.881734 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:30.880438 4163 raft_consensus.cc:729] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:30.882920 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8ad9f329a9c449e4ade20157c2601ee8" } new_table_name: "default.Table_605185b9f67e456ba194522b24112e39" modify_external_catalogs: true
I20260811 18:57:30.885205 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a9efd2f19d74a68a8536fd057377738 (table default.Table_e03268eb26a144ba8b53cbd960628036 [id=64009634513f46eeae6442cfadb7b0f5]). Expected version 3 got 0
I20260811 18:57:30.884346 4163 consensus_queue.cc:260] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:30.886318 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8b3822f191c44112a69c6300fa97a3ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.888021 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b3822f191c44112a69c6300fa97a3ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.889729 3953 kernel_stack_watchdog.cc:198] Thread 3966 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 22272ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a5e57 __sanitizer::internal_sched_yield() at ??:?
@ 0x81ec1d __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7b9d2ee4b kudu::rpc::ReactorThread::StartConnectionNegotiation() at ??:0
@ 0x7fc7b9d3050f kudu::rpc::ReactorThread::FindOrStartConnection() at ??:0
@ 0x7fc7b9d2fd32 kudu::rpc::ReactorThread::AssignOutboundCall() at ??:0
@ 0x7fc7b9d3f74e kudu::rpc::AssignOutboundCallTask::Run() at ??:0
@ 0x7fc7b9d2de8d kudu::rpc::ReactorThread::AsyncHandler() at ??:0
@ 0x7fc7b9d40075 ev::base<>::method_thunk<>() at ??:0
@ 0x7fc7b1b19443 ev_invoke_pending at ??:0
@ 0x7fc7b9d2b365 kudu::rpc::ReactorThread::InvokePendingCb() at ??:0
I20260811 18:57:30.892302 4163 ts_tablet_manager.cc:1428] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
W20260811 18:57:30.883718 3824 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.576s (SENT)
I20260811 18:57:30.894470 3824 leader_election.cc:302] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:30.895412 3824 raft_consensus.cc:2670] T 460e03cc14444c83b6c08ca194ca5a8d P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:30.896371 3849 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37334 (request call id 147) took 1190 ms. Trace:
I20260811 18:57:30.899133 3849 rpcz_store.cc:276] 0811 18:57:29.705499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.705566 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:30.573916 (+868350us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x6150000977a4. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.715444 (+141528us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x6150000bb424. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d1460a630 00007f4d181e2c7f 00007f4d12d4e016 00007f4d12d4dd45 00007f4d0fc3e9be 00007f4d0fc3d79e 00007f4d0fc43b25 00007f4d0fc4454f 00007f4d0fc44378 00000000009edb7c
0811 18:57:30.896249 (+180805us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":381380224}
I20260811 18:57:30.902524 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7300b005e34e4c95de73e8d62faa64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.904371 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f47ccc193d48efa15d60a06b148e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.904814 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 1474691d8bec4698981ff71ef23ff857 (DEFAULT_TABLE table=default.Table_b2be35bff506498281ab88f9cbb55de8 [id=32ffcc8e6240470683988a9c96d21573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.905419 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1474691d8bec4698981ff71ef23ff857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.906782 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ceba7bfc8240268b16c789f07813d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.907207 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb1c9787d3f4fd181f86deca28ad160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.910697 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531ba05901734d6b93bc2be506e8ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.911584 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.912918 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d44d69b76545b8879bdfc0e6b7008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913270 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f804703e20c4214b8b2a31efeaef59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913599 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1474691d8bec4698981ff71ef23ff857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.913921 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec5b2214e0c439ea122d11aa6e6f396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.914186 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab04517e943e410cb65ae11d4abf5286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.914499 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f47ccc193d48efa15d60a06b148e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.918078 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c519528f75423195b6bdf6683fd91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.918444 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d6d857025e4541bd9f480d0a3f4d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.922493 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b06fa572045f3a5fc5a0b123d77cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.953424 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_04e3e316fa374868a4be1497035d91e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.961050 3678 leader_election.cc:334] T e5fd9fce095e4a769dbb38c7903359cc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.673s (SENT)
I20260811 18:57:30.961321 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 37d6d857025e4541bd9f480d0a3f4d45 (DEFAULT_TABLE table=default.Table_3990d13c5ecf4f54b1a8535d5f9ae7fd [id=e3136d15218f463ab30849b81869fc5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.961308 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 531ba05901734d6b93bc2be506e8ec5b (DEFAULT_TABLE table=default.Table_495f858f5eec496680f982bccd44415d [id=40aeb51344cf4308ab20fa39bffbfecb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.959318 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04e3e316fa374868a4be1497035d91e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.979904 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa5a3d1ada554d778d507382c8836133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.981485 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 30d44d69b76545b8879bdfc0e6b7008e (DEFAULT_TABLE table=default.Table_ddc87769613d45caa57f2df0032dc979 [id=c2eaa36cde1c45c096e0d584a4448d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:30.983906 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d6d857025e4541bd9f480d0a3f4d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:30.983841 3709 tablet_service.cc:1467] Processing CreateTablet for tablet b702aef88e124443888002d8ccd47b2f (DEFAULT_TABLE table=default.Table_cef66a1c435645d6822188ec4776526b [id=73c0679bc54b446a927b0f8a96cc4afa]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:30.984068 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 80020.1us
I20260811 18:57:30.994205 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bf4fa780a3ba4824b3317ae7e9a379fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:30.995007 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa5a3d1ada554d778d507382c8836133. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:30.995400 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf4fa780a3ba4824b3317ae7e9a379fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:30.995584 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 547f6c33d6414bcfa3eeff20cb9a0302 consensus state has no leader
I20260811 18:57:30.996277 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_25e47ee54a2742a7839d23fc2adce361"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:30.996888 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25e47ee54a2742a7839d23fc2adce361 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:30.999069 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c0f68d33e1b14144b34c17e3c7201408
W20260811 18:57:30.994227 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7b38d3b9570467a9128bd7fe73acf2f consensus state has no leader
W20260811 18:57:31.001310 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99b788d7e78c4945bb82fe86597aa634 consensus state has no leader
I20260811 18:57:31.003870 3717 tablet_service.cc:1467] Processing CreateTablet for tablet afa57ed239ac4afebe517cc48d879645 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.004701 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf8eed41af347f6af7435854850a617 (DEFAULT_TABLE table=default.Table_b8dd0414c4784bbea975b8d3f1491cd6 [id=f2733be482bf4c9a97103d57fd5804f3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:31.005144 3678 leader_election.cc:334] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.134s (SENT)
I20260811 18:57:31.005451 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d44d69b76545b8879bdfc0e6b7008e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.012465 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3bf8eed41af347f6af7435854850a617 (DEFAULT_TABLE table=default.Table_b8dd0414c4784bbea975b8d3f1491cd6 [id=f2733be482bf4c9a97103d57fd5804f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.014642 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ad1334d0b14ca8adb813e85a429f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.020716 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.023564 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa57ed239ac4afebe517cc48d879645. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.024734 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.025267 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b702aef88e124443888002d8ccd47b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.026419 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab04517e943e410cb65ae11d4abf5286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.026516 3678 leader_election.cc:302] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:31.027262 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf54e295be64486a90918a8d2eb8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.028707 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7300b005e34e4c95de73e8d62faa64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.029947 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fde17971ea49aaad212a5836661b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.030877 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d747f03ad234805a4ea4ef60840d426 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d5c52e361c4f08904475acf62dae2b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.032315 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14799a6f61a946008acee8a09d050c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.027493 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 101994cc708a4fadb46fc8c5915a5587 (DEFAULT_TABLE table=default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.032982 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101994cc708a4fadb46fc8c5915a5587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.025142 3861 tablet_service.cc:1514] Processing DeleteTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.035370 3678 raft_consensus.cc:2670] T c2247194db704f9abd54a8a0df1ca566 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:31.039260 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c519528f75423195b6bdf6683fd91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.042121 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 622) took 5226 ms. Trace:
I20260811 18:57:31.044178 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0322ec9fd9f24338bb9196b099f56643" } modify_external_catalogs: true
I20260811 18:57:31.044572 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0322ec9fd9f24338bb9196b099f56643" } modify_external_catalogs: true
I20260811 18:57:31.045691 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c284a387f502480c8faa36ea0e40a8f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.046674 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c284a387f502480c8faa36ea0e40a8f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.047417 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_313ce0e9dd1042e2827c0eebceaf3729" } new_table_name: "default.Table_febb766a2cc1415b8977a6539a9e58eb" modify_external_catalogs: true
I20260811 18:57:31.052551 4169 rpcz_store.cc:276] 0811 18:57:25.815755 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.815884 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:30.733504 (+4917620us) ts_tablet_manager.cc:1145] Deleting tablet 01d564ad80064912a8ce8e31c1b6c4d2
0811 18:57:30.733513 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:30.765992 (+ 32479us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:30.813437 (+ 47445us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.042089 (+228652us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":138102,"tablet-delete.queue_time_us":4917279}
I20260811 18:57:31.053054 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf8eed41af347f6af7435854850a617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.055383 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b3822f191c44112a69c6300fa97a3ac" } modify_external_catalogs: true
I20260811 18:57:31.055748 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b3822f191c44112a69c6300fa97a3ac" } modify_external_catalogs: true
I20260811 18:57:31.058035 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1cdea1db192a43f4b210579b0813e33a" } new_table_name: "default.Table_bf526916d0504506a3e70cc658d22416" modify_external_catalogs: true
I20260811 18:57:31.063668 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:31.068710 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 101994cc708a4fadb46fc8c5915a5587 (DEFAULT_TABLE table=default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2 [id=1a843530cf6744b184807b679f56bf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.069972 4162 tablet_bootstrap.cc:654] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.071115 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e1af5a92fcee4bc4898572e55da333b5" } modify_external_catalogs: true
I20260811 18:57:31.071558 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e1af5a92fcee4bc4898572e55da333b5" } modify_external_catalogs: true
W20260811 18:57:31.072041 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 52461.4us
I20260811 18:57:31.073907 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531ba05901734d6b93bc2be506e8ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.074185 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5b06fa572045f3a5fc5a0b123d77cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.074867 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f804703e20c4214b8b2a31efeaef59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.083019 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 22abc6f8e9ef4c32ab69af31bdaad872 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.084055 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 3030c31bc2b24ee296f29b5d5aa402ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.084411 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 6009c9f6bbb0493ebe125120a1a47f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:31.087220 3678 leader_election.cc:334] T e51a08e6dbf04e3fa58689ee0cd45db3 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.345s (SENT)
I20260811 18:57:31.088115 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_160e44b6b513432c89082037a071fbb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.088714 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 101994cc708a4fadb46fc8c5915a5587. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.089251 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_160e44b6b513432c89082037a071fbb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.091264 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" } modify_external_catalogs: true
I20260811 18:57:31.091693 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8d1dc3a55b5f4fe6afb1c0dc4255173e" } modify_external_catalogs: true
I20260811 18:57:31.095220 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d747f03ad234805a4ea4ef60840d426 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d5c52e361c4f08904475acf62dae2b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.111922 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b702aef88e124443888002d8ccd47b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.122439 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bf8eed41af347f6af7435854850a617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.123870 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa57ed239ac4afebe517cc48d879645. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.132928 3678 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.941s (SENT)
W20260811 18:57:31.093063 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 141587us
I20260811 18:57:31.135876 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 01d564ad80064912a8ce8e31c1b6c4d2 (table default.Table_176c852b84c14616bc0ea0f110671639 [id=1bac61c204ef425996f55cba1df620f0]) successfully deleted
I20260811 18:57:31.137151 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 297988c33d4e42e886fb6f1bd37005a5 (DEFAULT_TABLE table=default.Table_16d7946e87be4700a338a7744c866d5e [id=db85e36a578f47638611ba09c924fa25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.137259 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fde17971ea49aaad212a5836661b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.137929 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297988c33d4e42e886fb6f1bd37005a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.138895 3710 tablet_service.cc:1467] Processing CreateTablet for tablet e7b6f913d82c4ea9830e6e680de3dda1 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.147728 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b6f913d82c4ea9830e6e680de3dda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.147599 3705 tablet_service.cc:1467] Processing CreateTablet for tablet c0f68d33e1b14144b34c17e3c7201408 (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.151103 5334 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.151556 5334 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.152798 5334 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.136160 5300 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:31.157831 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 810b9bdc330e40619f45c70e57724da0 consensus state has no leader
#5 0x7fc7bc21b0b7 in kudu::server::ServerBase::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:1171:23
I20260811 18:57:31.163825 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f68d33e1b14144b34c17e3c7201408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.167799 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f68d33e1b14144b34c17e3c7201408 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.182698 3546 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 06b472ad782c4f9fa9a99a5ae8582ec5
W20260811 18:57:31.189913 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60091bd891a94cc4aac6ef47673a4ee5 has no consensus state
I20260811 18:57:31.196255 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.197171 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.198855 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 99e9474ef8814514832f0e48c2b3495f from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.201282 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_016d0800749f4083ad87c2a640e058c8" } new_table_name: "default.Table_19b497f543db41bd9abc6e46da338cd5" modify_external_catalogs: true
I20260811 18:57:31.168520 5300 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.206836 4169 tablet_replica.cc:330] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.206770 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0036c28f59cb4f229da243389602bbd9" } modify_external_catalogs: true
I20260811 18:57:31.207098 4169 raft_consensus.cc:2229] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.207059 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0036c28f59cb4f229da243389602bbd9" } modify_external_catalogs: true
I20260811 18:57:31.207471 4169 raft_consensus.cc:2258] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.207470 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3de36d9c6d4542be9ee2c873f0bfcfca" } new_table_name: "default.Table_0a6176c962034100a92814898fb6e4d0" modify_external_catalogs: true
I20260811 18:57:31.208168 4169 ts_tablet_manager.cc:1905] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.210942 4163 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.217824 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:31.218521 3676 leader_election.cc:341] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.229998 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 01a5fdab68694761b7493b42a0bf7c5b
I20260811 18:57:31.231104 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.231828 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.216516 5300 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:31.239212 4162 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.239418 4162 ts_tablet_manager.cc:1397] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.530s user 0.008s sys 0.007s
I20260811 18:57:31.242743 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c944d5cfd9b3405b8151abfc9f2fd998
I20260811 18:57:31.243688 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f0090725e77446588856c07015fda936"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.244457 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0090725e77446588856c07015fda936 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.259936 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.263082 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.268101 4162 raft_consensus.cc:348] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.268952 4162 raft_consensus.cc:374] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.269208 4162 raft_consensus.cc:729] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.269476 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4e0747ef88574c68ae074732cb7095b9
I20260811 18:57:31.269611 4162 consensus_queue.cc:260] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.271613 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet c944d5cfd9b3405b8151abfc9f2fd998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.272387 4162 ts_tablet_manager.cc:1428] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.033s user 0.000s sys 0.003s
I20260811 18:57:31.274871 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_98f9637c9f7a47559887a4c73c52daea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.275672 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98f9637c9f7a47559887a4c73c52daea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.279943 3852 tablet_service.cc:1467] Processing CreateTablet for tablet e7b6f913d82c4ea9830e6e680de3dda1 (DEFAULT_TABLE table=default.Table_3a85d24a6d8d4b038094e6e7607ddbd1 [id=b6989fc5f9e44f10bdbd0b8bb7d5e071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.280301 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 297988c33d4e42e886fb6f1bd37005a5 (DEFAULT_TABLE table=default.Table_16d7946e87be4700a338a7744c866d5e [id=db85e36a578f47638611ba09c924fa25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.285143 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7e8ba858b0c44d588499e1463673957" } new_table_name: "default.Table_1314176a6f3346d7a808360af2c1cfd2" modify_external_catalogs: true
I20260811 18:57:31.285964 4162 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.288376 4162 tablet_bootstrap.cc:654] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.289148 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c0f68d33e1b14144b34c17e3c7201408 (DEFAULT_TABLE table=default.Table_65d5ebf213ce401984be01bac1813640 [id=0eb6683481c54e688597f647329853b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.289150 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7b6f913d82c4ea9830e6e680de3dda1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.290122 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a5fdab68694761b7493b42a0bf7c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.290323 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet c0f68d33e1b14144b34c17e3c7201408 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:30 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.290994 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297988c33d4e42e886fb6f1bd37005a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.291594 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0f68d33e1b14144b34c17e3c7201408. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.289182 3824 leader_election.cc:334] T 94943cfbe41a47b3a36c81957a3adaeb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.918s (SENT)
W20260811 18:57:31.293162 3824 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.395s (SENT)
I20260811 18:57:31.294610 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0747ef88574c68ae074732cb7095b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.319373 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0747ef88574c68ae074732cb7095b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.319419 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.319649 3743 raft_consensus.cc:2454] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.321945 4167 ts_tablet_manager.cc:1918] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.322088 4167 log.cc:1196] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c08436f33db043b1bc6395cd92102e59
I20260811 18:57:31.324996 3824 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.326136 4167 ts_tablet_manager.cc:1939] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.326534 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 677) took 4806 ms. Trace:
I20260811 18:57:31.326622 4167 rpcz_store.cc:276] 0811 18:57:26.520485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.520662 (+ 177us) service_pool.cc:227] Handling call
0811 18:57:30.685374 (+4164712us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:30.685385 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.218876 (+533491us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.321835 (+102959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.326516 (+ 4681us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164510}
I20260811 18:57:31.327029 4167 tablet_replica.cc:330] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.327205 4167 raft_consensus.cc:2229] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.327804 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c08436f33db043b1bc6395cd92102e59 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
I20260811 18:57:31.328984 4163 tablet_bootstrap.cc:654] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.330184 4167 raft_consensus.cc:2258] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.330778 4167 ts_tablet_manager.cc:1905] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.332149 5334 raft_consensus.cc:2781] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.332278 5334 raft_consensus.cc:482] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.332376 5334 raft_consensus.cc:3037] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.339773 5334 raft_consensus.cc:504] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.354269 5334 leader_election.cc:288] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.355221 4169 ts_tablet_manager.cc:1918] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.355383 4169 log.cc:1196] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/67e63ee9b9fd4b12ae791b9d5bc2e648
I20260811 18:57:31.355859 4169 ts_tablet_manager.cc:1939] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.356282 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 623) took 5436 ms. Trace:
I20260811 18:57:31.356390 4169 rpcz_store.cc:276] 0811 18:57:25.920076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.920285 (+ 209us) service_pool.cc:227] Handling call
0811 18:57:31.052919 (+5132634us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:31.204263 (+151344us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.323522 (+119259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.330554 (+ 7032us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.356258 (+ 25704us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5107381}
I20260811 18:57:31.356997 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 (table default.Table_bbeefd35da1a4d1584009f5edafc4642 [id=c7c762aa4e0d45ee8e78b5f7a03bcc5e]) successfully deleted
I20260811 18:57:31.359787 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_e03268eb26a144ba8b53cbd960628036 [id=64009634513f46eeae6442cfadb7b0f5] alter complete (version 3)
I20260811 18:57:31.360834 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 624) took 5414 ms. Trace:
I20260811 18:57:31.360932 4169 rpcz_store.cc:276] 0811 18:57:25.946900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.946959 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:31.360693 (+5413734us) ts_tablet_manager.cc:1145] Deleting tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa
0811 18:57:31.360705 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.360812 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5412354}
I20260811 18:57:31.361440 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 625) took 5401 ms. Trace:
I20260811 18:57:31.361522 4169 rpcz_store.cc:276] 0811 18:57:25.959568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.959646 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:31.361326 (+5401680us) ts_tablet_manager.cc:1145] Deleting tablet 84e152c74f33463e83843b92c0783803
0811 18:57:31.361337 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.361423 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401444}
I20260811 18:57:31.362035 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 627) took 5401 ms. Trace:
I20260811 18:57:31.362128 4169 rpcz_store.cc:276] 0811 18:57:25.960158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960237 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:31.361925 (+5401688us) ts_tablet_manager.cc:1145] Deleting tablet 86d690ec09524248bc147ad7408e2ee6
0811 18:57:31.361938 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.362017 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401495}
W20260811 18:57:31.363068 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bf26b079c5b4e5f9a6d74e73dd2c9fa already in progress: creating tablet
W20260811 18:57:31.363430 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 84e152c74f33463e83843b92c0783803 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84e152c74f33463e83843b92c0783803 already in progress: creating tablet
I20260811 18:57:31.363420 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f335e50ae57547ed80b144129faaffbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:11 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.364074 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.364285 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 626) took 5402 ms. Trace:
I20260811 18:57:31.364384 4169 rpcz_store.cc:276] 0811 18:57:25.960055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960356 (+ 301us) service_pool.cc:227] Handling call
0811 18:57:31.362521 (+5402165us) ts_tablet_manager.cc:1145] Deleting tablet 649cb98f553a418b911e307fdaf49917
0811 18:57:31.362532 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.362616 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5401984}
I20260811 18:57:31.364940 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 628) took 5404 ms. Trace:
I20260811 18:57:31.365121 4169 rpcz_store.cc:276] 0811 18:57:25.960296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.960472 (+ 176us) service_pool.cc:227] Handling call
0811 18:57:31.364800 (+5404328us) ts_tablet_manager.cc:1145] Deleting tablet 6d28c486e408451a990ae3905ef3c2ff
0811 18:57:31.364817 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.364919 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5404045}
I20260811 18:57:31.365725 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 629) took 5396 ms. Trace:
I20260811 18:57:31.365859 4169 rpcz_store.cc:276] 0811 18:57:25.969588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:25.973583 (+ 3995us) service_pool.cc:227] Handling call
0811 18:57:31.365616 (+5392033us) ts_tablet_manager.cc:1145] Deleting tablet b7b48a35acf1486bbb63680ca5da31dd
0811 18:57:31.365628 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.365707 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5390916}
W20260811 18:57:31.366539 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 649cb98f553a418b911e307fdaf49917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 649cb98f553a418b911e307fdaf49917 already in progress: creating tablet
W20260811 18:57:31.366881 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6d28c486e408451a990ae3905ef3c2ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d28c486e408451a990ae3905ef3c2ff already in progress: creating tablet
W20260811 18:57:31.367142 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b7b48a35acf1486bbb63680ca5da31dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b48a35acf1486bbb63680ca5da31dd already in progress: creating tablet
I20260811 18:57:31.369416 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5b76a052465c4749a059cbf9171ab6be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.369917 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02808605ee6c4634af47fc3fd57e9cd8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.370158 3743 raft_consensus.cc:3037] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:31.370671 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b76a052465c4749a059cbf9171ab6be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.372800 3743 raft_consensus.cc:2454] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.373742 3824 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.374250 5334 raft_consensus.cc:2781] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.374428 5334 raft_consensus.cc:686] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.374673 5334 consensus_queue.cc:237] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.375128 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 761902f7366d43ffbde226a0ade2e188
I20260811 18:57:31.375174 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 99e9474ef8814514832f0e48c2b3495f
I20260811 18:57:31.375707 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 99e9474ef8814514832f0e48c2b3495f deleted and replaced by tablet bcfad3da366e4554b06bfa51a2535419
I20260811 18:57:31.376058 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 3dd4ec30acde4585a1c3451927217cb3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.376539 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e9474ef8814514832f0e48c2b3495f with delete_type TABLET_DATA_DELETED (replaced by tablet bcfad3da366e4554b06bfa51a2535419) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.377559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 641) took 5267 ms. Trace:
W20260811 18:57:31.377609 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e4f4cc4dace944be894fe12690b18ccb consensus state has no leader
I20260811 18:57:31.377664 4169 rpcz_store.cc:276] 0811 18:57:26.110680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.110738 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:31.377405 (+5266667us) ts_tablet_manager.cc:1145] Deleting tablet ea4ad482bde44d6bab5ec809054aaebd
0811 18:57:31.377418 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.377531 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5262932}
I20260811 18:57:31.378424 4169 tablet_replica.cc:330] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.378285 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_96bf47f7003740cba4c66f780597a121"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.379182 4169 raft_consensus.cc:2229] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:31.379405 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96bf47f7003740cba4c66f780597a121 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.379521 4169 raft_consensus.cc:2258] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.382655 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 761902f7366d43ffbde226a0ade2e188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.383337 4169 ts_tablet_manager.cc:1905] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.383510 5283 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.383636 5283 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.384248 5283 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.386502 5313 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.387168 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e9474ef8814514832f0e48c2b3495f with delete_type TABLET_DATA_DELETED (replaced by tablet bcfad3da366e4554b06bfa51a2535419) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.387522 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 761902f7366d43ffbde226a0ade2e188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.388558 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ea4ad482bde44d6bab5ec809054aaebd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea4ad482bde44d6bab5ec809054aaebd already in progress: creating tablet
I20260811 18:57:31.388818 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_619d1e7c0e384107962b41e86e45c45b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.389784 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_619d1e7c0e384107962b41e86e45c45b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.392694 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.392987 3744 raft_consensus.cc:2454] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.393877 3824 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.394220 5283 raft_consensus.cc:2781] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.394327 5283 raft_consensus.cc:482] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.394438 5283 raft_consensus.cc:3037] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.397310 5283 raft_consensus.cc:504] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.398873 5283 leader_election.cc:288] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.399890 4163 tablet_bootstrap.cc:492] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:31.400954 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 60743.4us
I20260811 18:57:31.401727 4163 ts_tablet_manager.cc:1397] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.197s user 0.003s sys 0.007s
I20260811 18:57:31.403328 4163 raft_consensus.cc:348] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.406936 4163 raft_consensus.cc:374] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.407591 4163 raft_consensus.cc:729] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.407626 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.408340 4163 consensus_queue.cc:260] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.409806 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f335e50ae57547ed80b144129faaffbb" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.410566 3743 raft_consensus.cc:3037] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.409301 5361 tablet.cc:1722] T fb51e419dd24459caf3ba3adfbbd4517 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:31.411610 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73e15954a1604189af634a727cb48690 consensus state has no leader
I20260811 18:57:31.415506 4163 ts_tablet_manager.cc:1428] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:31.430784 4163 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.425846 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:31.441107 3678 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.285s (SENT)
I20260811 18:57:31.441912 3678 leader_election.cc:302] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:31.442186 3678 raft_consensus.cc:2670] T ef393c341bb54baf87176db5b5254ec7 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:31.444455 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9183f700c0544cd5942abebaaa8a121f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.445653 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9183f700c0544cd5942abebaaa8a121f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.451563 5313 tablet.cc:1722] T 23f41a2638e74ba19c1c527b9fc796df P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.458211 4163 tablet_bootstrap.cc:654] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.448707 3743 raft_consensus.cc:2454] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.465565 3824 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.465880 5334 raft_consensus.cc:2781] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.466024 5334 raft_consensus.cc:686] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.466472 5334 consensus_queue.cc:237] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.470428 5361 tablet.cc:1722] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:31.471313 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2698f49dd4b64fce9a411b1c3307abf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.472277 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2698f49dd4b64fce9a411b1c3307abf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.477109 3678 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.631s (SENT)
I20260811 18:57:31.477592 4167 ts_tablet_manager.cc:1918] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.480825 4163 tablet_bootstrap.cc:492] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.480999 4163 ts_tablet_manager.cc:1397] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.002s
I20260811 18:57:31.481752 4167 log.cc:1196] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:31.481980 4163 raft_consensus.cc:348] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.482223 4163 raft_consensus.cc:374] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.482293 4163 raft_consensus.cc:729] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.482609 4163 consensus_queue.cc:260] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.482756 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8214722ffbdc49bf98d5aa539aebed52" } new_table_name: "default.Table_6936ad30157d4323baa32612df13a29d" modify_external_catalogs: true
I20260811 18:57:31.483357 4163 ts_tablet_manager.cc:1428] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:31.483656 4163 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.484130 4167 ts_tablet_manager.cc:1939] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.488638 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 678) took 4781 ms. Trace:
I20260811 18:57:31.461525 4169 ts_tablet_manager.cc:1918] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.492537 4169 log.cc:1196] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ff1be026ca404eada53f60f257ea367c
I20260811 18:57:31.493014 4169 ts_tablet_manager.cc:1939] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.493400 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 649) took 5339 ms. Trace:
I20260811 18:57:31.493501 4169 rpcz_store.cc:276] 0811 18:57:26.153766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.153893 (+ 127us) service_pool.cc:227] Handling call
0811 18:57:31.378372 (+5224479us) ts_tablet_manager.cc:1145] Deleting tablet ff1be026ca404eada53f60f257ea367c
0811 18:57:31.378384 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.453563 (+ 75179us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.461411 (+ 7848us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.493382 (+ 31971us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5222934}
I20260811 18:57:31.493922 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 648) took 5340 ms. Trace:
I20260811 18:57:31.494012 4169 rpcz_store.cc:276] 0811 18:57:26.153613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.153853 (+ 240us) service_pool.cc:227] Handling call
0811 18:57:31.493808 (+5339955us) ts_tablet_manager.cc:1145] Deleting tablet c2247194db704f9abd54a8a0df1ca566
0811 18:57:31.493821 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.493903 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5338164}
I20260811 18:57:31.494410 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 654) took 5288 ms. Trace:
I20260811 18:57:31.497661 4163 tablet_bootstrap.cc:654] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:31.498924 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:31.499506 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_533afa7d5e834d5cb281ddc9619575f8" } modify_external_catalogs: true
I20260811 18:57:31.499871 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_533afa7d5e834d5cb281ddc9619575f8" } modify_external_catalogs: true
I20260811 18:57:31.494494 4169 rpcz_store.cc:276] 0811 18:57:26.206183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.206255 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:31.494281 (+5288026us) ts_tablet_manager.cc:1145] Deleting tablet 54fa32f8fa1144c4bd9dcd00a550001a
0811 18:57:31.494291 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.494392 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5283592}
I20260811 18:57:31.501606 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_378dd90b6af541b8ab95d9f19681a03f" } new_table_name: "default.Table_6090c727ddfa494594649314a3fdf57e" modify_external_catalogs: true
I20260811 18:57:31.502136 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 655) took 5295 ms. Trace:
I20260811 18:57:31.502246 4169 rpcz_store.cc:276] 0811 18:57:26.206304 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.210707 (+ 4403us) service_pool.cc:227] Handling call
0811 18:57:31.501986 (+5291279us) ts_tablet_manager.cc:1145] Deleting tablet b16cb4c1796147fbb10109f52071505e
0811 18:57:31.502002 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.502117 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5268892}
I20260811 18:57:31.504639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 653) took 5298 ms. Trace:
I20260811 18:57:31.505928 4169 rpcz_store.cc:276] 0811 18:57:26.206056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.206157 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:31.504498 (+5298341us) ts_tablet_manager.cc:1145] Deleting tablet 1d81e93739f44d9e99b6ebc647e1d0e4
0811 18:57:31.504513 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.504617 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5263880}
I20260811 18:57:31.507205 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 656) took 5236 ms. Trace:
I20260811 18:57:31.508425 4169 rpcz_store.cc:276] 0811 18:57:26.270719 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.270781 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:31.507077 (+5236296us) ts_tablet_manager.cc:1145] Deleting tablet 1811e5f62a714f8388460dfcb1717e81
0811 18:57:31.507089 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.507187 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5236031}
I20260811 18:57:31.509730 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 657) took 4910 ms. Trace:
I20260811 18:57:31.510908 4169 rpcz_store.cc:276] 0811 18:57:26.599056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.599617 (+ 561us) service_pool.cc:227] Handling call
0811 18:57:31.509600 (+4909983us) ts_tablet_manager.cc:1145] Deleting tablet 335fc15ea75d483990e6ae5a5872682f
0811 18:57:31.509614 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.509711 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4907122}
I20260811 18:57:31.494073 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ff1be026ca404eada53f60f257ea367c (table default.Table_3441d374bdb04681b2bd464f6ed910e2 [id=9c2695c4a638486cb9008ac8a59b68b4]) successfully deleted
W20260811 18:57:31.514632 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c2247194db704f9abd54a8a0df1ca566 because the tablet was not found. No further retry: Not found: Tablet not found: c2247194db704f9abd54a8a0df1ca566
W20260811 18:57:31.515009 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 54fa32f8fa1144c4bd9dcd00a550001a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54fa32f8fa1144c4bd9dcd00a550001a already in progress: creating tablet
W20260811 18:57:31.515278 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b16cb4c1796147fbb10109f52071505e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16cb4c1796147fbb10109f52071505e already in progress: creating tablet
W20260811 18:57:31.515518 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1d81e93739f44d9e99b6ebc647e1d0e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d81e93739f44d9e99b6ebc647e1d0e4 already in progress: creating tablet
W20260811 18:57:31.515800 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1811e5f62a714f8388460dfcb1717e81 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1811e5f62a714f8388460dfcb1717e81 already in progress: creating tablet
W20260811 18:57:31.516121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 335fc15ea75d483990e6ae5a5872682f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 335fc15ea75d483990e6ae5a5872682f already in progress: creating tablet
I20260811 18:57:31.495963 5091 raft_consensus.cc:482] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.522182 5091 raft_consensus.cc:504] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.522953 5091 leader_election.cc:288] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.523553 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f0cdcee681460084662e01152ad9dd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:31.523625 4167 rpcz_store.cc:276] 0811 18:57:26.706835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.707031 (+ 196us) service_pool.cc:227] Handling call
0811 18:57:31.326981 (+4619950us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:31.326992 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.350825 (+ 23833us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.474084 (+123259us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.488612 (+ 14528us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4619721}
W20260811 18:57:31.524071 3676 leader_election.cc:341] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.524659 4167 tablet_replica.cc:330] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.524844 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b0b199f4c9e14b468cb26cbd61fc8586 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:31.525044 4167 raft_consensus.cc:2229] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.525282 4167 raft_consensus.cc:2258] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.526007 4167 ts_tablet_manager.cc:1905] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.531213 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 659) took 4812 ms. Trace:
I20260811 18:57:31.532133 4169 rpcz_store.cc:276] 0811 18:57:26.719126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.719208 (+ 82us) service_pool.cc:227] Handling call
0811 18:57:31.531082 (+4811874us) ts_tablet_manager.cc:1145] Deleting tablet 8b7fc4a898cd4e2c9e27d6f5f964412a
0811 18:57:31.531094 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.531190 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4811553}
I20260811 18:57:31.533308 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 658) took 4814 ms. Trace:
I20260811 18:57:31.533696 4162 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.533939 4162 ts_tablet_manager.cc:1397] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.248s user 0.008s sys 0.018s
I20260811 18:57:31.534060 4169 rpcz_store.cc:276] 0811 18:57:26.718855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.718941 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:31.533173 (+4814232us) ts_tablet_manager.cc:1145] Deleting tablet 705605cd929940f8a252b6e95abbeb95
0811 18:57:31.533186 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.533289 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4804744}
I20260811 18:57:31.534893 4162 raft_consensus.cc:348] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.535118 4162 raft_consensus.cc:374] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.535178 4162 raft_consensus.cc:729] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.536173 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 680) took 4462 ms. Trace:
I20260811 18:57:31.536895 4169 rpcz_store.cc:276] 0811 18:57:27.073302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.073385 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:31.536047 (+4462662us) ts_tablet_manager.cc:1145] Deleting tablet 37be4f7579fa4ab99ddbacfa80282b42
0811 18:57:31.536058 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.536158 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4462381}
I20260811 18:57:31.537889 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 681) took 4464 ms. Trace:
I20260811 18:57:31.538491 4169 rpcz_store.cc:276] 0811 18:57:27.073538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.073594 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:31.537789 (+4464195us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:31.537800 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.537874 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4464019}
I20260811 18:57:31.539608 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 682) took 4465 ms. Trace:
I20260811 18:57:31.540270 4169 rpcz_store.cc:276] 0811 18:57:27.073959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.074008 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:31.539502 (+4465494us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:31.539514 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.539592 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4465336}
I20260811 18:57:31.541846 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 683) took 4441 ms. Trace:
I20260811 18:57:31.542372 4169 rpcz_store.cc:276] 0811 18:57:27.100057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.100124 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:31.541327 (+4441203us) ts_tablet_manager.cc:1145] Deleting tablet 28af2d2c61254b0ca399dddb05e9ee70
0811 18:57:31.541342 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.541822 (+ 480us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4440756}
I20260811 18:57:31.543363 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 684) took 4442 ms. Trace:
I20260811 18:57:31.543991 4169 rpcz_store.cc:276] 0811 18:57:27.100734 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.100810 (+ 76us) service_pool.cc:227] Handling call
0811 18:57:31.543251 (+4442441us) ts_tablet_manager.cc:1145] Deleting tablet 9722572533de436cb9149e6230570382
0811 18:57:31.543274 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.543348 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4442231}
I20260811 18:57:31.544909 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 691) took 4317 ms. Trace:
I20260811 18:57:31.545466 4169 rpcz_store.cc:276] 0811 18:57:27.227809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.232631 (+ 4822us) service_pool.cc:227] Handling call
0811 18:57:31.544816 (+4312185us) ts_tablet_manager.cc:1145] Deleting tablet 90576c068ce4468a9c572307dcb79b10
0811 18:57:31.544826 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.544896 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4311839}
I20260811 18:57:31.546324 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 696) took 3996 ms. Trace:
I20260811 18:57:31.546527 4162 consensus_queue.cc:260] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.547154 4169 rpcz_store.cc:276] 0811 18:57:27.549972 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.550526 (+ 554us) service_pool.cc:227] Handling call
0811 18:57:31.546207 (+3995681us) ts_tablet_manager.cc:1145] Deleting tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
0811 18:57:31.546218 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.546308 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3995447}
I20260811 18:57:31.548094 5334 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.548295 5334 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.549378 5334 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.549791 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 697) took 3999 ms. Trace:
I20260811 18:57:31.550745 4169 rpcz_store.cc:276] 0811 18:57:27.550173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.550805 (+ 632us) service_pool.cc:227] Handling call
0811 18:57:31.549680 (+3998875us) ts_tablet_manager.cc:1145] Deleting tablet a49b8b344177487fb5bd096ee9cdea2f
0811 18:57:31.549692 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.549775 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3998631}
I20260811 18:57:31.550727 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.551245 3740 raft_consensus.cc:2454] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.551661 3824 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.552031 5334 raft_consensus.cc:2781] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260811 18:57:31.533169 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8b7fc4a898cd4e2c9e27d6f5f964412a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b7fc4a898cd4e2c9e27d6f5f964412a already in progress: creating tablet
I20260811 18:57:31.552211 5334 raft_consensus.cc:482] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.552340 5334 raft_consensus.cc:3037] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:31.552634 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 705605cd929940f8a252b6e95abbeb95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 705605cd929940f8a252b6e95abbeb95 already in progress: creating tablet
W20260811 18:57:31.553051 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 37be4f7579fa4ab99ddbacfa80282b42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37be4f7579fa4ab99ddbacfa80282b42 already in progress: creating tablet
W20260811 18:57:31.553409 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0aee6beb2e70421083841520733087f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aee6beb2e70421083841520733087f8 already in progress: creating tablet
W20260811 18:57:31.553798 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7251204325dd424192a6f58e18e71c34 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7251204325dd424192a6f58e18e71c34 already in progress: creating tablet
W20260811 18:57:31.554085 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 28af2d2c61254b0ca399dddb05e9ee70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28af2d2c61254b0ca399dddb05e9ee70 already in progress: creating tablet
W20260811 18:57:31.554425 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9722572533de436cb9149e6230570382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9722572533de436cb9149e6230570382 already in progress: creating tablet
W20260811 18:57:31.554733 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 90576c068ce4468a9c572307dcb79b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90576c068ce4468a9c572307dcb79b10 already in progress: creating tablet
W20260811 18:57:31.555205 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 already in progress: creating tablet
W20260811 18:57:31.555534 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a49b8b344177487fb5bd096ee9cdea2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49b8b344177487fb5bd096ee9cdea2f already in progress: creating tablet
I20260811 18:57:31.555637 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 698) took 4005 ms. Trace:
I20260811 18:57:31.556449 4169 rpcz_store.cc:276] 0811 18:57:27.550316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.551082 (+ 766us) service_pool.cc:227] Handling call
0811 18:57:31.555507 (+4004425us) ts_tablet_manager.cc:1145] Deleting tablet c21eb02db3de4c97878fa8badccaa6e3
0811 18:57:31.555521 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.555617 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4004188}
I20260811 18:57:31.558825 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 699) took 4008 ms. Trace:
W20260811 18:57:31.558784 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c21eb02db3de4c97878fa8badccaa6e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c21eb02db3de4c97878fa8badccaa6e3 already in progress: creating tablet
I20260811 18:57:31.559975 4169 rpcz_store.cc:276] 0811 18:57:27.550461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.551385 (+ 924us) service_pool.cc:227] Handling call
0811 18:57:31.558697 (+4007312us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:31.558711 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.558806 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4007118}
I20260811 18:57:31.559908 4163 tablet_bootstrap.cc:492] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.560732 4163 ts_tablet_manager.cc:1397] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.006s sys 0.005s
W20260811 18:57:31.560760 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0cccebfc54484a9b9abffcce4e459fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cccebfc54484a9b9abffcce4e459fb5 already in progress: creating tablet
I20260811 18:57:31.561676 4163 raft_consensus.cc:348] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.562288 4163 raft_consensus.cc:374] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.562446 4163 raft_consensus.cc:729] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.562752 4163 consensus_queue.cc:260] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.558306 5334 raft_consensus.cc:504] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.562060 4169 tablet_replica.cc:330] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.564009 4169 raft_consensus.cc:2229] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.564635 5334 leader_election.cc:288] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.565074 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490f34d7a403416fbab64e46973746de" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.565366 3743 raft_consensus.cc:3037] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.569280 3743 raft_consensus.cc:2454] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.570613 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet c08436f33db043b1bc6395cd92102e59 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.572365 4162 ts_tablet_manager.cc:1428] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.038s user 0.002s sys 0.000s
I20260811 18:57:31.569718 3824 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.575850 5283 raft_consensus.cc:2781] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.575973 5283 raft_consensus.cc:686] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.576218 5283 consensus_queue.cc:237] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.581532 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" } modify_external_catalogs: true
I20260811 18:57:31.581836 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_275225f078f3430da2cbc95fcd578fa7" } modify_external_catalogs: true
I20260811 18:57:31.582317 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c05dd720298449e9aa85faeadb50b96"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.583009 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c05dd720298449e9aa85faeadb50b96 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.587055 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" } modify_external_catalogs: true
I20260811 18:57:31.575172 4169 pending_rounds.cc:62] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:31.587371 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a27cf06565547a39a8eee2420c9a31b" } modify_external_catalogs: true
I20260811 18:57:31.587529 4169 pending_rounds.cc:68] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400155312156672 op_type: NO_OP noop_request { }
I20260811 18:57:31.588119 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_680160aaac0a4c8da66634dc41d6abb9" } modify_external_catalogs: true
I20260811 18:57:31.588344 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_680160aaac0a4c8da66634dc41d6abb9" } modify_external_catalogs: true
I20260811 18:57:31.587702 4169 raft_consensus.cc:2866] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:31.588537 4169 raft_consensus.cc:2258] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.595225 4162 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.597196 4162 tablet_bootstrap.cc:654] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.599848 4169 ts_tablet_manager.cc:1905] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.575558 4163 ts_tablet_manager.cc:1428] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
W20260811 18:57:31.605400 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a2b9a2fd0c949a9977d2dfa83db9e8a consensus state has no leader
I20260811 18:57:31.605792 4163 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:31.607410 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cef66a1c435645d6822188ec4776526b" } modify_external_catalogs: true
I20260811 18:57:31.608039 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0090725e77446588856c07015fda936" } new_table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" modify_external_catalogs: true
I20260811 18:57:31.609104 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cef66a1c435645d6822188ec4776526b" } modify_external_catalogs: true
W20260811 18:57:31.617077 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9948c33ed58b4160932be82ef9615ba5 consensus state has no leader
I20260811 18:57:31.617654 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b08e57f933b24cd0aee53441350e6775" } new_table_name: "default.Table_7065b527f8104608b1efc6787f0b2b77" modify_external_catalogs: true
I20260811 18:57:31.619529 4163 tablet_bootstrap.cc:654] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.631253 4163 tablet_bootstrap.cc:492] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.632062 4163 ts_tablet_manager.cc:1397] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20260811 18:57:31.634958 4163 raft_consensus.cc:348] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.640156 4163 raft_consensus.cc:374] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.640460 4163 raft_consensus.cc:729] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.644471 5091 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.644629 5091 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.642205 4163 consensus_queue.cc:260] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.645277 5091 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.646011 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:31.646659 3676 leader_election.cc:341] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:31.648420 4167 ts_tablet_manager.cc:1918] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:31.649539 4167 log.cc:1196] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4687b95728fc485fa0e842bc447eb83f
#6 0x7fc7c23171a3 in kudu::tserver::TabletServer::Start() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server.cc:157:3
W20260811 18:57:31.651232 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb has no consensus state
I20260811 18:57:31.652364 4169 ts_tablet_manager.cc:1918] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.652544 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e0b184f30d7464fbb8120d29bd0a80b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.653347 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e0b184f30d7464fbb8120d29bd0a80b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.661899 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
#7 0x7fc7c2326380 in kudu::tserver::RunTabletServer() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server_runner.cc:86:3
W20260811 18:57:31.663946 3714 consensus_meta.cc:287] T e5f47ccc193d48efa15d60a06b148e00 P bd8c4bdbd811417ab011a7131f17d283: Time spent flushing consensus metadata: real 0.622s user 0.028s sys 0.001s
I20260811 18:57:31.666370 4169 log.cc:1196] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ef393c341bb54baf87176db5b5254ec7
I20260811 18:57:31.666129 4167 ts_tablet_manager.cc:1939] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:31.659942 5334 consensus_queue.cc:1035] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:31.670655 5334 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.670854 5334 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.672070 5334 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.669962 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 679) took 4962 ms. Trace:
I20260811 18:57:31.672427 4167 rpcz_store.cc:276] 0811 18:57:26.706983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:26.707313 (+ 330us) service_pool.cc:227] Handling call
0811 18:57:31.524597 (+4817284us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:31.524611 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.579886 (+ 55275us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.648231 (+ 68345us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.669434 (+ 21203us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":128,"tablet-delete.queue_time_us":4817065}
I20260811 18:57:31.673997 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4687b95728fc485fa0e842bc447eb83f (table default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]) successfully deleted
I20260811 18:57:31.653394 4163 ts_tablet_manager.cc:1428] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260811 18:57:31.678589 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:31.678854 3740 raft_consensus.cc:2454] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:31.679036 4163 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:31.679368 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 53039.4us
I20260811 18:57:31.681538 4169 ts_tablet_manager.cc:1939] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.684340 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb (DEFAULT_TABLE table=default.Table_4133a1bbf6474492bbf3683ca9608a52 [id=5d4c64c60c974852bceb5a5a71174e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.687107 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdab1c3c431c4dceb08b00eabcfa1ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.687299 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 711) took 4040 ms. Trace:
I20260811 18:57:31.688060 3824 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.687402 4169 rpcz_store.cc:276] 0811 18:57:27.646886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.646977 (+ 91us) service_pool.cc:227] Handling call
0811 18:57:31.562010 (+3915033us) ts_tablet_manager.cc:1145] Deleting tablet ef393c341bb54baf87176db5b5254ec7
0811 18:57:31.562022 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.628599 (+ 66577us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.652232 (+ 23633us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.687275 (+ 35043us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10753,"tablet-delete.queue_time_us":3914753}
I20260811 18:57:31.688912 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ef393c341bb54baf87176db5b5254ec7 (table default.Table_dc0e1659d31645b28fe322618e22cb6c [id=85691a7b24b14fe0b9aaa1b77f5f6be0]) successfully deleted
I20260811 18:57:31.689222 5334 raft_consensus.cc:2781] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:31.689318 5334 raft_consensus.cc:482] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:31.689419 5283 consensus_queue.cc:1035] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:31.689422 5334 raft_consensus.cc:3037] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.690842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 712) took 4043 ms. Trace:
I20260811 18:57:31.691290 4169 rpcz_store.cc:276] 0811 18:57:27.647027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.647087 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:31.690700 (+4043613us) ts_tablet_manager.cc:1145] Deleting tablet f287040b879642448367abe2b1c73cd9
0811 18:57:31.690714 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.690823 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4043266}
W20260811 18:57:31.692106 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f287040b879642448367abe2b1c73cd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f287040b879642448367abe2b1c73cd9 already in progress: creating tablet
I20260811 18:57:31.692559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 713) took 4044 ms. Trace:
I20260811 18:57:31.692873 4169 rpcz_store.cc:276] 0811 18:57:27.647726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.647800 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:31.692098 (+4044298us) ts_tablet_manager.cc:1145] Deleting tablet 2be6762e326448d7b677a5641e2b3103
0811 18:57:31.692115 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.692536 (+ 421us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4044020}
W20260811 18:57:31.693732 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2be6762e326448d7b677a5641e2b3103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be6762e326448d7b677a5641e2b3103 already in progress: creating tablet
I20260811 18:57:31.693801 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 721) took 3978 ms. Trace:
I20260811 18:57:31.694553 4169 rpcz_store.cc:276] 0811 18:57:27.715744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.715805 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:31.693670 (+3977865us) ts_tablet_manager.cc:1145] Deleting tablet 4059e5907b1e488ab9e1d04e563f96cd
0811 18:57:31.693682 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.693781 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3977538}
I20260811 18:57:31.694626 5334 raft_consensus.cc:504] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.696170 5334 leader_election.cc:288] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:31.696705 4169 tablet_replica.cc:330] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:31.696691 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4059e5907b1e488ab9e1d04e563f96cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4059e5907b1e488ab9e1d04e563f96cd already in progress: creating tablet
I20260811 18:57:31.696667 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a902ad70a25546be926ff8f7bf2ed64f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:31.696913 3745 raft_consensus.cc:3037] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:31.697273 4169 raft_consensus.cc:2229] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.698467 4169 raft_consensus.cc:2258] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.700875 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7f50a732b44c5a8b6f2eb392455409 (DEFAULT_TABLE table=default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 [id=e0a0d66062a04bd7843d699a54683785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.701403 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7f50a732b44c5a8b6f2eb392455409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.701331 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 825b6743798144deb24330304d6b5d2b (DEFAULT_TABLE table=default.Table_3a0329aed6694593bc21571dd8f59670 [id=37bcf611810c4cd8b8a42926c1064705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.701725 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825b6743798144deb24330304d6b5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.707494 3853 tablet_service.cc:1467] Processing CreateTablet for tablet fae33c49dab3444988738f31180de35a (DEFAULT_TABLE table=default.Table_d584e1cc7d8b45be912d8610e391f4f7 [id=b0d36b86f3114722acc3d45b30e041bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.707830 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7f50a732b44c5a8b6f2eb392455409 (DEFAULT_TABLE table=default.Table_94caf53b154f4e39b9ea7bf02dcb11c7 [id=e0a0d66062a04bd7843d699a54683785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.708098 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 75af8f21dcc7443fabd13e06061d19a6 (DEFAULT_TABLE table=default.Table_949ccbca1e8b44b79578377f67f2422c [id=2764171b8d1a4494bd0d30ecfd267a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.708233 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7f50a732b44c5a8b6f2eb392455409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.700500 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 825b6743798144deb24330304d6b5d2b (DEFAULT_TABLE table=default.Table_3a0329aed6694593bc21571dd8f59670 [id=37bcf611810c4cd8b8a42926c1064705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.710656 3709 tablet_service.cc:1467] Processing CreateTablet for tablet fae33c49dab3444988738f31180de35a (DEFAULT_TABLE table=default.Table_d584e1cc7d8b45be912d8610e391f4f7 [id=b0d36b86f3114722acc3d45b30e041bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.711047 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae33c49dab3444988738f31180de35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.699589 4169 ts_tablet_manager.cc:1905] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:31.717737 3745 raft_consensus.cc:2454] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:31.718278 3824 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:31.719453 5334 raft_consensus.cc:2781] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:31.720230 5334 raft_consensus.cc:686] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:31.720553 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4013b1905ea74bc6a06f9468d9025570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.720830 5334 consensus_queue.cc:237] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:31.721385 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4013b1905ea74bc6a06f9468d9025570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:31.734701 3953 kernel_stack_watchdog.cc:198] Thread 3967 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 20182ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:31.734668 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1eb30e64e86040098dcee5fc2e9b7266
I20260811 18:57:31.735757 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3418766144ec4b91bb802faa2ad8808e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.736508 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3418766144ec4b91bb802faa2ad8808e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.737203 3864 tablet_service.cc:1467] Processing CreateTablet for tablet aafc113d84a44fe9b291fa2efb69ef2e (DEFAULT_TABLE table=default.Table_aaf507832a664e6ebcdafffc433bbac7 [id=b3c972d878764259adfd841f2ddda03f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.738036 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 35ebc32eb5d5443eaaeba245b566f98a
I20260811 18:57:31.738945 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.738878 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a273b6a16afa44d196b4654b067735a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.739270 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.739414 4162 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:31.739534 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a273b6a16afa44d196b4654b067735a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.739562 4162 ts_tablet_manager.cc:1397] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.144s user 0.002s sys 0.001s
I20260811 18:57:31.740655 4162 raft_consensus.cc:348] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.741065 4162 raft_consensus.cc:374] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.741134 4162 raft_consensus.cc:729] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.741456 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_98f9637c9f7a47559887a4c73c52daea" } new_table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" modify_external_catalogs: true
I20260811 18:57:31.741369 4162 consensus_queue.cc:260] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.742264 4162 ts_tablet_manager.cc:1428] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:31.742509 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae33c49dab3444988738f31180de35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.742575 4162 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:31.742936 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3dd4ec30acde4585a1c3451927217cb3
I20260811 18:57:31.743567 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 3dd4ec30acde4585a1c3451927217cb3 deleted and replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4
I20260811 18:57:31.744060 4162 tablet_bootstrap.cc:654] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.743957 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_79f5b2394f094ecc85733e272dcfe369"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.744645 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79f5b2394f094ecc85733e272dcfe369 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.746510 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75af8f21dcc7443fabd13e06061d19a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.747826 3714 tablet_service.cc:1467] Processing CreateTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb (DEFAULT_TABLE table=default.Table_4133a1bbf6474492bbf3683ca9608a52 [id=5d4c64c60c974852bceb5a5a71174e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.749589 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.750141 3722 tablet_service.cc:1467] Processing CreateTablet for tablet aafc113d84a44fe9b291fa2efb69ef2e (DEFAULT_TABLE table=default.Table_aaf507832a664e6ebcdafffc433bbac7 [id=b3c972d878764259adfd841f2ddda03f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.750501 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aafc113d84a44fe9b291fa2efb69ef2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.750605 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aafc113d84a44fe9b291fa2efb69ef2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.754763 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 862a3101a9b8498693851d4034826ca3 (DEFAULT_TABLE table=default.Table_b2a92f77f0074806a1b56a31c303dd69 [id=6553759cad994e019ddc88e3915cecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.755489 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 72b2ebaee8d84fb1aae11d3160ad712b (DEFAULT_TABLE table=default.Table_827736b8218e48e086bf05354914d450 [id=a6444c78db944c00bf604e3e662c5cb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.755573 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757381 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 9b931eca7ba04c11af6571cb418f882e (DEFAULT_TABLE table=default.Table_ebca8dd587b144d7909761c052084855 [id=d51a676c300d49678703b388baddb9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757511 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 72b2ebaee8d84fb1aae11d3160ad712b (DEFAULT_TABLE table=default.Table_827736b8218e48e086bf05354914d450 [id=a6444c78db944c00bf604e3e662c5cb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757510 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 75af8f21dcc7443fabd13e06061d19a6 (DEFAULT_TABLE table=default.Table_949ccbca1e8b44b79578377f67f2422c [id=2764171b8d1a4494bd0d30ecfd267a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.757949 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0c50ef3021404d30a09396237ea30627 (DEFAULT_TABLE table=default.Table_80a7a76a26ab4fc9919bfef0a312f217 [id=2764e110843341bc99f1898c1445eeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758002 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b2ebaee8d84fb1aae11d3160ad712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.757921 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 862a3101a9b8498693851d4034826ca3 (DEFAULT_TABLE table=default.Table_b2a92f77f0074806a1b56a31c303dd69 [id=6553759cad994e019ddc88e3915cecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758108 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eb30e64e86040098dcee5fc2e9b7266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.758276 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862a3101a9b8498693851d4034826ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.758566 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 62c7286685244f558bfa80a9fc8abaf4 (DEFAULT_TABLE table=default.Table_3fbd6111333f4635a274f38a20902738 [id=068eb6b595f144789e9ad90c9a8f1bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.758940 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c7286685244f558bfa80a9fc8abaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.759294 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ebc32eb5d5443eaaeba245b566f98a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:31.762616 3678 leader_election.cc:334] T 3744b3d608754de68c4c57b072d1dc15 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.372s (SENT)
I20260811 18:57:31.762617 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.763017 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60091bd891a94cc4aac6ef47673a4ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.766588 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9b931eca7ba04c11af6571cb418f882e (DEFAULT_TABLE table=default.Table_ebca8dd587b144d7909761c052084855 [id=d51a676c300d49678703b388baddb9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.766958 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b931eca7ba04c11af6571cb418f882e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.757565 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 0c50ef3021404d30a09396237ea30627 (DEFAULT_TABLE table=default.Table_80a7a76a26ab4fc9919bfef0a312f217 [id=2764e110843341bc99f1898c1445eeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.770825 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c50ef3021404d30a09396237ea30627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774573 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862a3101a9b8498693851d4034826ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774796 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75af8f21dcc7443fabd13e06061d19a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.774940 5091 raft_consensus.cc:482] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:31.775076 5091 raft_consensus.cc:504] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.775674 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b2ebaee8d84fb1aae11d3160ad712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.775789 5091 leader_election.cc:288] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:31.776207 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c50ef3021404d30a09396237ea30627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.778646 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b931eca7ba04c11af6571cb418f882e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.778623 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c07150e31994bdf9a00ac2e7a954691" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:31.779543 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60091bd891a94cc4aac6ef47673a4ee5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.782552 3676 leader_election.cc:341] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:31.783285 3678 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.544s (SENT)
I20260811 18:57:31.783406 3678 leader_election.cc:302] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:31.783707 3678 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.141s (SENT)
I20260811 18:57:31.784173 5091 raft_consensus.cc:2726] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:31.785662 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd4ec30acde4585a1c3451927217cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 10e441b5df3d40fd93cf2dafe33bc2f4) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:31.791925 3824 consensus_peers.cc:487] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0b199f4c9e14b468cb26cbd61fc8586. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:31.771768 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 701) took 4709 ms. Trace:
I20260811 18:57:31.792588 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdab1c3c431c4dceb08b00eabcfa1ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.796331 4162 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.793063 4167 rpcz_store.cc:276] 0811 18:57:27.061614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.061700 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:31.674592 (+4612892us) ts_tablet_manager.cc:1145] Deleting tablet 37be4f7579fa4ab99ddbacfa80282b42
0811 18:57:31.674607 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.771201 (+ 96594us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4612505}
W20260811 18:57:31.797776 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 37be4f7579fa4ab99ddbacfa80282b42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37be4f7579fa4ab99ddbacfa80282b42 already in progress: creating tablet
I20260811 18:57:31.798548 3548 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_cbe771adb7364096802df77fa867acb0 [id=29a0d1e06bea499bacc5851fd6d0db5c] alter complete (version 2)
I20260811 18:57:31.799486 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b702aef88e124443888002d8ccd47b2f
I20260811 18:57:31.800184 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet c21eb02db3de4c97878fa8badccaa6e3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:31.800905 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f1d0ca0ad934be687840ccd25db713a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.801563 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f1d0ca0ad934be687840ccd25db713a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.802507 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 825b6743798144deb24330304d6b5d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.802644 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1800e9e2b064a6fb546b9d33cefa68b
I20260811 18:57:31.803333 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.803956 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b30b503bf0a841d6a0061c6430b70b1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.805356 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f2f31331974f42c3a22b3ac21b690f82
I20260811 18:57:31.806201 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4e3a3cb2f24408ba3e2e8510459eee4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:31.806691 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 703) took 4738 ms. Trace:
I20260811 18:57:31.806787 4167 rpcz_store.cc:276] 0811 18:57:27.067918 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.068068 (+ 150us) service_pool.cc:227] Handling call
0811 18:57:31.806515 (+4738447us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:31.806530 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.806667 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4738188}
W20260811 18:57:31.806931 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.807224 4162 ts_tablet_manager.cc:1397] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.000s
I20260811 18:57:31.807977 4162 raft_consensus.cc:348] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.808187 4162 raft_consensus.cc:374] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.808233 4162 raft_consensus.cc:729] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.809250 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91bea743ec0b491299f45f3ac23cb734"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.809819 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91bea743ec0b491299f45f3ac23cb734 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.810379 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 704) took 4742 ms. Trace:
I20260811 18:57:31.812657 4167 rpcz_store.cc:276] 0811 18:57:27.068024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.068374 (+ 350us) service_pool.cc:227] Handling call
0811 18:57:31.809711 (+4741337us) ts_tablet_manager.cc:1145] Deleting tablet 9722572533de436cb9149e6230570382
0811 18:57:31.809726 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.810263 (+ 537us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4741083}
I20260811 18:57:31.815263 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f31331974f42c3a22b3ac21b690f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.818022 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_efc46b70936f4bd292f775b723c1fbff [id=0fee6d656a1b486cb1fb77abf4042dd8] alter complete (version 2)
I20260811 18:57:31.819098 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_badc11f1b0554e629ddb3f8a20e151e2" } new_table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" modify_external_catalogs: true
W20260811 18:57:31.819787 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7251204325dd424192a6f58e18e71c34 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7251204325dd424192a6f58e18e71c34 already in progress: creating tablet
W20260811 18:57:31.820138 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9722572533de436cb9149e6230570382 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9722572533de436cb9149e6230570382 already in progress: creating tablet
I20260811 18:57:31.821632 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:31.822258 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:31.823033 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet c08436f33db043b1bc6395cd92102e59 with delete_type TABLET_DATA_DELETED (replaced by tablet 059b325df1db4fde9d4b26b559121316) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.823336 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1923ce0049448fe4e28f2dd258dc with delete_type TABLET_DATA_DELETED (replaced by tablet 55f97f37fa7c483cacaeed54f93e7a24) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.823546 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.823791 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.821906 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet b702aef88e124443888002d8ccd47b2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.824244 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f31331974f42c3a22b3ac21b690f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.824506 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet f1800e9e2b064a6fb546b9d33cefa68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.824612 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b702aef88e124443888002d8ccd47b2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:31 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:31.813650 4162 consensus_queue.cc:260] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.825488 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a902ad70a25546be926ff8f7bf2ed64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:31.827095 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5 (DEFAULT_TABLE table=default.Table_5b0e7b307dc84d88aff7e744fe4ed97b [id=cc52e710b7b64b74aca455760f151a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.827541 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.829046 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b426bc8778444ab3959ce53532b50959 (DEFAULT_TABLE table=default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b [id=868ded1d55df48ebb3116ca5e939599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.829486 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b426bc8778444ab3959ce53532b50959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.829640 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5 (DEFAULT_TABLE table=default.Table_5b0e7b307dc84d88aff7e744fe4ed97b [id=cc52e710b7b64b74aca455760f151a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.831187 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc967770de4ed486b638fb6b174db8 (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.831728 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed86c89f8b84294a6ab93e4faa9f4b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.833710 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 35b3ddaf14db47438ec396689cdb0d41 (DEFAULT_TABLE table=default.Table_634c196a72864956aedd6ceb1757173a [id=35751da09b674aebb70ab932d6f9dfcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.834079 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b3ddaf14db47438ec396689cdb0d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.834282 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc967770de4ed486b638fb6b174db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.836226 3717 tablet_service.cc:1467] Processing CreateTablet for tablet ee5b7d1e32514ad48177371582e85241 (DEFAULT_TABLE table=default.Table_fd5ee099cbc94fd285f1b71eea0cde80 [id=cffd7424406a4d50be8d46cf45d6e6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.836318 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 112de1a11805406b9183100c6c66f720 (DEFAULT_TABLE table=default.Table_93b2871d4c454120a6b609ae91e096e6 [id=7873608f194c42bc9b11d26cca8dafc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.836575 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5b7d1e32514ad48177371582e85241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.836740 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112de1a11805406b9183100c6c66f720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.838145 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 112de1a11805406b9183100c6c66f720 (DEFAULT_TABLE table=default.Table_93b2871d4c454120a6b609ae91e096e6 [id=7873608f194c42bc9b11d26cca8dafc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.838616 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112de1a11805406b9183100c6c66f720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.839881 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ee5b7d1e32514ad48177371582e85241 (DEFAULT_TABLE table=default.Table_fd5ee099cbc94fd285f1b71eea0cde80 [id=cffd7424406a4d50be8d46cf45d6e6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.840229 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5b7d1e32514ad48177371582e85241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.842634 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 35b3ddaf14db47438ec396689cdb0d41 (DEFAULT_TABLE table=default.Table_634c196a72864956aedd6ceb1757173a [id=35751da09b674aebb70ab932d6f9dfcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.843027 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b3ddaf14db47438ec396689cdb0d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.846097 3872 tablet_service.cc:1467] Processing CreateTablet for tablet b426bc8778444ab3959ce53532b50959 (DEFAULT_TABLE table=default.Table_3ca48ffc0e9e4ad3b34d11d49cd5a77b [id=868ded1d55df48ebb3116ca5e939599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.846508 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b426bc8778444ab3959ce53532b50959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.847090 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc967770de4ed486b638fb6b174db8 (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.847431 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc967770de4ed486b638fb6b174db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.826154 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 705) took 4754 ms. Trace:
I20260811 18:57:31.857717 4167 rpcz_store.cc:276] 0811 18:57:27.071363 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.071415 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:31.825675 (+4754260us) ts_tablet_manager.cc:1145] Deleting tablet 28af2d2c61254b0ca399dddb05e9ee70
0811 18:57:31.825696 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.826105 (+ 409us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4745081}
I20260811 18:57:31.826858 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 62c7286685244f558bfa80a9fc8abaf4 (DEFAULT_TABLE table=default.Table_3fbd6111333f4635a274f38a20902738 [id=068eb6b595f144789e9ad90c9a8f1bcd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:31.859858 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 28af2d2c61254b0ca399dddb05e9ee70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28af2d2c61254b0ca399dddb05e9ee70 already in progress: creating tablet
I20260811 18:57:31.860560 4162 ts_tablet_manager.cc:1428] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.053s user 0.001s sys 0.001s
I20260811 18:57:31.860817 4163 tablet_bootstrap.cc:654] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:31.888747 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 09366eccf38a4876840cc5ddddbbb103 (table default.Table_1314176a6f3346d7a808360af2c1cfd2 [id=307c31b5346949adb9b0c0db5da22ea0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:31.889093 3855 tablet_service.cc:1467] Processing CreateTablet for tablet bf1c037a37504b3fad9612f5a734dd33 (DEFAULT_TABLE table=default.Table_6f13efa6fe524f5bbd7154aa2f991d38 [id=1387459576db42fa870b89b1dab61400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.889954 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a05b9d2817ff41d7a1d516f34dad88d2 (DEFAULT_TABLE table=default.Table_2076434270fa4c9c99b4d12f1aa924c9 [id=173131c0dc16407d8634e08b2580c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.890391 3710 tablet_service.cc:1467] Processing CreateTablet for tablet a05b9d2817ff41d7a1d516f34dad88d2 (DEFAULT_TABLE table=default.Table_2076434270fa4c9c99b4d12f1aa924c9 [id=173131c0dc16407d8634e08b2580c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.890393 3709 tablet_service.cc:1467] Processing CreateTablet for tablet bf1c037a37504b3fad9612f5a734dd33 (DEFAULT_TABLE table=default.Table_6f13efa6fe524f5bbd7154aa2f991d38 [id=1387459576db42fa870b89b1dab61400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.894011 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c7286685244f558bfa80a9fc8abaf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.898371 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 702) took 4830 ms. Trace:
I20260811 18:57:31.898579 4167 rpcz_store.cc:276] 0811 18:57:27.067773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.071735 (+ 3962us) service_pool.cc:227] Handling call
0811 18:57:31.860257 (+4788522us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:31.898116 (+ 37859us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.898314 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4788270}
I20260811 18:57:31.899420 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 712) took 4678 ms. Trace:
I20260811 18:57:31.899595 4167 rpcz_store.cc:276] 0811 18:57:27.220742 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.226578 (+ 5836us) service_pool.cc:227] Handling call
0811 18:57:31.899292 (+4672714us) ts_tablet_manager.cc:1145] Deleting tablet 90576c068ce4468a9c572307dcb79b10
0811 18:57:31.899305 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.899401 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4668816}
I20260811 18:57:31.900179 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 717) took 4377 ms. Trace:
I20260811 18:57:31.900321 4167 rpcz_store.cc:276] 0811 18:57:27.523107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.524301 (+ 1194us) service_pool.cc:227] Handling call
0811 18:57:31.900040 (+4375739us) ts_tablet_manager.cc:1145] Deleting tablet 25fc10fa2ee24b1aaa68b360b5a8aa61
0811 18:57:31.900055 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.900165 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4375426}
I20260811 18:57:31.900887 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 718) took 4362 ms. Trace:
I20260811 18:57:31.901052 4167 rpcz_store.cc:276] 0811 18:57:27.538044 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.538125 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:31.900736 (+4362611us) ts_tablet_manager.cc:1145] Deleting tablet a49b8b344177487fb5bd096ee9cdea2f
0811 18:57:31.900748 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.900867 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4362188}
I20260811 18:57:31.901641 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 719) took 4352 ms. Trace:
I20260811 18:57:31.901800 4167 rpcz_store.cc:276] 0811 18:57:27.548798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.549039 (+ 241us) service_pool.cc:227] Handling call
0811 18:57:31.901522 (+4352483us) ts_tablet_manager.cc:1145] Deleting tablet c21eb02db3de4c97878fa8badccaa6e3
0811 18:57:31.901535 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.901623 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4352125}
I20260811 18:57:31.902319 4162 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:31.903834 3824 consensus_peers.cc:487] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4687b95728fc485fa0e842bc447eb83f. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:31.904979 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05b9d2817ff41d7a1d516f34dad88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.905093 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf1c037a37504b3fad9612f5a734dd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.905405 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05b9d2817ff41d7a1d516f34dad88d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.906742 4162 tablet_bootstrap.cc:654] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:31.907856 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 720) took 4358 ms. Trace:
I20260811 18:57:31.907939 4167 rpcz_store.cc:276] 0811 18:57:27.548968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.549459 (+ 491us) service_pool.cc:227] Handling call
0811 18:57:31.907738 (+4358279us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:31.907750 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.907838 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4358040}
I20260811 18:57:31.908380 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 731) took 4263 ms. Trace:
I20260811 18:57:31.908452 4167 rpcz_store.cc:276] 0811 18:57:27.644992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.645061 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:31.908289 (+4263228us) ts_tablet_manager.cc:1145] Deleting tablet f287040b879642448367abe2b1c73cd9
0811 18:57:31.908298 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.908367 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4262970}
I20260811 18:57:31.909049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 732) took 4263 ms. Trace:
I20260811 18:57:31.909211 4167 rpcz_store.cc:276] 0811 18:57:27.645516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.645565 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:31.908848 (+4263283us) ts_tablet_manager.cc:1145] Deleting tablet 2be6762e326448d7b677a5641e2b3103
0811 18:57:31.908890 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.909032 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4263112}
I20260811 18:57:31.909499 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf1c037a37504b3fad9612f5a734dd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.909731 4167 tablet_replica.cc:330] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:31.910063 4167 raft_consensus.cc:2229] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:31.910419 4167 raft_consensus.cc:2258] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.917747 4163 tablet_bootstrap.cc:492] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:31.917999 4163 ts_tablet_manager.cc:1397] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.239s user 0.003s sys 0.002s
W20260811 18:57:31.919718 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0aee6beb2e70421083841520733087f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aee6beb2e70421083841520733087f8 already in progress: creating tablet
W20260811 18:57:31.920100 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 90576c068ce4468a9c572307dcb79b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90576c068ce4468a9c572307dcb79b10 already in progress: creating tablet
W20260811 18:57:31.920383 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25fc10fa2ee24b1aaa68b360b5a8aa61 already in progress: creating tablet
W20260811 18:57:31.920611 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a49b8b344177487fb5bd096ee9cdea2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49b8b344177487fb5bd096ee9cdea2f already in progress: creating tablet
W20260811 18:57:31.921144 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c21eb02db3de4c97878fa8badccaa6e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c21eb02db3de4c97878fa8badccaa6e3 already in progress: creating tablet
W20260811 18:57:31.921414 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0cccebfc54484a9b9abffcce4e459fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0cccebfc54484a9b9abffcce4e459fb5 already in progress: creating tablet
I20260811 18:57:31.921605 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a8483f83b5cd4a65a0fe9097c2c80cb3 (DEFAULT_TABLE table=default.Table_aadfbb276f80467d884601ea6c92a401 [id=d8682a306ecd4a9f83cd49f3c358a536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.922295 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8483f83b5cd4a65a0fe9097c2c80cb3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:31.934931 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f287040b879642448367abe2b1c73cd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f287040b879642448367abe2b1c73cd9 already in progress: creating tablet
W20260811 18:57:31.935400 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2be6762e326448d7b677a5641e2b3103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be6762e326448d7b677a5641e2b3103 already in progress: creating tablet
I20260811 18:57:31.939574 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 97d5c52e361c4f08904475acf62dae2b (DEFAULT_TABLE table=default.Table_5727f2e5abe54ecaaeec9a52f1bd78da [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.940739 3712 tablet_service.cc:1467] Processing CreateTablet for tablet a8483f83b5cd4a65a0fe9097c2c80cb3 (DEFAULT_TABLE table=default.Table_aadfbb276f80467d884601ea6c92a401 [id=d8682a306ecd4a9f83cd49f3c358a536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.941694 3705 tablet_service.cc:1467] Processing CreateTablet for tablet cd04ae1903034a97a0bd086c1bd39a7d (DEFAULT_TABLE table=default.Table_6204a77bcccf427db16e32981a5621fb [id=0df5378171774f75b312a9205dae21dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.943202 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f89eebbc40234794a081734f834294ab (DEFAULT_TABLE table=default.Table_24c4140a00244c8d8446d91705a8d4a8 [id=d2508b5d391a441197b92e7f71fde5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.943466 4163 raft_consensus.cc:348] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.943815 4163 raft_consensus.cc:374] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.943970 4163 raft_consensus.cc:729] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.944384 4163 consensus_queue.cc:260] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:31.949278 4162 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:31.949434 4162 ts_tablet_manager.cc:1397] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.001s
I20260811 18:57:31.954689 3855 tablet_service.cc:1467] Processing CreateTablet for tablet cd04ae1903034a97a0bd086c1bd39a7d (DEFAULT_TABLE table=default.Table_6204a77bcccf427db16e32981a5621fb [id=0df5378171774f75b312a9205dae21dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.955240 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd04ae1903034a97a0bd086c1bd39a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.957549 4162 raft_consensus.cc:348] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.957803 4162 raft_consensus.cc:374] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:31.957871 4162 raft_consensus.cc:729] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:31.958112 4162 consensus_queue.cc:260] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:31.960970 4169 ts_tablet_manager.cc:1918] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:31.961159 4169 log.cc:1196] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b6f36ab6bb774257b5c2692c64eff63d
I20260811 18:57:31.967326 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8483f83b5cd4a65a0fe9097c2c80cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.968508 4169 ts_tablet_manager.cc:1939] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:31.970597 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd04ae1903034a97a0bd086c1bd39a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.975135 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89eebbc40234794a081734f834294ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.981935 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 722) took 4195 ms. Trace:
I20260811 18:57:31.982162 4169 rpcz_store.cc:276] 0811 18:57:27.786745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.786811 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:31.696655 (+3909844us) ts_tablet_manager.cc:1145] Deleting tablet b6f36ab6bb774257b5c2692c64eff63d
0811 18:57:31.696668 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:31.915315 (+218647us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.960855 (+ 45540us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:31.981899 (+ 21044us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3908775}
I20260811 18:57:31.992046 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b6f36ab6bb774257b5c2692c64eff63d (table default.Table_e0c0cbe2c2f846c2b85cda84670f448c [id=22529e4acc82410a93bca07dfda7d1b9]) successfully deleted
I20260811 18:57:31.993135 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 97d5c52e361c4f08904475acf62dae2b (DEFAULT_TABLE table=default.Table_5727f2e5abe54ecaaeec9a52f1bd78da [id=725e32a4a7234bb4bda3e501ce4da411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.993741 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d5c52e361c4f08904475acf62dae2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.995333 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:31.997381 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d5c52e361c4f08904475acf62dae2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:31.992422 4169 tablet_replica.cc:330] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:31.998664 4169 raft_consensus.cc:2229] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:31.998886 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f89eebbc40234794a081734f834294ab (DEFAULT_TABLE table=default.Table_24c4140a00244c8d8446d91705a8d4a8 [id=d2508b5d391a441197b92e7f71fde5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:31.999050 4169 raft_consensus.cc:2258] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:31.999437 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89eebbc40234794a081734f834294ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.000576 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2e58ff1b0c44ee8b71812ae6399b4da" } new_table_name: "default.Table_738fb827c1e14e659c918e66b8f3d079" modify_external_catalogs: true
I20260811 18:57:32.001914 4169 ts_tablet_manager.cc:1905] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.016832 3678 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.709s (SENT) [suppressed 27 similar messages]
I20260811 18:57:32.024920 4167 ts_tablet_manager.cc:1905] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.027009 4169 ts_tablet_manager.cc:1918] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.028321 4169 log.cc:1196] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:32.028990 4169 ts_tablet_manager.cc:1939] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.029863 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 724) took 4221 ms. Trace:
I20260811 18:57:32.030692 4163 ts_tablet_manager.cc:1428] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.113s user 0.002s sys 0.000s
I20260811 18:57:31.959146 4162 ts_tablet_manager.cc:1428] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:32.030527 4169 rpcz_store.cc:276] 0811 18:57:27.808665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.808732 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:31.982851 (+4174119us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:31.992365 (+ 9514us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.005773 (+ 13408us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.026885 (+ 21112us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.029842 (+ 2957us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4173826}
I20260811 18:57:32.037240 4162 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.038846 4162 tablet_bootstrap.cc:654] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.040110 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d2d7d986900244ddbc05dbe73a708dcf (table default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]) successfully deleted
I20260811 18:57:32.040215 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 723) took 4231 ms. Trace:
I20260811 18:57:32.040462 4169 rpcz_store.cc:276] 0811 18:57:27.808526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.818592 (+ 10066us) service_pool.cc:227] Handling call
0811 18:57:32.040032 (+4221440us) ts_tablet_manager.cc:1145] Deleting tablet 4093564a88264ed59c945523c9ddc9b3
0811 18:57:32.040046 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.040192 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4204355}
W20260811 18:57:32.041226 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4093564a88264ed59c945523c9ddc9b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4093564a88264ed59c945523c9ddc9b3 already in progress: creating tablet
I20260811 18:57:32.041225 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 725) took 3902 ms. Trace:
I20260811 18:57:32.041520 4169 rpcz_store.cc:276] 0811 18:57:28.139272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.139486 (+ 214us) service_pool.cc:227] Handling call
0811 18:57:32.041073 (+3901587us) ts_tablet_manager.cc:1145] Deleting tablet ae7da0868b114052ba421801b76fb33c
0811 18:57:32.041084 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.041197 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3901325}
I20260811 18:57:32.042464 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 726) took 3903 ms. Trace:
W20260811 18:57:32.042409 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ae7da0868b114052ba421801b76fb33c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae7da0868b114052ba421801b76fb33c already in progress: creating tablet
I20260811 18:57:32.042656 4169 rpcz_store.cc:276] 0811 18:57:28.139406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.139804 (+ 398us) service_pool.cc:227] Handling call
0811 18:57:32.042143 (+3902339us) ts_tablet_manager.cc:1145] Deleting tablet 6d988ef7e9dd43158d107998ea4d82b9
0811 18:57:32.042153 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.042445 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3902189}
W20260811 18:57:32.043213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6d988ef7e9dd43158d107998ea4d82b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d988ef7e9dd43158d107998ea4d82b9 already in progress: creating tablet
I20260811 18:57:32.043426 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 727) took 3898 ms. Trace:
I20260811 18:57:32.043548 4169 rpcz_store.cc:276] 0811 18:57:28.145398 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.145455 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:32.043132 (+3897677us) ts_tablet_manager.cc:1145] Deleting tablet b6c70b4163634984bd02dd9b47ffb472
0811 18:57:32.043144 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.043404 (+ 260us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3897473}
W20260811 18:57:32.044332 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b6c70b4163634984bd02dd9b47ffb472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6c70b4163634984bd02dd9b47ffb472 already in progress: creating tablet
I20260811 18:57:32.036933 4163 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.047806 4163 tablet_bootstrap.cc:654] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.044443 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 728) took 3872 ms. Trace:
I20260811 18:57:32.050469 4169 rpcz_store.cc:276] 0811 18:57:28.172188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.172328 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:32.044149 (+3871821us) ts_tablet_manager.cc:1145] Deleting tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
0811 18:57:32.044175 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.044409 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3871469}
I20260811 18:57:32.051134 4169 tablet_replica.cc:330] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:32.051225 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca already in progress: creating tablet
I20260811 18:57:32.051541 4169 raft_consensus.cc:2229] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:32.052237 3798 tablet.cc:2369] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:32.052008 4169 raft_consensus.cc:2258] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.054068 4169 ts_tablet_manager.cc:1905] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.063015 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46196.1us
I20260811 18:57:32.063495 4162 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.063661 4162 ts_tablet_manager.cc:1397] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
W20260811 18:57:32.067624 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 15dd14e2b4bd4a08b5b08f532a55291b has no consensus state
W20260811 18:57:32.068348 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4af28c051e8243eaa73d4a5ed1abcea6 consensus state has no leader
I20260811 18:57:32.069221 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4133a1bbf6474492bbf3683ca9608a52" } new_table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" modify_external_catalogs: true
I20260811 18:57:32.069846 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0da5d975ff884b9bb82d239828666af7" } modify_external_catalogs: true
I20260811 18:57:32.070672 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" } modify_external_catalogs: true
I20260811 18:57:32.071094 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26909aac9f0d4aee9b323b80abac8420" } modify_external_catalogs: true
I20260811 18:57:32.072196 4163 tablet_bootstrap.cc:492] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.064529 4162 raft_consensus.cc:348] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.073112 4163 ts_tablet_manager.cc:1397] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.007s
I20260811 18:57:32.073381 4162 raft_consensus.cc:374] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.074362 4163 raft_consensus.cc:348] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.074759 4163 raft_consensus.cc:374] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.074913 4163 raft_consensus.cc:729] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.075263 4163 consensus_queue.cc:260] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.075553 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" } modify_external_catalogs: true
I20260811 18:57:32.075940 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_267e0bd672f9419fbd1a46b93d0d8169" } modify_external_catalogs: true
I20260811 18:57:32.078470 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" } modify_external_catalogs: true
I20260811 18:57:32.078768 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c" } modify_external_catalogs: true
I20260811 18:57:32.079259 4163 ts_tablet_manager.cc:1428] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:57:32.079627 4163 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.074290 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_73bf9f47862a4f03ac5e35afacfc2463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.083878 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73bf9f47862a4f03ac5e35afacfc2463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.082295 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0da5d975ff884b9bb82d239828666af7" } modify_external_catalogs: true
I20260811 18:57:32.068348 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c7a814de93aa4d49a3ab5c7d2b167976" } new_table_name: "default.Table_0edbfd08288d4926a353671dfbf84fd5" modify_external_catalogs: true
I20260811 18:57:32.091142 4163 tablet_bootstrap.cc:654] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.098786 4162 raft_consensus.cc:729] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.099253 4162 consensus_queue.cc:260] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.100858 3942 tablet.cc:2369] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:32.106451 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fdab1c3c431c4dceb08b00eabcfa1ceb consensus state has no leader
I20260811 18:57:32.104300 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b752cb750e9447b092ecac296b5f836e" } modify_external_catalogs: true
I20260811 18:57:32.107519 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b752cb750e9447b092ecac296b5f836e" } modify_external_catalogs: true
#8 0xae894d in kudu::tools::(anonymous namespace)::TServerRun(kudu::tools::RunnerContext const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action_tserver.cc:114:10
I20260811 18:57:32.110563 4162 ts_tablet_manager.cc:1428] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260811 18:57:32.114296 4167 ts_tablet_manager.cc:1918] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.114478 4167 log.cc:1196] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:32.114953 4167 ts_tablet_manager.cc:1939] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.116884 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.117386 4162 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.119105 4162 tablet_bootstrap.cc:654] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.120949 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 734) took 4466 ms. Trace:
I20260811 18:57:32.121068 4167 rpcz_store.cc:276] 0811 18:57:27.654614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.654677 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:31.909664 (+4254987us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:31.909674 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.029235 (+119561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.090342 (+ 61107us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.120921 (+ 30579us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4254722}
I20260811 18:57:32.121973 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cbf6ce167fb348be8d61372791f0cbca (table default.Table_1558f156b5eb46a1858ca7673e4a9747 [id=dda7fb95dc1549ddbc88d8525ff023e8]) successfully deleted
I20260811 18:57:32.121510 4167 tablet_replica.cc:330] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.122471 4167 raft_consensus.cc:2229] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.122679 4167 raft_consensus.cc:2258] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.124862 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1dc240fd92f941489de07a83cdafb797"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.121585 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.125769 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1dc240fd92f941489de07a83cdafb797 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.128226 4167 ts_tablet_manager.cc:1905] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.131543 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:32.152248 4162 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.152463 4162 ts_tablet_manager.cc:1397] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.003s
I20260811 18:57:32.153865 4162 raft_consensus.cc:348] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.154158 4162 raft_consensus.cc:374] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.154235 4162 raft_consensus.cc:729] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.154752 4162 consensus_queue.cc:260] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.161288 4162 ts_tablet_manager.cc:1428] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260811 18:57:32.152851 5376 raft_consensus.cc:482] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.171770 4162 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.176730 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cdb92902c2584751af77d3117f52cb4e
I20260811 18:57:32.177019 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ccc63b46a4e346f79d1e03a82490794c
I20260811 18:57:32.177749 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet e1ad3860d03f46baadd5ede854f1a909 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:32.181684 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc63b46a4e346f79d1e03a82490794c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.181977 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb92902c2584751af77d3117f52cb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.185613 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1bfaa8ab4fcd40efb60e1718e424bccf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.189450 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.191916 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.193615 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 99de8ce45e514ff291cb9e5c0a4c00d1
I20260811 18:57:32.194037 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 14799a6f61a946008acee8a09d050c1e
W20260811 18:57:32.194947 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f06e5b6a30764caa978ba0022ca1d6bc consensus state has no leader
I20260811 18:57:32.195636 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 55f97f37fa7c483cacaeed54f93e7a24
W20260811 18:57:32.196542 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5a184b83373a4c089f2ca0169cbfcf66 consensus state has no leader
I20260811 18:57:32.197496 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.197773 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 14799a6f61a946008acee8a09d050c1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.198587 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.199213 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bfaa8ab4fcd40efb60e1718e424bccf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.168560 5376 raft_consensus.cc:504] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.209338 5376 leader_election.cc:288] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.181675 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb92902c2584751af77d3117f52cb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.212792 4162 tablet_bootstrap.cc:654] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.215606 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ccc63b46a4e346f79d1e03a82490794c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.217167 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 14799a6f61a946008acee8a09d050c1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.219563 4169 ts_tablet_manager.cc:1918] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.219738 4169 log.cc:1196] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/799ba9a044e145ec89d0025c03f391ea
I20260811 18:57:32.220103 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.226568 4169 ts_tablet_manager.cc:1939] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.227124 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 734) took 3784 ms. Trace:
I20260811 18:57:32.227315 4169 rpcz_store.cc:276] 0811 18:57:28.442482 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.443442 (+ 960us) service_pool.cc:227] Handling call
0811 18:57:32.051063 (+3607621us) ts_tablet_manager.cc:1145] Deleting tablet 799ba9a044e145ec89d0025c03f391ea
0811 18:57:32.051077 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.068521 (+ 17444us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.219427 (+150906us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.227096 (+ 7669us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3607381}
I20260811 18:57:32.228842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 735) took 3784 ms. Trace:
I20260811 18:57:32.229036 4169 rpcz_store.cc:276] 0811 18:57:28.444764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.444818 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:32.227792 (+3782974us) ts_tablet_manager.cc:1145] Deleting tablet d8146f47d9ae48889ac7efc954d4a6b7
0811 18:57:32.227804 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.228817 (+ 1013us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3782788}
I20260811 18:57:32.229604 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 736) took 3738 ms. Trace:
I20260811 18:57:32.229763 4169 rpcz_store.cc:276] 0811 18:57:28.491405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.491516 (+ 111us) service_pool.cc:227] Handling call
0811 18:57:32.229427 (+3737911us) ts_tablet_manager.cc:1145] Deleting tablet a38c172a9fef4a3ab8ee188335a7c97e
0811 18:57:32.229441 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.229574 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3737631}
I20260811 18:57:32.230288 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 737) took 3737 ms. Trace:
I20260811 18:57:32.230496 4169 rpcz_store.cc:276] 0811 18:57:28.493048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.493229 (+ 181us) service_pool.cc:227] Handling call
0811 18:57:32.230160 (+3736931us) ts_tablet_manager.cc:1145] Deleting tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
0811 18:57:32.230174 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.230267 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3736695}
I20260811 18:57:32.230860 5091 raft_consensus.cc:482] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.230973 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 738) took 3723 ms. Trace:
I20260811 18:57:32.231081 4169 rpcz_store.cc:276] 0811 18:57:28.507960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.508068 (+ 108us) service_pool.cc:227] Handling call
0811 18:57:32.230844 (+3722776us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:32.230858 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.230955 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3722533}
I20260811 18:57:32.231057 5091 raft_consensus.cc:504] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.231642 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 741) took 3604 ms. Trace:
I20260811 18:57:32.231763 4169 rpcz_store.cc:276] 0811 18:57:28.627533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.630131 (+ 2598us) service_pool.cc:227] Handling call
0811 18:57:32.231483 (+3601352us) ts_tablet_manager.cc:1145] Deleting tablet f13a14bd64704798bf6de4fc05201a31
0811 18:57:32.231497 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.231624 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3601036}
I20260811 18:57:32.232123 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 751) took 3514 ms. Trace:
I20260811 18:57:32.232293 5091 leader_election.cc:288] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.232211 4169 rpcz_store.cc:276] 0811 18:57:28.717828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.719682 (+ 1854us) service_pool.cc:227] Handling call
0811 18:57:32.231983 (+3512301us) ts_tablet_manager.cc:1145] Deleting tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
0811 18:57:32.231995 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.232107 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3512046}
I20260811 18:57:32.232569 4169 tablet_replica.cc:330] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.233268 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d4c26da86b49b8a37f2289f4ef6614" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.234688 3676 leader_election.cc:341] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:32.235952 3824 leader_election.cc:341] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c08436f33db043b1bc6395cd92102e59
I20260811 18:57:32.236225 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.236794 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 799ba9a044e145ec89d0025c03f391ea (table default.Table_56b89b2b2c4b45768b2866c74d9f1390 [id=7736472828a3499c8ad8026e164b9759]) successfully deleted
W20260811 18:57:32.237213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d8146f47d9ae48889ac7efc954d4a6b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8146f47d9ae48889ac7efc954d4a6b7 already in progress: creating tablet
W20260811 18:57:32.237537 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a38c172a9fef4a3ab8ee188335a7c97e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38c172a9fef4a3ab8ee188335a7c97e already in progress: creating tablet
W20260811 18:57:32.237823 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c already in progress: creating tablet
W20260811 18:57:32.238157 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cf6c8d542ae2489ba466ffadd518db12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf6c8d542ae2489ba466ffadd518db12 already in progress: creating tablet
W20260811 18:57:32.238791 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f13a14bd64704798bf6de4fc05201a31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13a14bd64704798bf6de4fc05201a31 already in progress: creating tablet
W20260811 18:57:32.239143 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 already in progress: creating tablet
I20260811 18:57:32.244575 4167 ts_tablet_manager.cc:1918] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.245649 4167 log.cc:1196] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f335e50ae57547ed80b144129faaffbb
I20260811 18:57:32.247768 4167 ts_tablet_manager.cc:1939] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.233352 4169 raft_consensus.cc:2229] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.251678 4169 raft_consensus.cc:2258] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.252516 4169 ts_tablet_manager.cc:1905] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.252257 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 735) took 4597 ms. Trace:
I20260811 18:57:32.256919 4162 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:32.257094 4162 ts_tablet_manager.cc:1397] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.086s user 0.006s sys 0.001s
I20260811 18:57:32.258458 4162 raft_consensus.cc:348] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:32.258903 4162 raft_consensus.cc:374] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.259033 4162 raft_consensus.cc:729] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.259500 4162 consensus_queue.cc:260] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:32.262276 3678 leader_election.cc:334] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.166s (SENT)
I20260811 18:57:32.264660 3678 leader_election.cc:302] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:32.264948 3678 raft_consensus.cc:2670] T a443fe209556489abb47f4faa5d696e5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.253592 4167 rpcz_store.cc:276] 0811 18:57:27.654741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.654792 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.121459 (+4466667us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:32.121473 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.190206 (+ 68733us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.244040 (+ 53834us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.251918 (+ 7878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4466482}
I20260811 18:57:32.268556 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f335e50ae57547ed80b144129faaffbb (table default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]) successfully deleted
I20260811 18:57:32.270200 4163 tablet_bootstrap.cc:492] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.270673 4163 ts_tablet_manager.cc:1397] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.191s user 0.004s sys 0.011s
I20260811 18:57:32.271730 4163 raft_consensus.cc:348] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.271546 4167 tablet_replica.cc:330] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.272151 4163 raft_consensus.cc:374] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.272298 4163 raft_consensus.cc:729] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.272670 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e1ad3860d03f46baadd5ede854f1a909
I20260811 18:57:32.272711 4163 consensus_queue.cc:260] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.274298 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ad3860d03f46baadd5ede854f1a909 with delete_type TABLET_DATA_DELETED (replaced by tablet 084b60eeb6b64e1da7336203d75770f4) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.274324 3553 catalog_manager.cc:6565] ReplaceTablet: tablet e1ad3860d03f46baadd5ede854f1a909 deleted and replaced by tablet 084b60eeb6b64e1da7336203d75770f4
I20260811 18:57:32.277391 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:32.277190 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4e35d031a62148d582b7bdb0240f63da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.278111 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e35d031a62148d582b7bdb0240f63da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.280308 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4dc767eb645648aebd0aacf2d46f9c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.281075 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dc767eb645648aebd0aacf2d46f9c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.286652 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d584e1cc7d8b45be912d8610e391f4f7" } new_table_name: "default.Table_cf2da094466b4bf597833b58e3402047" modify_external_catalogs: true
I20260811 18:57:32.289326 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6531b518884b4b9b8ecfdc09c2a0e15f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.278646 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.278705 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ad3860d03f46baadd5ede854f1a909 with delete_type TABLET_DATA_DELETED (replaced by tablet 084b60eeb6b64e1da7336203d75770f4) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.290162 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6531b518884b4b9b8ecfdc09c2a0e15f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.272470 4162 ts_tablet_manager.cc:1428] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:32.289319 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ad9f3fa268174f9aa74bcee2360bf4d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.291059 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.294867 4167 raft_consensus.cc:2229] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.298116 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f980c9b98994253b590548d86f6d145" } new_table_name: "default.Table_022048402d3f4ad2906b4c19cb3882bb" modify_external_catalogs: true
I20260811 18:57:32.297667 4162 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:32.297797 4167 raft_consensus.cc:2258] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.302763 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3309d84857df4970a9891fd64c181bf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.303686 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3309d84857df4970a9891fd64c181bf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.304538 4167 ts_tablet_manager.cc:1905] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#9 0x921350 in std::_Function_handler<kudu::Status (kudu::tools::RunnerContext const&), kudu::Status (*)(kudu::tools::RunnerContext const&)>::_M_invoke(std::_Any_data const&, kudu::tools::RunnerContext const&) ../../../include/c++/8/bits/std_function.h:282:9
I20260811 18:57:32.313313 4163 ts_tablet_manager.cc:1428] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20260811 18:57:32.315510 4163 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.317090 4163 tablet_bootstrap.cc:654] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.320792 4162 tablet_bootstrap.cc:654] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.324882 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a09ba72daf1440ba3921b736aeb429e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.325008 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_949ccbca1e8b44b79578377f67f2422c" } new_table_name: "default.Table_292cbd0196af4a9a8164ead40c13b401" modify_external_catalogs: true
W20260811 18:57:32.325677 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a09ba72daf1440ba3921b736aeb429e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:32.327600 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae33c49dab3444988738f31180de35a consensus state has no leader
I20260811 18:57:32.330806 5091 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.331025 5091 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.331877 5091 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.332690 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.333269 3676 leader_election.cc:341] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.337491 5376 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.337652 5376 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.338476 5376 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.338979 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.339325 3740 raft_consensus.cc:2454] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.339969 3824 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.340302 5376 raft_consensus.cc:2781] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.340405 5376 raft_consensus.cc:482] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.340485 5376 raft_consensus.cc:3037] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.343870 5376 raft_consensus.cc:504] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.344681 5376 leader_election.cc:288] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.347019 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3ec38f9b55489ba00ed76f9e4456f5" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.347268 3740 raft_consensus.cc:3037] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.350303 3740 raft_consensus.cc:2454] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.351128 3824 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.351476 5376 raft_consensus.cc:2781] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.352133 5376 raft_consensus.cc:686] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.352972 5376 consensus_queue.cc:237] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.354789 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a255a0922a92419191f18730d8b335bc consensus state has no leader
W20260811 18:57:32.362318 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75af8f21dcc7443fabd13e06061d19a6 consensus state has no leader
I20260811 18:57:32.370150 4163 tablet_bootstrap.cc:492] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.370313 4163 ts_tablet_manager.cc:1397] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.002s
I20260811 18:57:32.371093 4163 raft_consensus.cc:348] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.373142 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet bf2816cc2dfe45f0815ce50a5a929324 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:32.377439 5283 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.377620 5283 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.378051 4163 raft_consensus.cc:374] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.378152 4163 raft_consensus.cc:729] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.378398 4163 consensus_queue.cc:260] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.378412 5283 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.383893 4169 ts_tablet_manager.cc:1918] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.384198 4169 log.cc:1196] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/06b8f2a97c0f46ea8c56b3de2b97e60c
I20260811 18:57:32.385030 4169 ts_tablet_manager.cc:1939] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.385679 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 752) took 3595 ms. Trace:
I20260811 18:57:32.385716 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25" } modify_external_catalogs: true
I20260811 18:57:32.385871 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a13d9e49bc748e6b820a614e55ba4c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.386178 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e8554df7eb5f4fbbbe828834c1d45f25" } modify_external_catalogs: true
I20260811 18:57:32.385876 4169 rpcz_store.cc:276] 0811 18:57:28.789815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.790940 (+ 1125us) service_pool.cc:227] Handling call
0811 18:57:32.232510 (+3441570us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:32.232523 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.325458 (+ 92935us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.329092 (+ 3634us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.385642 (+ 56550us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3435186}
W20260811 18:57:32.386869 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a13d9e49bc748e6b820a614e55ba4c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.387269 4169 tablet_replica.cc:330] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.387172 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d" } modify_external_catalogs: true
I20260811 18:57:32.384754 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.387636 3740 raft_consensus.cc:2454] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.387687 4169 raft_consensus.cc:2229] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.388053 3824 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.388741 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (table default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]) successfully deleted
I20260811 18:57:32.391515 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b30b503bf0a841d6a0061c6430b70b1d" } modify_external_catalogs: true
I20260811 18:57:32.397936 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a09ba72daf1440ba3921b736aeb429e" } modify_external_catalogs: true
I20260811 18:57:32.397914 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_57453b6f4942454da731710b8510f4a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.398371 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a09ba72daf1440ba3921b736aeb429e" } modify_external_catalogs: true
W20260811 18:57:32.399040 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57453b6f4942454da731710b8510f4a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.400795 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4" } modify_external_catalogs: true
I20260811 18:57:32.401166 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b485cd94aa7c4642a0b3d0021cef63c4" } modify_external_catalogs: true
I20260811 18:57:32.404904 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_07499669fc704a31830a06152cd615cf" } modify_external_catalogs: true
I20260811 18:57:32.405180 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_07499669fc704a31830a06152cd615cf" } modify_external_catalogs: true
I20260811 18:57:32.406615 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3a0329aed6694593bc21571dd8f59670" } new_table_name: "default.Table_809eb8f582f14652bb5f9cd7301260c0" modify_external_catalogs: true
I20260811 18:57:32.409732 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5eb58df7cc2042f69300d10a7999719b" } new_table_name: "default.Table_8a69fc9e8b464e1cb431dcb6b30858e2" modify_external_catalogs: true
I20260811 18:57:32.419229 5283 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.419229 5286 raft_consensus.cc:2781] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.419422 5286 raft_consensus.cc:482] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.419529 5286 raft_consensus.cc:3037] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.419657 4169 pending_rounds.cc:62] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:32.419714 4169 pending_rounds.cc:68] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400161362292736 op_type: NO_OP noop_request { }
I20260811 18:57:32.419858 4169 raft_consensus.cc:2866] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:32.419888 4163 ts_tablet_manager.cc:1428] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.049s user 0.002s sys 0.000s
I20260811 18:57:32.419955 4169 raft_consensus.cc:2258] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.420182 4163 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.420704 4169 ts_tablet_manager.cc:1905] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.420742 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1a83eafb62054d789e5cf28ea8380f0a
I20260811 18:57:32.421995 4163 tablet_bootstrap.cc:654] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.419425 5283 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.423249 5283 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.424115 5286 raft_consensus.cc:504] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.426260 5286 leader_election.cc:288] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.426931 3551 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 79c67bc70d0d4ff892734ca757ada5a8
I20260811 18:57:32.426880 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1923ce0049448fe4e28f2dd258dc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.427114 3743 raft_consensus.cc:3037] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.427696 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.428550 3740 raft_consensus.cc:2454] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.430601 3824 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.430995 5286 raft_consensus.cc:2781] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.431152 5286 raft_consensus.cc:482] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260811 18:57:32.431201 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 825b6743798144deb24330304d6b5d2b consensus state has no leader
I20260811 18:57:32.431293 5286 raft_consensus.cc:3037] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.431665 3743 raft_consensus.cc:2454] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:32.431522 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0ff928a633e4cae9d49229b989216d8 consensus state has no leader
I20260811 18:57:32.432479 3824 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.432827 5283 raft_consensus.cc:2781] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.432969 5283 raft_consensus.cc:686] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.433307 5283 consensus_queue.cc:237] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.436359 5286 raft_consensus.cc:504] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.438921 5286 leader_election.cc:288] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.439550 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65234acc3d5a462683edbe8f223eeefc" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.439847 3743 raft_consensus.cc:3037] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.447130 3743 raft_consensus.cc:2454] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.448176 3824 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.448541 5286 raft_consensus.cc:2781] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.448822 5286 raft_consensus.cc:686] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.449142 5286 consensus_queue.cc:237] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.470605 5091 raft_consensus.cc:482] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.470803 5091 raft_consensus.cc:504] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.471535 5091 leader_election.cc:288] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.472286 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9bdc606be40558dfa33e1e7a8ebc7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:32.472883 3676 leader_election.cc:341] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.488894 4169 ts_tablet_manager.cc:1918] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.489172 4169 log.cc:1196] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ad424c8843ff434ba0449a86849bb5a0
I20260811 18:57:32.489727 4169 ts_tablet_manager.cc:1939] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.490387 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 753) took 3698 ms. Trace:
I20260811 18:57:32.496580 4163 tablet_bootstrap.cc:492] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.496886 4163 ts_tablet_manager.cc:1397] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.011s sys 0.000s
I20260811 18:57:32.495286 4167 ts_tablet_manager.cc:1918] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:32.498026 4163 raft_consensus.cc:348] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.498785 4167 log.cc:1196] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:32.502436 4163 raft_consensus.cc:374] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.502835 4163 raft_consensus.cc:729] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.503351 4163 consensus_queue.cc:260] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.504784 3678 leader_election.cc:334] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.218s (SENT)
I20260811 18:57:32.509227 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e691dcf4a764665aa9275bfa441105a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.510080 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e691dcf4a764665aa9275bfa441105a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.512475 4163 ts_tablet_manager.cc:1428] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260811 18:57:32.512828 4163 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.522421 4163 tablet_bootstrap.cc:654] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.492487 4169 rpcz_store.cc:276] 0811 18:57:28.791606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.791662 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:32.387212 (+3595550us) ts_tablet_manager.cc:1145] Deleting tablet ad424c8843ff434ba0449a86849bb5a0
0811 18:57:32.387228 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.447789 (+ 60561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.488771 (+ 40982us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.490359 (+ 1588us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31013,"tablet-delete.queue_time_us":3589796}
I20260811 18:57:32.523322 4169 tablet_replica.cc:330] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.523325 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ad424c8843ff434ba0449a86849bb5a0 (table default.Table_91346cfabb664d0ebfc876dacc125c50 [id=42de2add2fcc4c66b3d11e782adb100f]) successfully deleted
I20260811 18:57:32.523554 4169 raft_consensus.cc:2229] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.523767 4169 raft_consensus.cc:2258] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.524516 4169 ts_tablet_manager.cc:1905] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.532513 4163 tablet_bootstrap.cc:492] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.533684 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7075f060dbd942cca40c448af9eb5825" } new_table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" modify_external_catalogs: true
I20260811 18:57:32.533845 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a17ac8f677ce46efaade718a851a016d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.534790 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a17ac8f677ce46efaade718a851a016d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.536583 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e97d2b678784537a3e986a742e9f88b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.535104 5376 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.537473 5376 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.537711 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.538117 5376 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.538393 4163 ts_tablet_manager.cc:1397] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
W20260811 18:57:32.538450 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:32.540445 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e97d2b678784537a3e986a742e9f88b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.542296 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.542632 3743 raft_consensus.cc:2454] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.544066 3824 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.544345 5376 raft_consensus.cc:2781] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.544452 5376 raft_consensus.cc:482] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.544533 5376 raft_consensus.cc:3037] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.548220 5376 raft_consensus.cc:504] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.548821 5376 leader_election.cc:288] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:32.548659 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a7bc3ed83682403b84a5a23d14cf1c3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.549518 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7bc3ed83682403b84a5a23d14cf1c3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.551380 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc5448a555944dfba8ccae09e25cbe9" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.551620 3743 raft_consensus.cc:3037] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.553025 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db613e17da86441bb1ab37b46e730db9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.553519 4169 ts_tablet_manager.cc:1918] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.553647 4169 log.cc:1196] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/515d5b5afbd74ec5ab90a503aef2f4fb
W20260811 18:57:32.553714 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db613e17da86441bb1ab37b46e730db9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.554085 4169 ts_tablet_manager.cc:1939] T 515d5b5afbd74ec5ab90a503aef2f4fb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.556671 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]). Expected version 2 got 0
I20260811 18:57:32.557010 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:32.562383 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.564533 3743 raft_consensus.cc:2454] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.566319 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4e412149fc55445bb720ffed59a05618" } new_table_name: "default.Table_0f2efce46eed4e0b9626c257cbfe711c" modify_external_catalogs: true
I20260811 18:57:32.570466 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 7538ee8c4da948d480f907de8953bb9a (DEFAULT_TABLE table=default.Table_a273b6a16afa44d196b4654b067735a5 [id=babc4abc6e5c4bb0806b988aa2f5a7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.571019 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7538ee8c4da948d480f907de8953bb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.575691 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 0c108daf7ba848e585b07b3f0ba78bdc (DEFAULT_TABLE table=default.Table_4013b1905ea74bc6a06f9468d9025570 [id=e25121ff4ff6437786d67ff375ccc427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.576229 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c108daf7ba848e585b07b3f0ba78bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.587445 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 60fe0c1500c143c3a5fcefeaed303991 (DEFAULT_TABLE table=default.Table_2e0b184f30d7464fbb8120d29bd0a80b [id=292b0341d47f4ad29a78a33fe7f02a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.587958 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fe0c1500c143c3a5fcefeaed303991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.558634 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 122b7898fea44ef78f269b1505d8ddc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.562130 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 7538ee8c4da948d480f907de8953bb9a (DEFAULT_TABLE table=default.Table_a273b6a16afa44d196b4654b067735a5 [id=babc4abc6e5c4bb0806b988aa2f5a7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.592902 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7538ee8c4da948d480f907de8953bb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.565196 3824 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.594249 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: OK, instance_id { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" instance_seqno: 1786474650422925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:32.594631 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:32.594614 5376 raft_consensus.cc:2781] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.594743 5376 raft_consensus.cc:686] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.595031 5376 consensus_queue.cc:237] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:32.597996 3678 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.640s (SENT)
I20260811 18:57:32.598153 3678 leader_election.cc:302] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:32.598759 5091 raft_consensus.cc:2726] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:32.561875 4163 raft_consensus.cc:348] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.598917 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 60fe0c1500c143c3a5fcefeaed303991 (DEFAULT_TABLE table=default.Table_2e0b184f30d7464fbb8120d29bd0a80b [id=292b0341d47f4ad29a78a33fe7f02a87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.599400 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fe0c1500c143c3a5fcefeaed303991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.603473 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0c108daf7ba848e585b07b3f0ba78bdc (DEFAULT_TABLE table=default.Table_4013b1905ea74bc6a06f9468d9025570 [id=e25121ff4ff6437786d67ff375ccc427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.603873 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c108daf7ba848e585b07b3f0ba78bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.606976 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2460d58bb949e4a3a2c9f74dd800a8 (DEFAULT_TABLE table=default.Table_619d1e7c0e384107962b41e86e45c45b [id=1f4ab61624c84bde9f6689e2716a8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.607391 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2460d58bb949e4a3a2c9f74dd800a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.610390 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 756) took 3768 ms. Trace:
I20260811 18:57:32.610580 4169 rpcz_store.cc:276] 0811 18:57:28.841629 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.844781 (+ 3152us) service_pool.cc:227] Handling call
0811 18:57:32.523260 (+3678479us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:32.523275 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.550153 (+ 26878us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.553421 (+ 3268us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.610368 (+ 56947us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3678184}
I20260811 18:57:32.611192 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 515d5b5afbd74ec5ab90a503aef2f4fb (table default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]) successfully deleted
I20260811 18:57:32.611848 4169 tablet_replica.cc:330] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.612128 4169 raft_consensus.cc:2229] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.598917 4163 raft_consensus.cc:374] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.618291 4163 raft_consensus.cc:729] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:32.620924 3824 consensus_peers.cc:487] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f335e50ae57547ed80b144129faaffbb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:32.621232 5286 consensus_queue.cc:1035] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:32.612473 4169 raft_consensus.cc:2258] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:32.622488 3678 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.138s (SENT)
I20260811 18:57:32.622637 3678 leader_election.cc:302] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:32.622805 3678 raft_consensus.cc:2670] T ad424c8843ff434ba0449a86849bb5a0 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.623489 4169 ts_tablet_manager.cc:1905] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.620450 4163 consensus_queue.cc:260] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.635442 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfa85fca4bd41dc83acb02ac610cf97 (DEFAULT_TABLE table=default.Table_2698f49dd4b64fce9a411b1c3307abf2 [id=c30db61b225d42e491bc9529e2b91cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.635990 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfa85fca4bd41dc83acb02ac610cf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.638480 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 844bcd16f06c4788af9c0d7d320464aa
I20260811 18:57:32.639493 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2033e26987554c839ae47bd350d82580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.640264 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2033e26987554c839ae47bd350d82580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.640060 3547 catalog_manager.cc:5495] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
W20260811 18:57:32.642067 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3195082643264e0d90f70e1007ab9b24 consensus state has no leader
I20260811 18:57:32.642710 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_58b2359b8659428190fe3cd7688c1d4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.643088 4163 ts_tablet_manager.cc:1428] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.105s user 0.003s sys 0.000s
I20260811 18:57:32.643498 4163 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:32.643522 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58b2359b8659428190fe3cd7688c1d4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.645462 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0a5b06fa572045f3a5fc5a0b123d77cc
I20260811 18:57:32.646327 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c4384a8ba27a4a7696a6051325ddede2" } new_table_name: "default.Table_2ce145eabbad4a958c6799476314823c" modify_external_catalogs: true
I20260811 18:57:32.650650 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b (DEFAULT_TABLE table=default.Table_19742da9decc46818ae2cd5debeee68c [id=8f3414e814ed42cca911730ff2785356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.651069 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a3973dfdaac746a6af7a402277d2711b (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.651309 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15dd14e2b4bd4a08b5b08f532a55291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.651453 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 844bcd16f06c4788af9c0d7d320464aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.650784 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.651722 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3973dfdaac746a6af7a402277d2711b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.651618 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fae8f41d600f4fbfaed39f695cf6672a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:32.651871 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b50efab3c782447cad8e45b1fa51aa1f (DEFAULT_TABLE table=default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 [id=1f5525b9c44f471dbabaed8159de8784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.652218 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50efab3c782447cad8e45b1fa51aa1f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:32.652432 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fae8f41d600f4fbfaed39f695cf6672a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.654709 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:32.655424 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:32.658726 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b (DEFAULT_TABLE table=default.Table_19742da9decc46818ae2cd5debeee68c [id=8f3414e814ed42cca911730ff2785356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.658962 3712 tablet_service.cc:1467] Processing CreateTablet for tablet b50efab3c782447cad8e45b1fa51aa1f (DEFAULT_TABLE table=default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6 [id=1f5525b9c44f471dbabaed8159de8784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.659204 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15dd14e2b4bd4a08b5b08f532a55291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.659518 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50efab3c782447cad8e45b1fa51aa1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.663306 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a3973dfdaac746a6af7a402277d2711b (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.663750 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3973dfdaac746a6af7a402277d2711b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.666551 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5b06fa572045f3a5fc5a0b123d77cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.668939 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 844bcd16f06c4788af9c0d7d320464aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:32.685328 3953 kernel_stack_watchdog.cc:198] Thread 4165 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 21828ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32bfcef kudu::ThreadPool::Submit() at ??:0
@ 0x7fc7be16006d kudu::log::SegmentAllocator::AsyncAllocateSegmentUnlocked() at ??:0
@ 0x7fc7be15f0bf kudu::log::SegmentAllocator::AllocateSegmentAndRollOver() at ??:0
@ 0x7fc7be15eca6 kudu::log::SegmentAllocator::Init() at ??:0
@ 0x7fc7be166a02 kudu::log::Log::Init() at ??:0
@ 0x7fc7be165dbf kudu::log::Log::Open() at ??:0
@ 0x7fc7bf9f72ee kudu::tablet::TabletBootstrap::OpenNewLog() at ??:0
I20260811 18:57:32.689956 5091 raft_consensus.cc:482] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.689916 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 4abd6e9a89814dbaa84316475bbe1ae5 (DEFAULT_TABLE table=default.Table_04e3e316fa374868a4be1497035d91e8 [id=dcf9bb82d0794200a6f82208c97b2532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.690156 5091 raft_consensus.cc:504] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.690943 5091 leader_election.cc:288] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.693892 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7bfa85fca4bd41dc83acb02ac610cf97 (DEFAULT_TABLE table=default.Table_2698f49dd4b64fce9a411b1c3307abf2 [id=c30db61b225d42e491bc9529e2b91cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.694777 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bfa85fca4bd41dc83acb02ac610cf97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.684165 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b278420f8d2f4be8b973de3eb756f69a (DEFAULT_TABLE table=default.Table_c284a387f502480c8faa36ea0e40a8f7 [id=bc157b21c5424118bc3d36507b8d4a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.696789 3717 tablet_service.cc:1467] Processing CreateTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f (DEFAULT_TABLE table=default.Table_ddf22cbb415c485ebc1407d9104b715d [id=e01f868b4ac2491ea511820aa199319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.702286 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 4abd6e9a89814dbaa84316475bbe1ae5 (DEFAULT_TABLE table=default.Table_04e3e316fa374868a4be1497035d91e8 [id=dcf9bb82d0794200a6f82208c97b2532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.702741 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abd6e9a89814dbaa84316475bbe1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.702827 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f (DEFAULT_TABLE table=default.Table_ddf22cbb415c485ebc1407d9104b715d [id=e01f868b4ac2491ea511820aa199319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.703168 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86bfa65a8774b51bc7e7fe59442bf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.703099 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1374a80d15444cb0faff339275aa8d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:32.703737 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b9bdd685e23249119bbeac4997cbbf9c" } new_table_name: "default.Table_78bb9ed48dc34a4ea11b23eec0467238" modify_external_catalogs: true
I20260811 18:57:32.706127 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" } modify_external_catalogs: true
I20260811 18:57:32.706444 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ff02853da3f1406c90f961567adf92bf" } modify_external_catalogs: true
I20260811 18:57:32.707315 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b278420f8d2f4be8b973de3eb756f69a (DEFAULT_TABLE table=default.Table_c284a387f502480c8faa36ea0e40a8f7 [id=bc157b21c5424118bc3d36507b8d4a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.708015 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278420f8d2f4be8b973de3eb756f69a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:32.708799 3676 leader_election.cc:341] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:32.709666 4167 ts_tablet_manager.cc:1939] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:32.714035 4169 ts_tablet_manager.cc:1918] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:32.714391 4169 log.cc:1196] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:32.714912 4169 ts_tablet_manager.cc:1939] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:32.725315 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2460d58bb949e4a3a2c9f74dd800a8 (DEFAULT_TABLE table=default.Table_619d1e7c0e384107962b41e86e45c45b [id=1f4ab61624c84bde9f6689e2716a8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.734184 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86bfa65a8774b51bc7e7fe59442bf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.737161 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278420f8d2f4be8b973de3eb756f69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.737447 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 (DEFAULT_TABLE table=default.Table_aadee0d920fc4be5aa7f965bc9db93e4 [id=3886fb421ea4409585d7020e74f2221e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.718189 4163 tablet_bootstrap.cc:654] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.731600 5334 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.738627 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abd6e9a89814dbaa84316475bbe1ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.738447 5334 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.739486 5334 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.738057 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 (DEFAULT_TABLE table=default.Table_aadee0d920fc4be5aa7f965bc9db93e4 [id=3886fb421ea4409585d7020e74f2221e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.742831 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.743137 3745 raft_consensus.cc:2454] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.746374 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2460d58bb949e4a3a2c9f74dd800a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.746645 3824 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.746990 5334 raft_consensus.cc:2781] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.747095 5334 raft_consensus.cc:482] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.747185 5334 raft_consensus.cc:3037] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.750763 5334 raft_consensus.cc:504] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.751957 5334 leader_election.cc:288] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.754750 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.755738 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 736) took 5100 ms. Trace:
I20260811 18:57:32.755857 4167 rpcz_store.cc:276] 0811 18:57:27.655124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.655175 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.270874 (+4615699us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:32.271193 (+ 319us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.449810 (+178617us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.493485 (+ 43675us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.755604 (+262119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4613561}
I20260811 18:57:32.756691 4167 tablet_replica.cc:330] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:32.756985 4167 raft_consensus.cc:2229] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:32.757309 4167 raft_consensus.cc:2258] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.758891 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a902ad70a25546be926ff8f7bf2ed64f (table default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]) successfully deleted
I20260811 18:57:32.760986 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe203387bc9c45ba82afe6b058aa5e5f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.761263 3745 raft_consensus.cc:3037] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.764796 3745 raft_consensus.cc:2454] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:32.765379 3824 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.765771 5334 raft_consensus.cc:2781] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:32.765905 5334 raft_consensus.cc:686] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:32.766213 5334 consensus_queue.cc:237] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.777338 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ccf89d0b4e84205ad1d02c7c985a9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.781170 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b609177908d04a829b1fe744ef0f53e1 (DEFAULT_TABLE table=default.Table_03b64ac0b80f46288fcf2b34397efcaa [id=4845fddf6562470ebb7275b67fad076e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.781668 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b609177908d04a829b1fe744ef0f53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.786603 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 758) took 3863 ms. Trace:
I20260811 18:57:32.786787 4169 rpcz_store.cc:276] 0811 18:57:28.922855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.922916 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:32.611800 (+3688884us) ts_tablet_manager.cc:1145] Deleting tablet 0ba6246e6d29418cbca5f751b34f4ef1
0811 18:57:32.611812 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.670414 (+ 58602us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.713894 (+ 43480us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:32.786565 (+ 72671us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3688638}
I20260811 18:57:32.787499 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 759) took 3864 ms. Trace:
I20260811 18:57:32.787657 4169 rpcz_store.cc:276] 0811 18:57:28.923346 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.923410 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:32.787373 (+3863963us) ts_tablet_manager.cc:1145] Deleting tablet b4dc2df4c6464d71bb13fe25e0046758
0811 18:57:32.787386 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.787482 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3863752}
I20260811 18:57:32.788220 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 760) took 3864 ms. Trace:
I20260811 18:57:32.788369 4169 rpcz_store.cc:276] 0811 18:57:28.923777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.923822 (+ 45us) service_pool.cc:227] Handling call
0811 18:57:32.788093 (+3864271us) ts_tablet_manager.cc:1145] Deleting tablet 2474ae972e9242b69f488b7a2fdbc481
0811 18:57:32.788106 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.788200 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3864098}
I20260811 18:57:32.788412 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ba6246e6d29418cbca5f751b34f4ef1 (table default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]) successfully deleted
W20260811 18:57:32.788899 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b4dc2df4c6464d71bb13fe25e0046758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4dc2df4c6464d71bb13fe25e0046758 already in progress: creating tablet
I20260811 18:57:32.789086 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 757) took 3866 ms. Trace:
W20260811 18:57:32.789438 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2474ae972e9242b69f488b7a2fdbc481 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2474ae972e9242b69f488b7a2fdbc481 already in progress: creating tablet
I20260811 18:57:32.789796 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b609177908d04a829b1fe744ef0f53e1 (DEFAULT_TABLE table=default.Table_03b64ac0b80f46288fcf2b34397efcaa [id=4845fddf6562470ebb7275b67fad076e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.790706 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b609177908d04a829b1fe744ef0f53e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.795146 4169 rpcz_store.cc:276] 0811 18:57:28.922554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924383 (+ 1829us) service_pool.cc:227] Handling call
0811 18:57:32.788894 (+3864511us) ts_tablet_manager.cc:1145] Deleting tablet b203df4124364737839bf4c1ae072988
0811 18:57:32.788908 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.788999 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3864309}
W20260811 18:57:32.798942 3824 leader_election.cc:334] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.146s (SENT)
W20260811 18:57:32.801517 3824 consensus_peers.cc:487] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a902ad70a25546be926ff8f7bf2ed64f. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:32.801496 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b203df4124364737839bf4c1ae072988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b203df4124364737839bf4c1ae072988 already in progress: creating tablet
I20260811 18:57:32.803648 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 761) took 3875 ms. Trace:
I20260811 18:57:32.804241 4169 rpcz_store.cc:276] 0811 18:57:28.928622 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.928902 (+ 280us) service_pool.cc:227] Handling call
0811 18:57:32.803059 (+3874157us) ts_tablet_manager.cc:1145] Deleting tablet 8e120a45539f49bb82ab0fb95eac36f9
0811 18:57:32.803079 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.803619 (+ 540us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3873958}
I20260811 18:57:32.806171 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 0b07f6b410654740aed79e58fb281007 (DEFAULT_TABLE table=default.Table_9183f700c0544cd5942abebaaa8a121f [id=af4584f5e44548e3a40317d2301eef07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.806910 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b07f6b410654740aed79e58fb281007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.806170 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 0b07f6b410654740aed79e58fb281007 (DEFAULT_TABLE table=default.Table_9183f700c0544cd5942abebaaa8a121f [id=af4584f5e44548e3a40317d2301eef07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.807621 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b07f6b410654740aed79e58fb281007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.807782 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 762) took 3741 ms. Trace:
W20260811 18:57:32.807967 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8e120a45539f49bb82ab0fb95eac36f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e120a45539f49bb82ab0fb95eac36f9 already in progress: creating tablet
I20260811 18:57:32.809928 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 10e441b5df3d40fd93cf2dafe33bc2f4 (DEFAULT_TABLE table=default.Table_0db19ab70f764e00b17ee2e0f4742b8b [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.810467 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e441b5df3d40fd93cf2dafe33bc2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.821727 4169 rpcz_store.cc:276] 0811 18:57:29.066239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.066311 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:32.807608 (+3741297us) ts_tablet_manager.cc:1145] Deleting tablet c77882ff8d974057a61f037f1943f58f
0811 18:57:32.807626 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.807760 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3741024}
I20260811 18:57:32.826850 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 10e441b5df3d40fd93cf2dafe33bc2f4 (DEFAULT_TABLE table=default.Table_0db19ab70f764e00b17ee2e0f4742b8b [id=f4f8845932e942c7968f3f24c670559e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.830652 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10e441b5df3d40fd93cf2dafe33bc2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.837330 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 77ae7711d3ce4d769e8e7fa66d285329 (DEFAULT_TABLE table=default.Table_160e44b6b513432c89082037a071fbb4 [id=421aa2aad7ed4696b94c748b9b7f9d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.837894 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ae7711d3ce4d769e8e7fa66d285329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.838274 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 77ae7711d3ce4d769e8e7fa66d285329 (DEFAULT_TABLE table=default.Table_160e44b6b513432c89082037a071fbb4 [id=421aa2aad7ed4696b94c748b9b7f9d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.838773 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ae7711d3ce4d769e8e7fa66d285329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.839425 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 772) took 3179 ms. Trace:
I20260811 18:57:32.840962 4169 rpcz_store.cc:276] 0811 18:57:29.660448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.660646 (+ 198us) service_pool.cc:227] Handling call
0811 18:57:32.826238 (+3165592us) ts_tablet_manager.cc:1145] Deleting tablet 1d721b0e55384d8a8efd88a0e47b1504
0811 18:57:32.828227 (+ 1989us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.839406 (+ 11179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3165408}
W20260811 18:57:32.843047 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1d721b0e55384d8a8efd88a0e47b1504 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d721b0e55384d8a8efd88a0e47b1504 already in progress: creating tablet
I20260811 18:57:32.843391 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 773) took 3182 ms. Trace:
I20260811 18:57:32.845436 4169 rpcz_store.cc:276] 0811 18:57:29.661360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.661491 (+ 131us) service_pool.cc:227] Handling call
0811 18:57:32.843243 (+3181752us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:32.843259 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.843371 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3181553}
W20260811 18:57:32.846079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1e1374a80d15444cb0faff339275aa8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e1374a80d15444cb0faff339275aa8d already in progress: creating tablet
I20260811 18:57:32.847842 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 774) took 3185 ms. Trace:
I20260811 18:57:32.847951 4169 rpcz_store.cc:276] 0811 18:57:29.662481 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.662610 (+ 129us) service_pool.cc:227] Handling call
0811 18:57:32.847703 (+3185093us) ts_tablet_manager.cc:1145] Deleting tablet 47a782b8470b4d14ac89b3b7d463a125
0811 18:57:32.847716 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.847822 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3184902}
W20260811 18:57:32.848553 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 47a782b8470b4d14ac89b3b7d463a125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47a782b8470b4d14ac89b3b7d463a125 already in progress: creating tablet
I20260811 18:57:32.852102 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 775) took 3171 ms. Trace:
I20260811 18:57:32.852221 4169 rpcz_store.cc:276] 0811 18:57:29.680714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.680920 (+ 206us) service_pool.cc:227] Handling call
0811 18:57:32.851958 (+3171038us) ts_tablet_manager.cc:1145] Deleting tablet 1c74abe9dcf642e9b4d40df6ff3639ef
0811 18:57:32.851974 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.852081 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3170727}
W20260811 18:57:32.853380 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1c74abe9dcf642e9b4d40df6ff3639ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c74abe9dcf642e9b4d40df6ff3639ef already in progress: creating tablet
I20260811 18:57:32.853663 3852 tablet_service.cc:1467] Processing CreateTablet for tablet bcfad3da366e4554b06bfa51a2535419 (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.854128 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfad3da366e4554b06bfa51a2535419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.758320 4167 ts_tablet_manager.cc:1905] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:32.857671 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e7d75d8468fc4c938fde0b2951d84562
I20260811 18:57:32.858968 4163 tablet_bootstrap.cc:492] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.859122 4163 ts_tablet_manager.cc:1397] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.216s user 0.003s sys 0.007s
I20260811 18:57:32.859933 4163 raft_consensus.cc:348] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.859969 5327 log.cc:824] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Log is configured to *not* fsync() on all Append() calls
W20260811 18:57:32.860097 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:32.860124 4163 raft_consensus.cc:374] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.860174 4163 raft_consensus.cc:729] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.860337 4163 consensus_queue.cc:260] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.860561 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_706c784347b146089c032d699f6a9abe" } new_table_name: "default.Table_25c8439e034242f7bc3d1c546188fc8e" modify_external_catalogs: true
I20260811 18:57:32.860877 4163 ts_tablet_manager.cc:1428] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:32.861955 5376 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.862164 5376 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.862300 3824 leader_election.cc:334] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 4.384s (SENT)
I20260811 18:57:32.862721 4163 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:32.862865 5376 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.865048 4163 tablet_bootstrap.cc:654] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:32.865208 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" } modify_external_catalogs: true
I20260811 18:57:32.865482 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1662a43ab5334f8689e0d7e654519650" } modify_external_catalogs: true
I20260811 18:57:32.874814 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d75d8468fc4c938fde0b2951d84562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:32.875569 5091 raft_consensus.cc:482] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.880887 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 776) took 3199 ms. Trace:
I20260811 18:57:32.882511 4169 rpcz_store.cc:276] 0811 18:57:29.681979 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.687792 (+ 5813us) service_pool.cc:227] Handling call
0811 18:57:32.879943 (+3192151us) ts_tablet_manager.cc:1145] Deleting tablet 6880ad27a3eb4599b407edeb2969a08f
0811 18:57:32.880729 (+ 786us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.880862 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3191835}
I20260811 18:57:32.884363 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 (DEFAULT_TABLE table=default.Table_b79839172cff41c3a8235675d44c21fa [id=8033ac0465864ef3870d27a6cf52244d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.885010 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fb15b72c1d49d2ac4961bc04655a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.886902 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 785) took 2877 ms. Trace:
I20260811 18:57:32.888366 4169 rpcz_store.cc:276] 0811 18:57:30.009454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.009577 (+ 123us) service_pool.cc:227] Handling call
0811 18:57:32.886752 (+2877175us) ts_tablet_manager.cc:1145] Deleting tablet f3670f8908d54ebb82f698cdd207e48f
0811 18:57:32.886774 (+ 22us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:32.886881 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2859157}
I20260811 18:57:32.891156 4169 tablet_replica.cc:330] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:32.891551 4169 raft_consensus.cc:2229] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:32.875725 5091 raft_consensus.cc:504] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:32.892436 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c0b355a0d3984012be4e9406998fd4a0 consensus state has no leader
I20260811 18:57:32.892542 5091 leader_election.cc:288] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:32.892941 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb (DEFAULT_TABLE table=default.Table_6743ac70503f4bbc8811494daa73361f [id=6a0643492e8847e4883fa191f98d5f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.895733 3719 tablet_service.cc:1467] Processing CreateTablet for tablet a0a12cb16855433eaf447c3bca2f4bad (DEFAULT_TABLE table=default.Table_96bf47f7003740cba4c66f780597a121 [id=782aced1be1a402980d996e519b959df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896520 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 (DEFAULT_TABLE table=default.Table_b79839172cff41c3a8235675d44c21fa [id=8033ac0465864ef3870d27a6cf52244d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896664 3714 tablet_service.cc:1467] Processing CreateTablet for tablet bcfad3da366e4554b06bfa51a2535419 (DEFAULT_TABLE table=default.Table_15ce5cf32fd24c2c839bc9dfcffb3d2d [id=3db29cfff5ae473db76de82ea12fb159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896859 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 922c6fcbcb184e99bf396b6e69e84299 (DEFAULT_TABLE table=default.Table_bf4fa780a3ba4824b3317ae7e9a379fc [id=66d2227c128e4a41ab2754c4f788c0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.896914 3853 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb (DEFAULT_TABLE table=default.Table_6743ac70503f4bbc8811494daa73361f [id=6a0643492e8847e4883fa191f98d5f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897204 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2fb15b72c1d49d2ac4961bc04655a29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.897425 3855 tablet_service.cc:1467] Processing CreateTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897430 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 80b31661fc59476496371e71ed1a21e1 (DEFAULT_TABLE table=default.Table_5b76a052465c4749a059cbf9171ab6be [id=d82bca795c574e6896ed5e93201570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.897814 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25b1eddd4a64847b51ad3cdc0238ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.899363 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.900053 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfad3da366e4554b06bfa51a2535419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.901324 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.901569 3745 raft_consensus.cc:2454] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.902683 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 922c6fcbcb184e99bf396b6e69e84299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.902642 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 80b31661fc59476496371e71ed1a21e1 (DEFAULT_TABLE table=default.Table_5b76a052465c4749a059cbf9171ab6be [id=d82bca795c574e6896ed5e93201570e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.903000 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b31661fc59476496371e71ed1a21e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.903139 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b31661fc59476496371e71ed1a21e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.904839 3824 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.905279 5376 consensus_queue.cc:1035] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:32.905715 5376 raft_consensus.cc:2781] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.905795 5376 raft_consensus.cc:482] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.905859 5376 raft_consensus.cc:3037] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.906630 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.908591 5376 raft_consensus.cc:504] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.909152 5376 leader_election.cc:288] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.910944 3857 tablet_service.cc:1467] Processing CreateTablet for tablet a0a12cb16855433eaf447c3bca2f4bad (DEFAULT_TABLE table=default.Table_96bf47f7003740cba4c66f780597a121 [id=782aced1be1a402980d996e519b959df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.911355 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0a12cb16855433eaf447c3bca2f4bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.912214 4169 pending_rounds.cc:62] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:32.912299 4169 pending_rounds.cc:68] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400169131261952 op_type: NO_OP noop_request { }
I20260811 18:57:32.912483 4169 raft_consensus.cc:2866] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:32.912596 4169 raft_consensus.cc:2258] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:32.913323 4169 ts_tablet_manager.cc:1905] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:32.914399 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6880ad27a3eb4599b407edeb2969a08f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6880ad27a3eb4599b407edeb2969a08f already in progress: creating tablet
W20260811 18:57:32.914934 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f3670f8908d54ebb82f698cdd207e48f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3670f8908d54ebb82f698cdd207e48f already in progress: creating tablet
I20260811 18:57:32.912314 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 922c6fcbcb184e99bf396b6e69e84299 (DEFAULT_TABLE table=default.Table_bf4fa780a3ba4824b3317ae7e9a379fc [id=66d2227c128e4a41ab2754c4f788c0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.917536 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 922c6fcbcb184e99bf396b6e69e84299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.918533 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d75d8468fc4c938fde0b2951d84562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.918777 5283 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.918957 5283 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.919878 3709 tablet_service.cc:1467] Processing CreateTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.920406 5283 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.927886 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 81c02b97930249d18b1e6fd3fd695e4b (DEFAULT_TABLE table=default.Table_3418766144ec4b91bb802faa2ad8808e [id=d22eb4f13976484a86cdbe8f85f55fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.927985 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 81c02b97930249d18b1e6fd3fd695e4b (DEFAULT_TABLE table=default.Table_3418766144ec4b91bb802faa2ad8808e [id=d22eb4f13976484a86cdbe8f85f55fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.928716 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c02b97930249d18b1e6fd3fd695e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.929980 3871 tablet_service.cc:1467] Processing CreateTablet for tablet e06e3db1c605429895482ac91b4b8053 (DEFAULT_TABLE table=default.Table_0c05dd720298449e9aa85faeadb50b96 [id=cb3bbf69e7b14f54badc8a9f679b7550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:32.930773 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06e3db1c605429895482ac91b4b8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.933364 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25b1eddd4a64847b51ad3cdc0238ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.933673 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81c02b97930249d18b1e6fd3fd695e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:32.931367 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:32.946852 3744 raft_consensus.cc:2454] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:32.946827 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71df3738a5cf4b12820dda131ced62e1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.947090 3745 raft_consensus.cc:3037] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.947381 3824 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:32.947669 5283 raft_consensus.cc:2781] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:32.947747 5283 raft_consensus.cc:482] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:32.947814 5283 raft_consensus.cc:3037] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.951349 5283 raft_consensus.cc:504] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.952586 5283 leader_election.cc:288] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.975025 5283 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:32.975343 5283 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:32.976680 3857 tablet_service.cc:1514] Processing DeleteTablet for tablet c26851245cab4a2d80e94d4fe278fd50 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a64df7c295c4d9baded1b63a9980581) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:32.977248 5283 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:32.980270 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc5448a555944dfba8ccae09e25cbe9 with delete_type TABLET_DATA_DELETED (replaced by tablet d00d943b778741b4aeec076ba46a9c40) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:32.980904 3678 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.739s (SENT)
I20260811 18:57:32.981043 3678 leader_election.cc:302] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:32.981566 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122b7898fea44ef78f269b1505d8ddc4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:32.981887 3743 raft_consensus.cc:3037] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:32.986229 4163 tablet_bootstrap.cc:492] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:32.986485 4163 ts_tablet_manager.cc:1397] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.004s sys 0.002s
I20260811 18:57:32.987440 4163 raft_consensus.cc:348] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.987836 4163 raft_consensus.cc:374] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:32.987954 4163 raft_consensus.cc:729] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:32.988257 4163 consensus_queue.cc:260] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:32.988958 4163 ts_tablet_manager.cc:1428] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:32.993856 4163 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:32.981216 3678 raft_consensus.cc:2670] T ff1be026ca404eada53f60f257ea367c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:32.995747 3720 tablet_service.cc:1467] Processing CreateTablet for tablet e06e3db1c605429895482ac91b4b8053 (DEFAULT_TABLE table=default.Table_0c05dd720298449e9aa85faeadb50b96 [id=cb3bbf69e7b14f54badc8a9f679b7550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.003852 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0a12cb16855433eaf447c3bca2f4bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.009902 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06e3db1c605429895482ac91b4b8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.010624 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.010948 3744 raft_consensus.cc:2454] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.011898 3824 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.012667 5283 raft_consensus.cc:2781] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.012872 5283 raft_consensus.cc:482] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.013031 5283 raft_consensus.cc:3037] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.023237 3745 raft_consensus.cc:2454] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.025441 3824 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.025852 5334 raft_consensus.cc:2781] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.026007 5334 raft_consensus.cc:686] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.026259 5334 consensus_queue.cc:237] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.028584 3743 raft_consensus.cc:2454] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.029212 3824 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.029558 5376 raft_consensus.cc:2781] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.029701 5376 raft_consensus.cc:686] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.029933 5376 consensus_queue.cc:237] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.047895 3676 leader_election.cc:341] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Not found: Tablet not found: 4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:33.048596 5407 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.048877 5407 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.049849 5407 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.057433 4167 ts_tablet_manager.cc:1918] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.057614 4167 log.cc:1196] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d2d7d986900244ddbc05dbe73a708dcf
I20260811 18:57:33.058140 4167 ts_tablet_manager.cc:1939] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.058774 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 737) took 5403 ms. Trace:
I20260811 18:57:33.058915 4167 rpcz_store.cc:276] 0811 18:57:27.655613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.655664 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:32.756607 (+5100943us) ts_tablet_manager.cc:1145] Deleting tablet d2d7d986900244ddbc05dbe73a708dcf
0811 18:57:32.756620 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.024147 (+267527us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.052956 (+ 28809us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.058750 (+ 5794us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5100717}
I20260811 18:57:33.059803 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 745) took 5346 ms. Trace:
I20260811 18:57:33.059922 4167 rpcz_store.cc:276] 0811 18:57:27.713460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.713527 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:33.059617 (+5346090us) ts_tablet_manager.cc:1145] Deleting tablet 4059e5907b1e488ab9e1d04e563f96cd
0811 18:57:33.059633 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.059765 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5345696}
I20260811 18:57:33.060465 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 746) took 5079 ms. Trace:
I20260811 18:57:33.060570 4167 rpcz_store.cc:276] 0811 18:57:27.980794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:27.984328 (+ 3534us) service_pool.cc:227] Handling call
0811 18:57:33.060338 (+5076010us) ts_tablet_manager.cc:1145] Deleting tablet 4093564a88264ed59c945523c9ddc9b3
0811 18:57:33.060351 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.060448 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5073227}
I20260811 18:57:33.020853 5283 raft_consensus.cc:504] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.063416 5283 leader_election.cc:288] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:33.065805 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 100091us
I20260811 18:57:33.069216 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3ec38f9b55489ba00ed76f9e4456f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.075932 5418 tablet.cc:1722] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:33.081126 4167 tablet_replica.cc:330] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.081574 4167 raft_consensus.cc:2229] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.081908 4167 raft_consensus.cc:2258] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.082120 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d2d7d986900244ddbc05dbe73a708dcf (table default.Table_b591f65f304846aab2374c6ada0dd0ec [id=880fac186e1d412395e5e9778c8a19b8]) successfully deleted
W20260811 18:57:33.082793 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4059e5907b1e488ab9e1d04e563f96cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4059e5907b1e488ab9e1d04e563f96cd already in progress: creating tablet
I20260811 18:57:33.082923 4167 ts_tablet_manager.cc:1905] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.083241 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4093564a88264ed59c945523c9ddc9b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4093564a88264ed59c945523c9ddc9b3 already in progress: creating tablet
W20260811 18:57:33.083829 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73f05e1d5cb947fe9dbcc2e876a23788 consensus state has no leader
I20260811 18:57:33.080457 4162 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.084383 4162 ts_tablet_manager.cc:1397] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.789s user 0.028s sys 0.000s
I20260811 18:57:33.085253 4162 raft_consensus.cc:348] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.085537 4162 raft_consensus.cc:374] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.085613 4162 raft_consensus.cc:729] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.085862 4162 consensus_queue.cc:260] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.086598 4162 ts_tablet_manager.cc:1428] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.088552 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7cabaea271024c3daf3ffc96c5825f1e" } new_table_name: "default.Table_f5e5c61d6ae446fdbe466f320a17ed6c" modify_external_catalogs: true
W20260811 18:57:33.088749 3676 consensus_peers.cc:487] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 -> Peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Couldn't send request to peer 57e8f62fa63a4fee97f268f5b7d3ae24. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 515d5b5afbd74ec5ab90a503aef2f4fb. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.089713 4163 tablet_bootstrap.cc:654] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.090258 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1afc7507935345b2a081a64ed01106da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.091357 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edd393d9c4d24aab8a170e1329400656 consensus state has no leader
I20260811 18:57:33.092876 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" } modify_external_catalogs: true
I20260811 18:57:33.093182 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f06e0fcaff2b45be93186f68b2c62dfe" } modify_external_catalogs: true
I20260811 18:57:33.091682 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c515c153df4317b3fb427983ee15e0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:33.095176 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f868e00d47d34ad995459fb265d9157a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.096586 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1afc7507935345b2a081a64ed01106da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:33.097393 3676 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:33.098284 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f868e00d47d34ad995459fb265d9157a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.099164 4162 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.100378 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6080085ba7594b18b5594e8868f0b80e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.104146 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6080085ba7594b18b5594e8868f0b80e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.107323 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a4328549b8d4ad08d0135327263c811" } modify_external_catalogs: true
I20260811 18:57:33.107609 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8a4328549b8d4ad08d0135327263c811" } modify_external_catalogs: true
I20260811 18:57:33.112727 4162 tablet_bootstrap.cc:654] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.117805 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:33.118245 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 60688us
I20260811 18:57:33.119323 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e902efbb774244d18e3ba165bfb7c695" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.119573 3740 raft_consensus.cc:3037] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.122763 3740 raft_consensus.cc:2454] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.131449 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.132221 3824 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.132503 5376 raft_consensus.cc:2781] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.132741 5376 raft_consensus.cc:686] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.133118 5376 consensus_queue.cc:237] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.136451 4167 ts_tablet_manager.cc:1918] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.136636 4167 log.cc:1196] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/06b8f2a97c0f46ea8c56b3de2b97e60c
I20260811 18:57:33.137081 4167 ts_tablet_manager.cc:1939] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.137022 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2fc380eb763b488e815a72be20c8f563"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.138537 5422 tablet.cc:1722] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:33.144275 4162 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.144584 4162 ts_tablet_manager.cc:1397] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.000s
I20260811 18:57:33.145155 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6df5545532e346f7abcdc4cd228b84da
I20260811 18:57:33.145545 4162 raft_consensus.cc:348] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.145825 4162 raft_consensus.cc:374] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.145887 4162 raft_consensus.cc:729] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.146139 4162 consensus_queue.cc:260] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.146793 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df5545532e346f7abcdc4cd228b84da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.152844 4162 ts_tablet_manager.cc:1428] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
W20260811 18:57:33.159797 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fc380eb763b488e815a72be20c8f563 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.163645 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 747) took 5060 ms. Trace:
I20260811 18:57:33.163791 4167 rpcz_store.cc:276] 0811 18:57:28.103105 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.103163 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:33.081054 (+4977891us) ts_tablet_manager.cc:1145] Deleting tablet 06b8f2a97c0f46ea8c56b3de2b97e60c
0811 18:57:33.081071 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.114992 (+ 33921us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.136305 (+ 21313us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.163614 (+ 27309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4977572}
I20260811 18:57:33.164305 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 06b8f2a97c0f46ea8c56b3de2b97e60c (table default.Table_a1329ea40dbd4f748a0e35686a5e867f [id=8732a8cdc7fd4dd78223e3a49d51ec75]) successfully deleted
W20260811 18:57:33.185596 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae75d6652c704e1995966c6dd4c06307 consensus state has no leader
I20260811 18:57:33.188793 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f414c31b8cff4e889368e4207afb1b53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.189416 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f414c31b8cff4e889368e4207afb1b53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.193493 4163 tablet_bootstrap.cc:492] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.193818 4163 ts_tablet_manager.cc:1397] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.200s user 0.008s sys 0.000s
I20260811 18:57:33.194291 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 6df5545532e346f7abcdc4cd228b84da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.194849 4163 raft_consensus.cc:348] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.195068 4163 raft_consensus.cc:374] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.195113 4163 raft_consensus.cc:729] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.195314 4163 consensus_queue.cc:260] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.195996 4163 ts_tablet_manager.cc:1428] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.198144 5407 raft_consensus.cc:482] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.198303 5407 raft_consensus.cc:504] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:33.198511 3554 catalog_manager.cc:5495] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:33.199031 5407 leader_election.cc:288] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.199077 3554 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:33.200062 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:33.202936 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.207504 4162 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.209781 4162 tablet_bootstrap.cc:654] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.210938 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 748) took 4965 ms. Trace:
I20260811 18:57:33.211063 4167 rpcz_store.cc:276] 0811 18:57:28.245441 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.245540 (+ 99us) service_pool.cc:227] Handling call
0811 18:57:33.165932 (+4920392us) ts_tablet_manager.cc:1145] Deleting tablet ae7da0868b114052ba421801b76fb33c
0811 18:57:33.210760 (+ 44828us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.210911 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4920074}
I20260811 18:57:33.211560 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 749) took 4965 ms. Trace:
I20260811 18:57:33.211666 4167 rpcz_store.cc:276] 0811 18:57:28.245647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.245918 (+ 271us) service_pool.cc:227] Handling call
0811 18:57:33.211430 (+4965512us) ts_tablet_manager.cc:1145] Deleting tablet 6d988ef7e9dd43158d107998ea4d82b9
0811 18:57:33.211445 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.211541 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957400}
I20260811 18:57:33.212041 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 750) took 4957 ms. Trace:
I20260811 18:57:33.212126 4167 rpcz_store.cc:276] 0811 18:57:28.254289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.254369 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:33.211922 (+4957553us) ts_tablet_manager.cc:1145] Deleting tablet b6c70b4163634984bd02dd9b47ffb472
0811 18:57:33.211935 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.212023 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957350}
I20260811 18:57:33.212427 4167 tablet_replica.cc:330] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.212711 4167 raft_consensus.cc:2229] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.212947 4167 raft_consensus.cc:2258] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.213729 4167 ts_tablet_manager.cc:1905] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.203657 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6432abd4fc5c4f9ca95971ca4795e1cf" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.218897 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ae7da0868b114052ba421801b76fb33c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae7da0868b114052ba421801b76fb33c already in progress: creating tablet
W20260811 18:57:33.219095 3676 leader_election.cc:341] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.206453 5286 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:33.219255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6d988ef7e9dd43158d107998ea4d82b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d988ef7e9dd43158d107998ea4d82b9 already in progress: creating tablet
I20260811 18:57:33.219362 5286 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.219570 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b6c70b4163634984bd02dd9b47ffb472 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6c70b4163634984bd02dd9b47ffb472 already in progress: creating tablet
I20260811 18:57:33.220072 5286 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.220463 4163 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.220942 5407 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.221076 5407 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.221644 5407 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.222379 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7251204325dd424192a6f58e18e71c34" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.222929 3676 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:33.224912 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f3c4536fc7440db83851fc8d7b3de10 consensus state has no leader
I20260811 18:57:33.209756 4169 ts_tablet_manager.cc:1918] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.217684 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet b2718cd4a98742dfa8014a8a38c69c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.229417 4169 log.cc:1196] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/4687b95728fc485fa0e842bc447eb83f
I20260811 18:57:33.229295 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.229595 3740 raft_consensus.cc:2454] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.229930 4169 ts_tablet_manager.cc:1939] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.230106 3824 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.230548 5286 raft_consensus.cc:2781] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.230685 5286 raft_consensus.cc:482] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.230813 5286 raft_consensus.cc:3037] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.230778 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 790) took 3105 ms. Trace:
I20260811 18:57:33.230953 4169 rpcz_store.cc:276] 0811 18:57:30.125771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.273241 (+147470us) service_pool.cc:227] Handling call
0811 18:57:32.890747 (+2617506us) ts_tablet_manager.cc:1145] Deleting tablet 4687b95728fc485fa0e842bc447eb83f
0811 18:57:32.891103 (+ 356us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.071413 (+180310us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.209634 (+138221us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.230736 (+ 21102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2617289}
I20260811 18:57:33.231567 4169 tablet_replica.cc:330] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.232400 4169 raft_consensus.cc:2229] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.233039 4169 raft_consensus.cc:2258] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.233585 4167 ts_tablet_manager.cc:1918] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.233755 4167 log.cc:1196] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/867259e37fe44c33a35823ec2d896754
I20260811 18:57:33.233952 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 4687b95728fc485fa0e842bc447eb83f (table default.Table_1a8b47adc7454c64a44013e919b3a485 [id=ca087c91349d472b88a50e73c7f3cc28]) successfully deleted
I20260811 18:57:33.234247 4167 ts_tablet_manager.cc:1939] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.235468 5286 raft_consensus.cc:504] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.237732 4163 tablet_bootstrap.cc:654] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.238113 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:33.239550 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1e71d261fb704c1296dfc437453d6f38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.240296 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e71d261fb704c1296dfc437453d6f38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.249704 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4af28c051e8243eaa73d4a5ed1abcea6
I20260811 18:57:33.251550 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.252286 4162 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.252539 4162 ts_tablet_manager.cc:1397] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.000s
I20260811 18:57:33.253422 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 4af28c051e8243eaa73d4a5ed1abcea6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.253788 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:33.254719 4162 raft_consensus.cc:348] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.255143 4162 raft_consensus.cc:374] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.255280 4162 raft_consensus.cc:729] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.255676 4162 consensus_queue.cc:260] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.259362 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "221d0d8d69f142a29ec7065a82b3f769" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.259603 3740 raft_consensus.cc:3037] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.260582 4162 ts_tablet_manager.cc:1428] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.262764 3740 raft_consensus.cc:2454] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.263386 3824 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.263787 5376 raft_consensus.cc:2781] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.263916 5376 raft_consensus.cc:686] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.264145 5376 consensus_queue.cc:237] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.265545 3953 kernel_stack_watchdog.cc:198] Thread 3965 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 19962ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7fc7b6584980 (unknown) at ??:0
@ 0x8a61c4 __sanitizer::BlockingMutex::Lock() at ??:?
@ 0x8abfab __sanitizer::ThreadRegistry::CreateThread() at ??:?
@ 0x895faf __asan::AsanThread::Create() at ??:?
@ 0x81ec0c __interceptor_pthread_create at ??:?
@ 0x7fc7b329ebfb kudu::Thread::StartThread() at ??:0
@ 0x9da5bc kudu::Thread::Create() at ??:?
@ 0x7fc7b32befd8 kudu::ThreadPool::CreateThread() at ??:0
@ 0x7fc7b32babeb kudu::ThreadPool::DoSubmit() at ??:0
@ 0x7fc7b32b90af kudu::ThreadPoolToken::Submit() at ??:0
@ 0x7fc7be6b5ef5 kudu::consensus::Peer::SignalRequest() at ??:0
@ 0x7fc7be6be1a4 kudu::consensus::Peer::Init()::$_0::operator()() at ??:0
@ 0x7fc7be6bdeee std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc7b9d1a9bd kudu::rpc::PeriodicTimer::Callback() at ??:0
@ 0x7fc7b9d1bb7f kudu::rpc::PeriodicTimer::Callback()::$_0::operator()() at ??:0
I20260811 18:57:33.267428 5286 leader_election.cc:288] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.268824 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" } modify_external_catalogs: true
I20260811 18:57:33.269136 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d58fee3ba6e24378a66f7a00f2ab6fb2" } modify_external_catalogs: true
I20260811 18:57:33.280232 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 751) took 5025 ms. Trace:
I20260811 18:57:33.280429 4167 rpcz_store.cc:276] 0811 18:57:28.254737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.254793 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:33.212375 (+4957582us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:33.212387 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.225492 (+ 13105us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.233434 (+ 7942us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.280193 (+ 46759us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4957393}
I20260811 18:57:33.280936 4167 tablet_replica.cc:330] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.281199 4167 raft_consensus.cc:2229] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.281421 4167 raft_consensus.cc:2258] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.281955 4162 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.284346 4162 tablet_bootstrap.cc:654] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.280920 4163 tablet_bootstrap.cc:492] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.286844 4163 ts_tablet_manager.cc:1397] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.002s
I20260811 18:57:33.287741 4163 raft_consensus.cc:348] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.287973 4163 raft_consensus.cc:374] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.288040 4163 raft_consensus.cc:729] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.288411 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 867259e37fe44c33a35823ec2d896754 (table default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]) successfully deleted
I20260811 18:57:33.290200 4167 ts_tablet_manager.cc:1905] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.293762 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_972a064a812a441bb4068503de32341e" } new_table_name: "default.Table_227406eaa3694731a78479da6280faac" modify_external_catalogs: true
I20260811 18:57:33.301700 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.302657 4167 ts_tablet_manager.cc:1918] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.302838 4167 log.cc:1196] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/490f34d7a403416fbab64e46973746de
I20260811 18:57:33.303328 4167 ts_tablet_manager.cc:1939] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.305712 4163 consensus_queue.cc:260] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.307442 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 752) took 5052 ms. Trace:
I20260811 18:57:33.307607 4167 rpcz_store.cc:276] 0811 18:57:28.255116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255165 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:33.280878 (+5025713us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:33.280892 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.295090 (+ 14198us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.302500 (+ 7410us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.307407 (+ 4907us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5025551}
I20260811 18:57:33.308290 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 490f34d7a403416fbab64e46973746de (table default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]) successfully deleted
I20260811 18:57:33.308498 4163 ts_tablet_manager.cc:1428] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260811 18:57:33.308835 4163 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.310721 4162 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.310983 4162 ts_tablet_manager.cc:1397] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
I20260811 18:57:33.312465 4162 raft_consensus.cc:348] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.312938 4162 raft_consensus.cc:374] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.313073 4162 raft_consensus.cc:729] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.313416 4162 consensus_queue.cc:260] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.317925 4162 ts_tablet_manager.cc:1428] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.331637 4163 tablet_bootstrap.cc:654] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.334638 4167 tablet_replica.cc:330] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.334981 4167 raft_consensus.cc:2229] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.335202 4167 raft_consensus.cc:2258] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.335729 4162 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.341403 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.342314 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.343119 4167 ts_tablet_manager.cc:1905] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.346707 4162 tablet_bootstrap.cc:654] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.322491 4169 ts_tablet_manager.cc:1905] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.354113 4167 ts_tablet_manager.cc:1918] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.358754 4167 log.cc:1196] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:33.359946 4167 ts_tablet_manager.cc:1939] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.362851 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 753) took 5107 ms. Trace:
I20260811 18:57:33.363807 4167 rpcz_store.cc:276] 0811 18:57:28.255452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255499 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:33.309785 (+5054286us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:33.334543 (+ 24758us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.349755 (+ 15212us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.353968 (+ 4213us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.362820 (+ 8852us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5054114}
I20260811 18:57:33.365088 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 02808605ee6c4634af47fc3fd57e9cd8 (table default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]) successfully deleted
I20260811 18:57:33.365849 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 754) took 5110 ms. Trace:
I20260811 18:57:33.366827 4167 rpcz_store.cc:276] 0811 18:57:28.255798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.255846 (+ 48us) service_pool.cc:227] Handling call
0811 18:57:33.365670 (+5109824us) ts_tablet_manager.cc:1145] Deleting tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca
0811 18:57:33.365690 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.365825 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5109644}
W20260811 18:57:33.368121 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac8e1d9dc7b4d869f4bfe4d43aae1ca already in progress: creating tablet
I20260811 18:57:33.368707 4167 tablet_replica.cc:330] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.371001 4167 raft_consensus.cc:2229] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.371748 4167 raft_consensus.cc:2258] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.373011 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9608d6ddcad749cd9e72dd6f265db0da
I20260811 18:57:33.373049 4167 ts_tablet_manager.cc:1905] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.374698 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.375136 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 9608d6ddcad749cd9e72dd6f265db0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:33.375811 4162 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.376042 4162 ts_tablet_manager.cc:1397] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.010s
I20260811 18:57:33.377257 4162 raft_consensus.cc:348] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.377511 5376 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.377655 4162 raft_consensus.cc:374] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.377738 4162 raft_consensus.cc:729] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.377677 5376 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.378024 4162 consensus_queue.cc:260] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.378754 5376 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.378852 4162 ts_tablet_manager.cc:1428] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:33.379453 4162 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.379654 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.379974 3740 raft_consensus.cc:2454] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.380805 3824 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.381129 5376 raft_consensus.cc:2781] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.381219 5376 raft_consensus.cc:482] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.381289 5376 raft_consensus.cc:3037] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.382191 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aa50e46e061b41f79d5d6d048ea15132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.383122 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa50e46e061b41f79d5d6d048ea15132 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.386998 5376 raft_consensus.cc:504] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.388186 5376 leader_election.cc:288] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.388525 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26851245cab4a2d80e94d4fe278fd50" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.388760 3740 raft_consensus.cc:3037] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.390146 4169 ts_tablet_manager.cc:1918] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.390321 4169 log.cc:1196] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/867259e37fe44c33a35823ec2d896754
I20260811 18:57:33.390899 4169 ts_tablet_manager.cc:1939] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.391513 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 791) took 3265 ms. Trace:
I20260811 18:57:33.391647 4169 rpcz_store.cc:276] 0811 18:57:30.125903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.273472 (+147569us) service_pool.cc:227] Handling call
0811 18:57:33.231470 (+2957998us) ts_tablet_manager.cc:1145] Deleting tablet 867259e37fe44c33a35823ec2d896754
0811 18:57:33.231486 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.355666 (+124180us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.390009 (+ 34343us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.391469 (+ 1460us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2957820}
I20260811 18:57:33.392702 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 867259e37fe44c33a35823ec2d896754 (table default.Table_ac4a8df2cd7346ffa89549c0750760ea [id=b606048699b4406db064683cbb7e46fa]) successfully deleted
#10 0x7fc7c6b82128 in std::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7c6b7dcc5 in kudu::tools::Action::Run(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action.cc:245:10
I20260811 18:57:33.410027 4162 tablet_bootstrap.cc:654] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.418668 4169 tablet_replica.cc:330] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.418989 4169 raft_consensus.cc:2229] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.419404 4169 pending_rounds.cc:62] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:33.419521 4169 pending_rounds.cc:68] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400169550319616 op_type: NO_OP noop_request { }
I20260811 18:57:33.419749 4169 raft_consensus.cc:2866] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.420368 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b0759818f6cc4cd8b2747ae532c19ccc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.421231 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0759818f6cc4cd8b2747ae532c19ccc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.423039 3740 raft_consensus.cc:2454] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.423664 3824 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.425930 5376 raft_consensus.cc:2781] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.426719 5376 raft_consensus.cc:686] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.427057 5376 consensus_queue.cc:237] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.431197 4167 ts_tablet_manager.cc:1918] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.431386 4167 log.cc:1196] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4b6ec419ed7747c3b24dc11992518e88
I20260811 18:57:33.431780 4167 ts_tablet_manager.cc:1939] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.439646 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_be8ef75ca590418486730a869708b637"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.440487 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be8ef75ca590418486730a869708b637 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.443853 4169 raft_consensus.cc:2258] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.445369 4162 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.445608 4162 ts_tablet_manager.cc:1397] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.066s user 0.004s sys 0.009s
I20260811 18:57:33.446763 4162 raft_consensus.cc:348] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.447132 4162 raft_consensus.cc:374] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.445016 4169 ts_tablet_manager.cc:1905] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.447271 4162 raft_consensus.cc:729] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.447602 4162 consensus_queue.cc:260] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.448450 4162 ts_tablet_manager.cc:1428] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:33.449563 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 755) took 5192 ms. Trace:
I20260811 18:57:33.449759 4167 rpcz_store.cc:276] 0811 18:57:28.256963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.257015 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:33.368645 (+5111630us) ts_tablet_manager.cc:1145] Deleting tablet 4b6ec419ed7747c3b24dc11992518e88
0811 18:57:33.368660 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.386257 (+ 17597us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.430305 (+ 44048us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.449536 (+ 19231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5111403}
I20260811 18:57:33.450183 4167 tablet_replica.cc:330] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.450482 4167 raft_consensus.cc:2229] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.450661 4167 raft_consensus.cc:2258] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.451303 4167 ts_tablet_manager.cc:1905] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.458204 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4b6ec419ed7747c3b24dc11992518e88 (table default.Table_b4ff060dff9a4bf19670e304147315ac [id=b3b3a6f2569d4489ae5bfb3a5086c88b]) successfully deleted
I20260811 18:57:33.459093 4167 ts_tablet_manager.cc:1918] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.459239 4167 log.cc:1196] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0b3ec38f9b55489ba00ed76f9e4456f5
I20260811 18:57:33.459543 4167 ts_tablet_manager.cc:1939] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.460115 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 761) took 5001 ms. Trace:
I20260811 18:57:33.460253 4167 rpcz_store.cc:276] 0811 18:57:28.459059 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.459178 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:33.450124 (+4990946us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:33.450139 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.455868 (+ 5729us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.458989 (+ 3121us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.460092 (+ 1103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4990642}
I20260811 18:57:33.460793 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
I20260811 18:57:33.460915 4162 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.462208 4163 tablet_bootstrap.cc:492] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.462498 4163 ts_tablet_manager.cc:1397] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.154s user 0.010s sys 0.001s
I20260811 18:57:33.462764 4162 tablet_bootstrap.cc:654] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.463587 4163 raft_consensus.cc:348] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.463963 4163 raft_consensus.cc:374] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.464242 4163 raft_consensus.cc:729] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.465067 4163 consensus_queue.cc:260] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.466909 4163 ts_tablet_manager.cc:1428] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:33.470249 4163 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.463171 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 762) took 5003 ms. Trace:
I20260811 18:57:33.479283 4167 rpcz_store.cc:276] 0811 18:57:28.459254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.459503 (+ 249us) service_pool.cc:227] Handling call
0811 18:57:33.462997 (+5003494us) ts_tablet_manager.cc:1145] Deleting tablet d8146f47d9ae48889ac7efc954d4a6b7
0811 18:57:33.463012 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.463146 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5002805}
W20260811 18:57:33.480631 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d8146f47d9ae48889ac7efc954d4a6b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8146f47d9ae48889ac7efc954d4a6b7 already in progress: creating tablet
I20260811 18:57:33.480929 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 764) took 5007 ms. Trace:
I20260811 18:57:33.481062 4167 rpcz_store.cc:276] 0811 18:57:28.473894 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.474478 (+ 584us) service_pool.cc:227] Handling call
0811 18:57:33.480730 (+5006252us) ts_tablet_manager.cc:1145] Deleting tablet a38c172a9fef4a3ab8ee188335a7c97e
0811 18:57:33.480751 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.480903 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4999848}
W20260811 18:57:33.481734 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a38c172a9fef4a3ab8ee188335a7c97e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a38c172a9fef4a3ab8ee188335a7c97e already in progress: creating tablet
I20260811 18:57:33.487018 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 765) took 5002 ms. Trace:
I20260811 18:57:33.492159 4163 tablet_bootstrap.cc:654] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.494534 4167 rpcz_store.cc:276] 0811 18:57:28.484583 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.485058 (+ 475us) service_pool.cc:227] Handling call
0811 18:57:33.486807 (+5001749us) ts_tablet_manager.cc:1145] Deleting tablet 2c3ac1bc93404622a9b6a74aea8b7c1c
0811 18:57:33.486830 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.486983 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5001095}
W20260811 18:57:33.495693 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2c3ac1bc93404622a9b6a74aea8b7c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c3ac1bc93404622a9b6a74aea8b7c1c already in progress: creating tablet
I20260811 18:57:33.495927 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 766) took 4995 ms. Trace:
I20260811 18:57:33.497040 4167 rpcz_store.cc:276] 0811 18:57:28.500080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.500954 (+ 874us) service_pool.cc:227] Handling call
0811 18:57:33.495750 (+4994796us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:33.495770 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.495904 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4994199}
W20260811 18:57:33.499393 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cf6c8d542ae2489ba466ffadd518db12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf6c8d542ae2489ba466ffadd518db12 already in progress: creating tablet
I20260811 18:57:33.499502 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 769) took 4877 ms. Trace:
I20260811 18:57:33.499609 4167 rpcz_store.cc:276] 0811 18:57:28.622502 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.622583 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:33.499342 (+4876759us) ts_tablet_manager.cc:1145] Deleting tablet f13a14bd64704798bf6de4fc05201a31
0811 18:57:33.499358 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.499482 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4876511}
W20260811 18:57:33.500232 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f13a14bd64704798bf6de4fc05201a31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13a14bd64704798bf6de4fc05201a31 already in progress: creating tablet
I20260811 18:57:33.498728 4162 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.500466 4162 ts_tablet_manager.cc:1397] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260811 18:57:33.501868 4162 raft_consensus.cc:348] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.506153 4162 raft_consensus.cc:374] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.510542 4162 raft_consensus.cc:729] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.514739 4162 consensus_queue.cc:260] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.519073 4162 ts_tablet_manager.cc:1428] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260811 18:57:33.519778 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c3c124426696404c9e9cb5200f63055e" } new_table_name: "default.Table_c12987db235f4f4fb949580479275ec2" modify_external_catalogs: true
I20260811 18:57:33.520273 4162 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.502707 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 779) took 4782 ms. Trace:
I20260811 18:57:33.520532 4167 rpcz_store.cc:276] 0811 18:57:28.720461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.720825 (+ 364us) service_pool.cc:227] Handling call
0811 18:57:33.502496 (+4781671us) ts_tablet_manager.cc:1145] Deleting tablet 44e0a05b87bc46bfbfb2a8b55f9ed724
0811 18:57:33.502513 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.502681 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4781477}
I20260811 18:57:33.521147 4167 tablet_replica.cc:330] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.521436 4167 raft_consensus.cc:2229] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:33.521539 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 44e0a05b87bc46bfbfb2a8b55f9ed724 already in progress: creating tablet
I20260811 18:57:33.521835 4167 raft_consensus.cc:2258] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.522702 4167 ts_tablet_manager.cc:1905] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.524720 3678 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.417s (SENT)
I20260811 18:57:33.524858 3678 leader_election.cc:302] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:33.525027 3678 raft_consensus.cc:2670] T 4687b95728fc485fa0e842bc447eb83f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
#12 0xb1ad42 in kudu::tools::DispatchCommand(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::deque<std::string, std::allocator<std::string> > const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:138:15
#13 0xb1c3b9 in kudu::tools::RunTool(int, char**, bool) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:210:16
#14 0xb1d0c2 in main /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:305:10
#15 0x7fc7af40fc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
Thread T137 (tablet-delete [) created by T59 (rpc worker-4011) here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b32befd7 in kudu::ThreadPool::CreateThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:848:10
#4 0x7fc7b32babea in kudu::ThreadPool::DoSubmit(std::function<void ()>, kudu::ThreadPoolToken*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:643:21
I20260811 18:57:33.550895 5376 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.551236 5376 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.552475 5376 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:33.553284 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab03996c5c624e5eb17267b124e27f45 consensus state has no leader
I20260811 18:57:33.554071 4162 tablet_bootstrap.cc:654] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.554224 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet f7d4c26da86b49b8a37f2289f4ef6614 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.555148 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.555449 3740 raft_consensus.cc:2454] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.556123 3824 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.556692 5286 raft_consensus.cc:2781] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.556811 5286 raft_consensus.cc:482] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.556888 5286 raft_consensus.cc:3037] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.559114 4162 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.559270 4162 ts_tablet_manager.cc:1397] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.000s
I20260811 18:57:33.560313 4162 raft_consensus.cc:348] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.560591 4162 raft_consensus.cc:374] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.560663 4162 raft_consensus.cc:729] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.560812 5286 raft_consensus.cc:504] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.561564 5286 leader_election.cc:288] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.563465 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3060822021431584a494e6941be756" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.563874 3743 raft_consensus.cc:3037] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.565260 4163 tablet_bootstrap.cc:492] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.565423 4163 ts_tablet_manager.cc:1397] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.095s user 0.000s sys 0.005s
I20260811 18:57:33.566329 4163 raft_consensus.cc:348] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.566587 4163 raft_consensus.cc:374] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.566685 4163 raft_consensus.cc:729] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.567001 4163 consensus_queue.cc:260] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.567689 5376 consensus_queue.cc:1035] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.568746 3743 raft_consensus.cc:2454] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.571332 3824 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.571658 5286 raft_consensus.cc:2781] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.571779 5286 raft_consensus.cc:686] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.572008 5286 consensus_queue.cc:237] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.577669 4162 consensus_queue.cc:260] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.579190 4162 ts_tablet_manager.cc:1428] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20260811 18:57:33.498208 4169 ts_tablet_manager.cc:1918] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.581185 4169 log.cc:1196] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b0b199f4c9e14b468cb26cbd61fc8586
I20260811 18:57:33.581928 4162 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.582118 4169 ts_tablet_manager.cc:1939] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.582906 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 797) took 3438 ms. Trace:
I20260811 18:57:33.581071 4167 ts_tablet_manager.cc:1918] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.587164 4167 log.cc:1196] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/515d5b5afbd74ec5ab90a503aef2f4fb
I20260811 18:57:33.588024 4167 ts_tablet_manager.cc:1939] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.589118 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 780) took 4783 ms. Trace:
I20260811 18:57:33.594558 4167 rpcz_store.cc:276] 0811 18:57:28.805314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.805387 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:33.521084 (+4715697us) ts_tablet_manager.cc:1145] Deleting tablet 515d5b5afbd74ec5ab90a503aef2f4fb
0811 18:57:33.521100 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.547667 (+ 26567us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.580929 (+ 33262us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.589096 (+ 8167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4715507}
#5 0x7fc7b32bfcee in kudu::ThreadPool::Submit(std::function<void ()>) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:525:10
#6 0x7fc7c23e5265 in kudu::tserver::TSTabletManager::DeleteTabletAsync(std::string const&, kudu::tablet::TabletDataState, std::optional<long> const&, std::function<void (kudu::Status const&, kudu::tserver::TabletServerErrorPB_Code)> const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:1121:35
I20260811 18:57:33.599836 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 515d5b5afbd74ec5ab90a503aef2f4fb (table default.Table_be1e5fa6b9304fd6bcb4304038be24f3 [id=bca5a986e14d4f61ab4d178f870ba087]) successfully deleted
I20260811 18:57:33.600036 4167 tablet_replica.cc:330] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.600346 4167 raft_consensus.cc:2229] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.601272 4167 raft_consensus.cc:2258] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.596465 4162 tablet_bootstrap.cc:654] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
#7 0x7fc7c23337bb in kudu::tserver::TabletServiceAdminImpl::DeleteTablet(kudu::tserver::DeleteTabletRequestPB const*, kudu::tserver::DeleteTabletResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1532:30
I20260811 18:57:33.609658 4167 ts_tablet_manager.cc:1905] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
#8 0x7fc7bc95b926 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_3::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:215:13
#9 0x7fc7bc95b655 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_3>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
#10 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
#12 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#13 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#14 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Thread T59 (rpc worker-4011) created by T0 here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b9db98c6 in kudu::rpc::ServicePool::Init(int) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#4 0x7fc7bc1faa67 in kudu::RpcServer::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/rpc_server.cc:238:3
#5 0x7fc7bc21b0b7 in kudu::server::ServerBase::RegisterService(std::unique_ptr<kudu::rpc::ServiceIf, std::default_delete<kudu::rpc::ServiceIf> >) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/server/server_base.cc:1171:23
#6 0x7fc7c23171a3 in kudu::tserver::TabletServer::Start() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server.cc:157:3
#7 0x7fc7c2326380 in kudu::tserver::RunTabletServer() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_server_runner.cc:86:3
#8 0xae894d in kudu::tools::(anonymous namespace)::TServerRun(kudu::tools::RunnerContext const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action_tserver.cc:114:10
#9 0x921350 in std::_Function_handler<kudu::Status (kudu::tools::RunnerContext const&), kudu::Status (*)(kudu::tools::RunnerContext const&)>::_M_invoke(std::_Any_data const&, kudu::tools::RunnerContext const&) ../../../include/c++/8/bits/std_function.h:282:9
#10 0x7fc7c6b82128 in std::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const ../../../include/c++/8/bits/std_function.h:687:14
#11 0x7fc7c6b7dcc5 in kudu::tools::Action::Run(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, std::vector<std::string, std::allocator<std::string> > const&) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_action.cc:245:10
#12 0xb1ad42 in kudu::tools::DispatchCommand(std::vector<kudu::tools::Mode*, std::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::deque<std::string, std::allocator<std::string> > const&) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:138:15
#13 0xb1c3b9 in kudu::tools::RunTool(int, char**, bool) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:210:16
#14 0xb1d0c2 in main /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tools/tool_main.cc:305:10
#15 0x7fc7af40fc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
Thread T135 (tablet-open [wo) created by T59 (rpc worker-4011) here:
#0 0x81eb40 in pthread_create /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors.cpp:214:3
#1 0x7fc7b329ebfa in kudu::Thread::StartThread(std::string, std::string, std::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:638:15
I20260811 18:57:33.624686 4162 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
#2 0x9da5bb in kudu::Thread::Create(std::string, std::string, std::function<void ()>, scoped_refptr<kudu::Thread>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.h:146:12
#3 0x7fc7b32befd7 in kudu::ThreadPool::CreateThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:848:10
#4 0x7fc7b32babea in kudu::ThreadPool::DoSubmit(std::function<void ()>, kudu::ThreadPoolToken*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:643:21
I20260811 18:57:33.625213 4162 ts_tablet_manager.cc:1397] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.002s
#5 0x7fc7b32bfcee in kudu::ThreadPool::Submit(std::function<void ()>) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:525:10
#6 0x7fc7c23d9e06 in kudu::tserver::TSTabletManager::CreateNewTablet(std::string const&, std::string const&, kudu::Partition const&, std::string const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::optional<kudu::TableExtraConfigPB>, std::optional<std::string>, std::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/ts_tablet_manager.cc:703:3
#7 0x7fc7c233140b in kudu::tserver::TabletServiceAdminImpl::CreateTablet(kudu::tserver::CreateTabletRequestPB const*, kudu::tserver::CreateTabletResponsePB*, kudu::rpc::RpcContext*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:1474:34
#8 0x7fc7bc95aac6 in kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data0/somelongdirectorytoavoidrpathissues/src/kudu/build/asan/src/kudu/tserver/tserver_admin.service.cc:189:13
#9 0x7fc7bc95a7f5 in std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) ../../../include/c++/8/bits/std_function.h:297:2
I20260811 18:57:33.626631 4162 raft_consensus.cc:348] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
#10 0x7fc7b9db79be in std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const ../../../include/c++/8/bits/std_function.h:687:14
I20260811 18:57:33.584052 5334 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.627296 5334 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.628051 5334 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:33.588132 3824 consensus_peers.cc:487] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b3ec38f9b55489ba00ed76f9e4456f5. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.627202 4169 rpcz_store.cc:276] 0811 18:57:30.144329 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.330528 (+186199us) service_pool.cc:227] Handling call
0811 18:57:33.416641 (+3086113us) ts_tablet_manager.cc:1145] Deleting tablet b0b199f4c9e14b468cb26cbd61fc8586
0811 18:57:33.416659 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.481235 (+ 64576us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.498045 (+ 16810us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.582878 (+ 84833us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3018892}
I20260811 18:57:33.632076 5334 consensus_queue.cc:1035] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.632826 4169 tablet_replica.cc:330] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.632772 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b0b199f4c9e14b468cb26cbd61fc8586 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:33.633165 4169 raft_consensus.cc:2229] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.633476 4169 raft_consensus.cc:2258] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.606462 4163 ts_tablet_manager.cc:1428] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.041s user 0.002s sys 0.001s
I20260811 18:57:33.634573 4163 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.640077 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" } modify_external_catalogs: true
I20260811 18:57:33.640378 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2c3d24c7dc234432b8b0fb28b1ed0400" } modify_external_catalogs: true
I20260811 18:57:33.640599 4162 raft_consensus.cc:374] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.640730 4162 raft_consensus.cc:729] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.630622 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.641141 3745 raft_consensus.cc:2454] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.641364 4162 consensus_queue.cc:260] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.641705 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9409f5b4b0034412aedc183b302556a3" } new_table_name: "default.Table_7b329f3d8a5543659bfb5de93453f46b" modify_external_catalogs: true
W20260811 18:57:33.643556 3678 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.582s (SENT)
I20260811 18:57:33.643925 3824 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.644281 5376 raft_consensus.cc:2781] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.644376 5376 raft_consensus.cc:482] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.644474 5376 raft_consensus.cc:3037] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.644950 4162 ts_tablet_manager.cc:1428] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:33.645263 4162 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.645404 4169 ts_tablet_manager.cc:1905] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.647699 5376 raft_consensus.cc:504] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.656306 5376 leader_election.cc:288] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:33.657438 3824 leader_election.cc:334] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.511s (SENT)
I20260811 18:57:33.657620 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7065b527f8104608b1efc6787f0b2b77" } new_table_name: "default.Table_ed17e76c67444161b978d62bf0bef64f" modify_external_catalogs: true
I20260811 18:57:33.659535 4163 tablet_bootstrap.cc:654] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.659780 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f" } modify_external_catalogs: true
I20260811 18:57:33.660073 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ea1ee62f594e4ee798829d582bc2b53f" } modify_external_catalogs: true
I20260811 18:57:33.662159 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2718cd4a98742dfa8014a8a38c69c45" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.663923 3745 raft_consensus.cc:3037] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.667488 3745 raft_consensus.cc:2454] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.667968 4167 ts_tablet_manager.cc:1918] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.668862 3824 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.669116 4167 log.cc:1196] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ba6246e6d29418cbca5f751b34f4ef1
I20260811 18:57:33.669243 5376 raft_consensus.cc:2781] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.669528 5376 raft_consensus.cc:686] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.669750 4167 ts_tablet_manager.cc:1939] T 0ba6246e6d29418cbca5f751b34f4ef1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.669926 5376 consensus_queue.cc:237] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.670799 5334 consensus_queue.cc:1035] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.674865 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 784) took 4754 ms. Trace:
I20260811 18:57:33.677819 5443 raft_consensus.cc:482] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.678632 5443 raft_consensus.cc:504] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.679821 5443 leader_election.cc:288] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.680074 4167 rpcz_store.cc:276] 0811 18:57:28.920764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.922506 (+ 1742us) service_pool.cc:227] Handling call
0811 18:57:33.599959 (+4677453us) ts_tablet_manager.cc:1145] Deleting tablet 0ba6246e6d29418cbca5f751b34f4ef1
0811 18:57:33.599979 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.630331 (+ 30352us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.667831 (+ 37500us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.674842 (+ 7011us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4660485}
I20260811 18:57:33.680220 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45fde25bcb40493aafb083e161aa6932" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.681089 3676 leader_election.cc:341] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.681097 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ba6246e6d29418cbca5f751b34f4ef1 (table default.Table_c6dd21c795924a47a8a2435b5de1afb7 [id=19fe99295ac44c6bb812f19708e15b72]) successfully deleted
I20260811 18:57:33.681694 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 786) took 4757 ms. Trace:
I20260811 18:57:33.683720 4169 ts_tablet_manager.cc:1918] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.685214 4167 rpcz_store.cc:276] 0811 18:57:28.924040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924102 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:33.681522 (+4757420us) ts_tablet_manager.cc:1145] Deleting tablet 2474ae972e9242b69f488b7a2fdbc481
0811 18:57:33.681540 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.681670 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4741920}
W20260811 18:57:33.686237 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2474ae972e9242b69f488b7a2fdbc481 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2474ae972e9242b69f488b7a2fdbc481 already in progress: creating tablet
I20260811 18:57:33.686389 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 785) took 4765 ms. Trace:
I20260811 18:57:33.686491 4167 rpcz_store.cc:276] 0811 18:57:28.921008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924270 (+ 3262us) service_pool.cc:227] Handling call
0811 18:57:33.686223 (+4761953us) ts_tablet_manager.cc:1145] Deleting tablet b4dc2df4c6464d71bb13fe25e0046758
0811 18:57:33.686239 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.686368 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4746373}
I20260811 18:57:33.687228 4169 log.cc:1196] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/02808605ee6c4634af47fc3fd57e9cd8
I20260811 18:57:33.688666 4169 ts_tablet_manager.cc:1939] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.688844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 787) took 4763 ms. Trace:
I20260811 18:57:33.689357 4167 rpcz_store.cc:276] 0811 18:57:28.925472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.925537 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:33.688690 (+4763153us) ts_tablet_manager.cc:1145] Deleting tablet 8e120a45539f49bb82ab0fb95eac36f9
0811 18:57:33.688704 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.688820 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4748649}
I20260811 18:57:33.690233 4167 tablet_replica.cc:330] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.692018 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 800) took 3187 ms. Trace:
W20260811 18:57:33.692826 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b4dc2df4c6464d71bb13fe25e0046758 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4dc2df4c6464d71bb13fe25e0046758 already in progress: creating tablet
W20260811 18:57:33.693235 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8e120a45539f49bb82ab0fb95eac36f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e120a45539f49bb82ab0fb95eac36f9 already in progress: creating tablet
I20260811 18:57:33.696329 4169 rpcz_store.cc:276] 0811 18:57:30.504847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.504903 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:33.632760 (+3127857us) ts_tablet_manager.cc:1145] Deleting tablet 02808605ee6c4634af47fc3fd57e9cd8
0811 18:57:33.632778 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.651359 (+ 18581us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.683601 (+ 32242us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.691989 (+ 8388us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3080401}
I20260811 18:57:33.698413 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 02808605ee6c4634af47fc3fd57e9cd8 (table default.Table_d3392d9de7234297a490a976dfe10060 [id=2a146449ee0045d4aa487625a1afebc8]) successfully deleted
I20260811 18:57:33.698290 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 799) took 3193 ms. Trace:
I20260811 18:57:33.700527 4169 rpcz_store.cc:276] 0811 18:57:30.504654 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.504728 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:33.698141 (+3193413us) ts_tablet_manager.cc:1145] Deleting tablet 85140a3800f34a23b716703fd4e73d17
0811 18:57:33.698156 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.698271 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34,"tablet-delete.queue_time_us":3145774}
I20260811 18:57:33.700984 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 798) took 3196 ms. Trace:
I20260811 18:57:33.701064 4169 rpcz_store.cc:276] 0811 18:57:30.504451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.505484 (+ 1033us) service_pool.cc:227] Handling call
0811 18:57:33.700853 (+3195369us) ts_tablet_manager.cc:1145] Deleting tablet 98ad733726594d2a8087f857f155dbe2
0811 18:57:33.700865 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.700967 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3144586}
W20260811 18:57:33.701231 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 85140a3800f34a23b716703fd4e73d17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85140a3800f34a23b716703fd4e73d17 already in progress: creating tablet
I20260811 18:57:33.701303 4169 tablet_replica.cc:330] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.701483 4169 raft_consensus.cc:2229] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.701838 4169 raft_consensus.cc:2258] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:33.702039 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 98ad733726594d2a8087f857f155dbe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98ad733726594d2a8087f857f155dbe2 already in progress: creating tablet
I20260811 18:57:33.703768 4167 raft_consensus.cc:2229] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.704353 4167 pending_rounds.cc:62] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Trying to abort 1 pending ops.
I20260811 18:57:33.704480 4167 pending_rounds.cc:68] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400177733836800 op_type: NO_OP noop_request { }
I20260811 18:57:33.704732 4167 raft_consensus.cc:2866] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.704922 4167 raft_consensus.cc:2258] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.706498 4167 ts_tablet_manager.cc:1905] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.710299 3824 consensus_peers.cc:487] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:33.710600 4169 ts_tablet_manager.cc:1905] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.715847 4163 tablet_bootstrap.cc:492] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.716164 4163 ts_tablet_manager.cc:1397] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.000s
I20260811 18:57:33.716956 4163 raft_consensus.cc:348] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.717583 4163 raft_consensus.cc:374] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.717724 4163 raft_consensus.cc:729] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.718031 4163 consensus_queue.cc:260] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.722553 4163 ts_tablet_manager.cc:1428] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:57:33.723166 4163 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
#11 0x7fc7b9db679e in kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:137:3
#12 0x7fc7b9dbcb25 in kudu::rpc::ServicePool::RunThread() /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:231:15
#13 0x7fc7b9dbd54f in kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:92:5
#14 0x7fc7b9dbd378 in std::_Function_handler<void (), kudu::rpc::ServicePool::Init(int)::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/8/bits/std_function.h:297:2
#15 0x9edb7c in std::function<void ()>::operator()() const ../../../include/c++/8/bits/std_function.h:687:14
#16 0x7fc7b329feae in kudu::Thread::SuperviseThread(void*) /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:694:3
#17 0x7fc7b65796da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
SUMMARY: AddressSanitizer: heap-use-after-free /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/gutil/atomicops-internals-x86.h:200:10 in base::subtle::NoBarrier_Load(int const volatile*)
Shadow bytes around the buggy address:
0x0c328000fee0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000fef0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c328000ff20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c328000ff30: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328000ff80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
W20260811 18:57:33.725891 3966 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 28.062s user 1.075s sys 2.859s
W20260811 18:57:33.726989 4378 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:05.663892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:38889 (local address 127.3.72.67:39173)
0811 18:57:33.726206 (+28062314us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:33.726395 (+ 189us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.72.125:38889: Timeout exceeded waiting to connect
Metrics: {"client-negotiator.queue_time_us":28062104,"threads_started":1}
W20260811 18:57:33.727398 3966 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 28.063s user 1.075s sys 2.859s
I20260811 18:57:33.733657 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc16d2310b294ccb8c18945b693c6fce" } new_table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" modify_external_catalogs: true
I20260811 18:57:33.735702 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870] alter complete (version 2)
I20260811 18:57:33.737002 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6" } modify_external_catalogs: true
I20260811 18:57:33.737331 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7fac7298cb540bc89b4fc9cb2651eb6" } modify_external_catalogs: true
I20260811 18:57:33.739962 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 2c0f6df7a291456d80538e2b76417873 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:33.740785 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3ee627ac9abf47889b5fed972cecd97e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.741433 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ee627ac9abf47889b5fed972cecd97e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.743157 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.745592 4163 tablet_bootstrap.cc:654] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
==3947==ABORTING
W20260811 18:57:33.764217 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae consensus state has no leader
I20260811 18:57:33.779249 4167 ts_tablet_manager.cc:1918] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.779585 4167 log.cc:1196] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:33.780247 4167 ts_tablet_manager.cc:1939] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.781833 4163 tablet_bootstrap.cc:492] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.782011 4163 ts_tablet_manager.cc:1397] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.059s user 0.007s sys 0.002s
I20260811 18:57:33.783020 4163 raft_consensus.cc:348] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.783288 4163 raft_consensus.cc:374] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.783368 4163 raft_consensus.cc:729] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.783587 4163 consensus_queue.cc:260] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.783990 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 788) took 4854 ms. Trace:
I20260811 18:57:33.784150 4167 rpcz_store.cc:276] 0811 18:57:28.929636 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.929697 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:33.690171 (+4760474us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:33.690187 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.704242 (+ 14055us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x6150001453a4. stack: 00007fcdd297d66e 00007fcdd297d430 00007fcdd1a045ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fcdddeb504d 00007fcddf1595ec 00007fcde1ae82e6 00007fcde1b1a404 00007fcde1afb870 00007fcde1afb5fd 00000000009edb7c 00007fcdd29c3344 00007fcdd29c691b 00007fcdd29c6798
0811 18:57:33.728949 (+ 24707us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.779130 (+ 50181us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.783968 (+ 4838us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":29714176,"tablet-delete.queue_time_us":4750062}
I20260811 18:57:33.784976 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 783) took 4864 ms. Trace:
I20260811 18:57:33.784976 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet fe203387bc9c45ba82afe6b058aa5e5f (table default.Table_e67c791ed9cd4da88ae7ce53ba362301 [id=14b4d412d33c43fdb3d20c85b82c1670]) successfully deleted
I20260811 18:57:33.785112 4167 rpcz_store.cc:276] 0811 18:57:28.920626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:28.924679 (+ 4053us) service_pool.cc:227] Handling call
0811 18:57:33.784860 (+4860181us) ts_tablet_manager.cc:1145] Deleting tablet b203df4124364737839bf4c1ae072988
0811 18:57:33.784871 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.784961 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4844665}
I20260811 18:57:33.785674 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 789) took 4713 ms. Trace:
W20260811 18:57:33.785645 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b203df4124364737839bf4c1ae072988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b203df4124364737839bf4c1ae072988 already in progress: creating tablet
I20260811 18:57:33.785751 4167 rpcz_store.cc:276] 0811 18:57:29.072014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.072389 (+ 375us) service_pool.cc:227] Handling call
0811 18:57:33.785579 (+4713190us) ts_tablet_manager.cc:1145] Deleting tablet c77882ff8d974057a61f037f1943f58f
0811 18:57:33.785589 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.785661 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1540,"tablet-delete.queue_time_us":4712927}
I20260811 18:57:33.786248 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 801) took 4278 ms. Trace:
I20260811 18:57:33.786331 4167 rpcz_store.cc:276] 0811 18:57:29.508132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.508423 (+ 291us) service_pool.cc:227] Handling call
0811 18:57:33.786154 (+4277731us) ts_tablet_manager.cc:1145] Deleting tablet 6880ad27a3eb4599b407edeb2969a08f
0811 18:57:33.786164 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.786232 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4277474}
W20260811 18:57:33.788066 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6880ad27a3eb4599b407edeb2969a08f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6880ad27a3eb4599b407edeb2969a08f already in progress: creating tablet
I20260811 18:57:33.788067 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 802) took 4279 ms. Trace:
I20260811 18:57:33.788172 4167 rpcz_store.cc:276] 0811 18:57:29.508228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.508308 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:33.787962 (+4279654us) ts_tablet_manager.cc:1145] Deleting tablet 1d721b0e55384d8a8efd88a0e47b1504
0811 18:57:33.787972 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.788050 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4279259}
W20260811 18:57:33.788700 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1d721b0e55384d8a8efd88a0e47b1504 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d721b0e55384d8a8efd88a0e47b1504 already in progress: creating tablet
I20260811 18:57:33.789578 4163 ts_tablet_manager.cc:1428] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:33.791754 4163 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.792513 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 800) took 4284 ms. Trace:
I20260811 18:57:33.794464 4162 tablet_bootstrap.cc:654] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.792608 4167 rpcz_store.cc:276] 0811 18:57:29.508029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.509625 (+ 1596us) service_pool.cc:227] Handling call
0811 18:57:33.792408 (+4282783us) ts_tablet_manager.cc:1145] Deleting tablet 1c74abe9dcf642e9b4d40df6ff3639ef
0811 18:57:33.792418 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.792497 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4282447}
I20260811 18:57:33.796656 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 799) took 4288 ms. Trace:
I20260811 18:57:33.796757 4167 rpcz_store.cc:276] 0811 18:57:29.507902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.510017 (+ 2115us) service_pool.cc:227] Handling call
0811 18:57:33.796536 (+4286519us) ts_tablet_manager.cc:1145] Deleting tablet 47a782b8470b4d14ac89b3b7d463a125
0811 18:57:33.796549 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.796638 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4286265}
W20260811 18:57:33.796880 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1c74abe9dcf642e9b4d40df6ff3639ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c74abe9dcf642e9b4d40df6ff3639ef already in progress: creating tablet
I20260811 18:57:33.797160 4167 tablet_replica.cc:330] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:33.797266 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 47a782b8470b4d14ac89b3b7d463a125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47a782b8470b4d14ac89b3b7d463a125 already in progress: creating tablet
I20260811 18:57:33.797380 4167 raft_consensus.cc:2229] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.797544 4167 raft_consensus.cc:2258] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.798103 4167 ts_tablet_manager.cc:1905] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.802093 5443 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.802228 5443 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.802842 5443 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.803591 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.806191 3676 leader_election.cc:341] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.806236 5376 consensus_queue.cc:1035] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:33.808905 4163 tablet_bootstrap.cc:654] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:33.812261 3824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.667s (SENT) [suppressed 29 similar messages]
W20260811 18:57:33.820340 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
I20260811 18:57:33.828589 4163 tablet_bootstrap.cc:492] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.829063 4163 ts_tablet_manager.cc:1397] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.003s
I20260811 18:57:33.810165 4169 ts_tablet_manager.cc:1918] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:33.830673 4169 log.cc:1196] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/490f34d7a403416fbab64e46973746de
I20260811 18:57:33.831197 4162 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.831341 4162 ts_tablet_manager.cc:1397] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.186s user 0.022s sys 0.005s
I20260811 18:57:33.831964 4162 raft_consensus.cc:348] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.832154 4162 raft_consensus.cc:374] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.832204 4162 raft_consensus.cc:729] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.832396 4162 consensus_queue.cc:260] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:33.832902 4162 ts_tablet_manager.cc:1428] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 18:57:33.833145 4162 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.833536 4167 ts_tablet_manager.cc:1918] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.833635 4167 log.cc:1196] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:33.833925 4167 ts_tablet_manager.cc:1939] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.834395 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 798) took 4326 ms. Trace:
I20260811 18:57:33.834479 4167 rpcz_store.cc:276] 0811 18:57:29.507635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.510310 (+ 2675us) service_pool.cc:227] Handling call
0811 18:57:33.797099 (+4286789us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:33.797110 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.801333 (+ 4223us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.833439 (+ 32106us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.834374 (+ 935us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4286523}
I20260811 18:57:33.835143 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 1e1374a80d15444cb0faff339275aa8d (table default.Table_647ae82e63464976abb7f26c9526a349 [id=e0ef202b868f47269e4f2a63cb5a390a]) successfully deleted
I20260811 18:57:33.836016 4162 tablet_bootstrap.cc:654] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:33.840632 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae96e10a2ed4407c9d42a359d82d389e consensus state has no leader
I20260811 18:57:33.841249 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4464846117e74da2b263ff545f9937b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.841977 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4464846117e74da2b263ff545f9937b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.852567 4163 raft_consensus.cc:348] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.852831 4163 raft_consensus.cc:374] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.852893 4163 raft_consensus.cc:729] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.852772 4169 ts_tablet_manager.cc:1939] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:33.855803 4163 consensus_queue.cc:260] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.858438 4163 ts_tablet_manager.cc:1428] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260811 18:57:33.859513 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 801) took 3137 ms. Trace:
I20260811 18:57:33.860569 4163 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.860381 4169 rpcz_store.cc:276] 0811 18:57:30.720438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.720499 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:33.701264 (+2980765us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:33.701272 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.715816 (+ 14544us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.810026 (+ 94210us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.858236 (+ 48210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2966965}
I20260811 18:57:33.863093 4167 tablet_replica.cc:330] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.863423 4167 raft_consensus.cc:2229] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.863612 4167 raft_consensus.cc:2258] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.864020 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 490f34d7a403416fbab64e46973746de (table default.Table_a0c4cb29b3ec48749d83adaa68477c84 [id=9e9485a3820945b09a1840e83c3eae08]) successfully deleted
I20260811 18:57:33.864307 4167 ts_tablet_manager.cc:1905] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.866159 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 825) took 2983 ms. Trace:
I20260811 18:57:33.870428 4163 tablet_bootstrap.cc:654] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.866896 4169 rpcz_store.cc:276] 0811 18:57:30.882816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.020544 (+137728us) service_pool.cc:227] Handling call
0811 18:57:33.865060 (+2844516us) ts_tablet_manager.cc:1145] Deleting tablet 22abc6f8e9ef4c32ab69af31bdaad872
0811 18:57:33.865075 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.865863 (+ 788us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2843896}
I20260811 18:57:33.872179 4162 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:33.872330 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 22abc6f8e9ef4c32ab69af31bdaad872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22abc6f8e9ef4c32ab69af31bdaad872 already in progress: creating tablet
I20260811 18:57:33.872416 4162 ts_tablet_manager.cc:1397] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.007s
I20260811 18:57:33.873219 4162 raft_consensus.cc:348] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.873567 4162 raft_consensus.cc:374] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.873694 4162 raft_consensus.cc:729] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.874054 4162 consensus_queue.cc:260] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.875982 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7e50906e16414e8b99c43763a538d6be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.876745 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e50906e16414e8b99c43763a538d6be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:33.877959 3824 leader_election.cc:334] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.733s (SENT)
I20260811 18:57:33.877995 4162 ts_tablet_manager.cc:1428] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:33.878278 4162 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.879966 4162 tablet_bootstrap.cc:654] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.884321 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 826) took 3001 ms. Trace:
I20260811 18:57:33.885830 4167 ts_tablet_manager.cc:1918] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.885957 4167 log.cc:1196] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6f3060822021431584a494e6941be756
I20260811 18:57:33.885433 4169 rpcz_store.cc:276] 0811 18:57:30.882927 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.020952 (+138025us) service_pool.cc:227] Handling call
0811 18:57:33.881640 (+2860688us) ts_tablet_manager.cc:1145] Deleting tablet 3030c31bc2b24ee296f29b5d5aa402ff
0811 18:57:33.881936 (+ 296us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.884094 (+ 2158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2848775}
W20260811 18:57:33.890143 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3030c31bc2b24ee296f29b5d5aa402ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3030c31bc2b24ee296f29b5d5aa402ff already in progress: creating tablet
I20260811 18:57:33.891891 4163 tablet_bootstrap.cc:492] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.892095 4163 ts_tablet_manager.cc:1397] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.000s
I20260811 18:57:33.892359 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 828) took 3009 ms. Trace:
I20260811 18:57:33.893006 4163 raft_consensus.cc:348] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.893302 4163 raft_consensus.cc:374] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.893415 4163 raft_consensus.cc:729] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.892791 4169 rpcz_store.cc:276] 0811 18:57:30.883203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.029533 (+146330us) service_pool.cc:227] Handling call
0811 18:57:33.891164 (+2861631us) ts_tablet_manager.cc:1145] Deleting tablet 90fde17971ea49aaad212a5836661b08
0811 18:57:33.891568 (+ 404us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.892177 (+ 609us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2860849}
I20260811 18:57:33.893728 4163 consensus_queue.cc:260] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:33.894284 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 90fde17971ea49aaad212a5836661b08 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90fde17971ea49aaad212a5836661b08 already in progress: creating tablet
I20260811 18:57:33.897133 4163 ts_tablet_manager.cc:1428] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:33.897446 4163 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:33.899220 4163 tablet_bootstrap.cc:654] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.895689 4162 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:33.901515 4162 ts_tablet_manager.cc:1397] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
W20260811 18:57:33.902199 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae96e10a2ed4407c9d42a359d82d389e (table default.Table_60eefdf4cac142ff98c75bfd637b21af [id=b28c814fee894ee7a12d05872eea8e92], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae96e10a2ed4407c9d42a359d82d389e consensus state has no leader
I20260811 18:57:33.902187 4162 raft_consensus.cc:348] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.902539 4162 raft_consensus.cc:374] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.902667 4162 raft_consensus.cc:729] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.902935 4162 consensus_queue.cc:260] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.886435 4167 ts_tablet_manager.cc:1939] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.907119 4162 ts_tablet_manager.cc:1428] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 18:57:33.907516 4162 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:33.909569 4162 tablet_bootstrap.cc:654] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.910290 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 808) took 4381 ms. Trace:
I20260811 18:57:33.910422 4167 rpcz_store.cc:276] 0811 18:57:29.529231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.529290 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:33.863032 (+4333742us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:33.863047 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.879744 (+ 16697us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.885728 (+ 5984us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.908221 (+ 22493us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x61500011f6a4. stack: 00007fcdd297d66e 00007fcdd297d430 00007fcdd1a045ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fcdddeb4edd 00007fcddde74de4 00007fcddf15b426 00007fcde1ae86e3 00007fcde1b1a404 00007fcde1afb870 00007fcde1afb5fd 00000000009edb7c 00007fcdd29c3344 00007fcdd29c691b
0811 18:57:33.910269 (+ 2048us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5452672,"tablet-delete.queue_time_us":4333516}
I20260811 18:57:33.910976 4167 tablet_replica.cc:330] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.910925 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6f3060822021431584a494e6941be756 (table default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]) successfully deleted
I20260811 18:57:33.911229 4167 raft_consensus.cc:2229] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.911417 4167 raft_consensus.cc:2258] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.911990 4167 ts_tablet_manager.cc:1905] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:33.914057 4163 tablet_bootstrap.cc:492] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:33.914213 4163 ts_tablet_manager.cc:1397] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260811 18:57:33.915020 4163 raft_consensus.cc:348] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.915257 4163 raft_consensus.cc:374] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.915323 4163 raft_consensus.cc:729] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.915544 4163 consensus_queue.cc:260] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.916285 4163 ts_tablet_manager.cc:1428] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:33.915647 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 829) took 3024 ms. Trace:
I20260811 18:57:33.920020 3548 catalog_manager.cc:5495] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:33.919145 4169 rpcz_store.cc:276] 0811 18:57:30.883305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.030513 (+147208us) service_pool.cc:227] Handling call
0811 18:57:33.896111 (+2865598us) ts_tablet_manager.cc:1145] Deleting tablet 7d747f03ad234805a4ea4ef60840d426
0811 18:57:33.896124 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.908033 (+ 11909us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2864808}
I20260811 18:57:33.920805 4163 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:33.921797 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c6852aaef1d5486b87e5462eae532719 consensus state has no leader
I20260811 18:57:33.922799 4163 tablet_bootstrap.cc:654] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.923959 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f08236e0af834f29abafce228034a18a (DEFAULT_TABLE table=default.Table_a17ac8f677ce46efaade718a851a016d [id=41b250c044d94c37879aa50e57415f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924014 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 0128fecf972b48c2973a5c36e997f71b (DEFAULT_TABLE table=default.Table_a7bc3ed83682403b84a5a23d14cf1c3b [id=4278f7ec09d147ae8f8441f33bc4a6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924304 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0128fecf972b48c2973a5c36e997f71b (DEFAULT_TABLE table=default.Table_a7bc3ed83682403b84a5a23d14cf1c3b [id=4278f7ec09d147ae8f8441f33bc4a6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924343 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 82b04e37fc264fc08e0ed9517356a17d (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924459 3855 tablet_service.cc:1467] Processing CreateTablet for tablet fc967ce6d10b49f09731f6326cbf066c (DEFAULT_TABLE table=default.Table_7e97d2b678784537a3e986a742e9f88b [id=1bdec829404a41ad84ab102c515ac0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.924695 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08236e0af834f29abafce228034a18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924727 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0128fecf972b48c2973a5c36e997f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924907 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc967ce6d10b49f09731f6326cbf066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.924863 3720 tablet_service.cc:1467] Processing CreateTablet for tablet f08236e0af834f29abafce228034a18a (DEFAULT_TABLE table=default.Table_a17ac8f677ce46efaade718a851a016d [id=41b250c044d94c37879aa50e57415f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.925153 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0128fecf972b48c2973a5c36e997f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.925212 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08236e0af834f29abafce228034a18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.926587 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 82b04e37fc264fc08e0ed9517356a17d (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.926908 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b04e37fc264fc08e0ed9517356a17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.930109 3719 tablet_service.cc:1467] Processing CreateTablet for tablet fc967ce6d10b49f09731f6326cbf066c (DEFAULT_TABLE table=default.Table_7e97d2b678784537a3e986a742e9f88b [id=1bdec829404a41ad84ab102c515ac0b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.930481 5443 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.930612 5443 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.931236 5443 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:33.931545 5417 raft_consensus.cc:482] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:33.931676 5417 raft_consensus.cc:504] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.922255 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9d18547a0114b83bfbed5110e565661" } modify_external_catalogs: true
I20260811 18:57:33.932085 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9d18547a0114b83bfbed5110e565661" } modify_external_catalogs: true
I20260811 18:57:33.932211 5417 leader_election.cc:288] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.932413 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b04e37fc264fc08e0ed9517356a17d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.937976 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:33.938644 3676 leader_election.cc:341] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:33.939127 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3263da75f0d84528b7c8cc55dcaeaa46" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:33.939343 3743 raft_consensus.cc:2454] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:33.940604 3724 tablet_service.cc:1467] Processing CreateTablet for tablet de6245131a3a4513b88611fd4c2e031c (DEFAULT_TABLE table=default.Table_db613e17da86441bb1ab37b46e730db9 [id=d0c9376b40d34652839aec9e4e711def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.930490 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc967ce6d10b49f09731f6326cbf066c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.944483 4167 ts_tablet_manager.cc:1918] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:33.944605 4167 log.cc:1196] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:33.945339 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e1922c7317c24aa984945aaaa6ce8003 (DEFAULT_TABLE table=default.Table_3309d84857df4970a9891fd64c181bf5 [id=24a68c86a16a494a8d4b2108c7b06c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.946237 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1922c7317c24aa984945aaaa6ce8003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.946593 3705 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4d5464594c0eb19f1739c29c0b45 (DEFAULT_TABLE table=default.Table_57453b6f4942454da731710b8510f4a7 [id=c1e2053ce9d340659a6dd593465dea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.946919 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4d5464594c0eb19f1739c29c0b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.948735 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 827) took 3065 ms. Trace:
I20260811 18:57:33.948836 4169 rpcz_store.cc:276] 0811 18:57:30.883024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.024982 (+141958us) service_pool.cc:227] Handling call
0811 18:57:33.948559 (+2923577us) ts_tablet_manager.cc:1145] Deleting tablet 6009c9f6bbb0493ebe125120a1a47f3f
0811 18:57:33.948574 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.948709 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2914587}
I20260811 18:57:33.949154 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 836) took 2690 ms. Trace:
I20260811 18:57:33.949216 4169 rpcz_store.cc:276] 0811 18:57:31.259151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.259756 (+ 605us) service_pool.cc:227] Handling call
0811 18:57:33.949068 (+2689312us) ts_tablet_manager.cc:1145] Deleting tablet c944d5cfd9b3405b8151abfc9f2fd998
0811 18:57:33.949077 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949141 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2685784}
I20260811 18:57:33.949487 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 835) took 2690 ms. Trace:
I20260811 18:57:33.949558 4169 rpcz_store.cc:276] 0811 18:57:31.259055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.261289 (+ 2234us) service_pool.cc:227] Handling call
0811 18:57:33.949390 (+2688101us) ts_tablet_manager.cc:1145] Deleting tablet 01a5fdab68694761b7493b42a0bf7c5b
0811 18:57:33.949399 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949472 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659116}
I20260811 18:57:33.945346 4167 ts_tablet_manager.cc:1939] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:33.949862 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 834) took 2690 ms. Trace:
I20260811 18:57:33.949930 4169 rpcz_store.cc:276] 0811 18:57:31.258934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.260802 (+ 1868us) service_pool.cc:227] Handling call
0811 18:57:33.949760 (+2688958us) ts_tablet_manager.cc:1145] Deleting tablet c0f68d33e1b14144b34c17e3c7201408
0811 18:57:33.949771 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.949846 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2659260}
I20260811 18:57:33.950193 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 837) took 2655 ms. Trace:
I20260811 18:57:33.950255 4169 rpcz_store.cc:276] 0811 18:57:31.294282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.294528 (+ 246us) service_pool.cc:227] Handling call
0811 18:57:33.950111 (+2655583us) ts_tablet_manager.cc:1145] Deleting tablet 4e0747ef88574c68ae074732cb7095b9
0811 18:57:33.950119 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.950180 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2655361}
I20260811 18:57:33.950517 4169 tablet_replica.cc:330] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:33.950721 4169 raft_consensus.cc:2229] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:33.951023 4169 pending_rounds.cc:62] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:33.951076 4169 pending_rounds.cc:68] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400172410499072 op_type: NO_OP noop_request { }
W20260811 18:57:33.951136 3678 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Timed out: RequestConsensusVote RPC to 127.3.72.67:36027 timed out after 3.149s (SENT)
I20260811 18:57:33.951215 4169 raft_consensus.cc:2866] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:33.951304 4169 raft_consensus.cc:2258] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.951251 3678 leader_election.cc:302] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:33.951897 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6245131a3a4513b88611fd4c2e031c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.950253 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 809) took 4420 ms. Trace:
I20260811 18:57:33.952517 4167 rpcz_store.cc:276] 0811 18:57:29.529410 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.529463 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:33.910924 (+4381461us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:33.910938 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.923606 (+ 12668us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.944406 (+ 20800us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:33.950231 (+ 5825us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4380950}
I20260811 18:57:33.952865 4167 tablet_replica.cc:330] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:33.953058 4167 raft_consensus.cc:2229] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:33.954037 4167 raft_consensus.cc:2258] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:33.954735 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 221d0d8d69f142a29ec7065a82b3f769 (table default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]) successfully deleted
W20260811 18:57:33.955199 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7d747f03ad234805a4ea4ef60840d426 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d747f03ad234805a4ea4ef60840d426 already in progress: creating tablet
I20260811 18:57:33.955214 3824 leader_election.cc:302] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:33.955407 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6009c9f6bbb0493ebe125120a1a47f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6009c9f6bbb0493ebe125120a1a47f3f already in progress: creating tablet
W20260811 18:57:33.955602 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c944d5cfd9b3405b8151abfc9f2fd998 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c944d5cfd9b3405b8151abfc9f2fd998 already in progress: creating tablet
W20260811 18:57:33.955770 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 01a5fdab68694761b7493b42a0bf7c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01a5fdab68694761b7493b42a0bf7c5b already in progress: creating tablet
W20260811 18:57:33.955940 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c0f68d33e1b14144b34c17e3c7201408 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f68d33e1b14144b34c17e3c7201408 already in progress: creating tablet
I20260811 18:57:33.955920 3871 tablet_service.cc:1467] Processing CreateTablet for tablet de6245131a3a4513b88611fd4c2e031c (DEFAULT_TABLE table=default.Table_db613e17da86441bb1ab37b46e730db9 [id=d0c9376b40d34652839aec9e4e711def]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:33.956140 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4e0747ef88574c68ae074732cb7095b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e0747ef88574c68ae074732cb7095b9 already in progress: creating tablet
I20260811 18:57:33.956357 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de6245131a3a4513b88611fd4c2e031c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.956355 3714 tablet_service.cc:1467] Processing CreateTablet for tablet c44eee516d634d9a986107b55410dfc3 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.956686 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44eee516d634d9a986107b55410dfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.956976 3709 tablet_service.cc:1467] Processing CreateTablet for tablet bc5105a88ad540adb326a15271237b5c (DEFAULT_TABLE table=default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 [id=30255d6b39ce43e6b7e98a2f2d9b210a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.957297 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5105a88ad540adb326a15271237b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.958580 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 683f257f18054935adcf17b00470d784 (DEFAULT_TABLE table=default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 [id=539b401b507c4ef9ad1fe5579b13dcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958626 3867 tablet_service.cc:1467] Processing CreateTablet for tablet fcac4d5464594c0eb19f1739c29c0b45 (DEFAULT_TABLE table=default.Table_57453b6f4942454da731710b8510f4a7 [id=c1e2053ce9d340659a6dd593465dea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958626 3855 tablet_service.cc:1467] Processing CreateTablet for tablet bc5105a88ad540adb326a15271237b5c (DEFAULT_TABLE table=default.Table_ad9f3fa268174f9aa74bcee2360bf4d6 [id=30255d6b39ce43e6b7e98a2f2d9b210a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.958899 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683f257f18054935adcf17b00470d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.959043 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5105a88ad540adb326a15271237b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.962631 3857 tablet_service.cc:1467] Processing CreateTablet for tablet c44eee516d634d9a986107b55410dfc3 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.962800 5417 raft_consensus.cc:2781] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:33.962896 5417 raft_consensus.cc:482] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:33.962961 5417 raft_consensus.cc:3037] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.962984 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c44eee516d634d9a986107b55410dfc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.964697 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6b28d478364f4057a8c2619e5d90b850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.964895 4167 ts_tablet_manager.cc:1905] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:33.965271 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b28d478364f4057a8c2619e5d90b850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.965478 5417 raft_consensus.cc:504] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.966118 5417 leader_election.cc:288] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:33.966598 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 683f257f18054935adcf17b00470d784 (DEFAULT_TABLE table=default.Table_6a73c7e3fbac4e8ba030a4c4d0c2c175 [id=539b401b507c4ef9ad1fe5579b13dcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.966945 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683f257f18054935adcf17b00470d784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.968086 5443 raft_consensus.cc:2726] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:33.976804 3853 tablet_service.cc:1467] Processing CreateTablet for tablet dae10844d9574b1da443358b37c7ab38 (DEFAULT_TABLE table=default.Table_73bf9f47862a4f03ac5e35afacfc2463 [id=685d668b3f3f42549c9e228ea9cdb981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.976982 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe (DEFAULT_TABLE table=default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c [id=7247bf0c19b6435795201ca8b085cb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.977265 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae10844d9574b1da443358b37c7ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977490 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe (DEFAULT_TABLE table=default.Table_56b0e5a4252b4788bb35fc7e0f1dee6c [id=7247bf0c19b6435795201ca8b085cb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.977666 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3263da75f0d84528b7c8cc55dcaeaa46" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:33.977859 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977977 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d69f2a71f4b9dab2afdbf9ef97cfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.977869 3743 raft_consensus.cc:3037] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.982657 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f86ba123808f41669c4e400da7c639ce (DEFAULT_TABLE table=default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 [id=dff3f101d29e4ce180b8afa24d7b248d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:33.983021 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86ba123808f41669c4e400da7c639ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.985396 3743 raft_consensus.cc:2454] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:33.987996 3824 leader_election.cc:302] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:33.991561 5417 raft_consensus.cc:2781] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.991819 5417 raft_consensus.cc:686] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:33.992065 5417 consensus_queue.cc:237] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:33.992903 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_489a043a9e89488ca2fbcd69855c4e5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:33.993476 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcac4d5464594c0eb19f1739c29c0b45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:33.993547 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_489a043a9e89488ca2fbcd69855c4e5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.994997 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 05cba6b392b642e59321570aac3541d0
I20260811 18:57:33.995739 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_33ef706171fc47669bf54e9a71d90905"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:33.996356 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33ef706171fc47669bf54e9a71d90905 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:33.999545 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cba6b392b642e59321570aac3541d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:33.999817 4169 ts_tablet_manager.cc:1905] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.000769 3872 tablet_service.cc:1467] Processing CreateTablet for tablet e1922c7317c24aa984945aaaa6ce8003 (DEFAULT_TABLE table=default.Table_3309d84857df4970a9891fd64c181bf5 [id=24a68c86a16a494a8d4b2108c7b06c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.001192 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1922c7317c24aa984945aaaa6ce8003. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.001312 3678 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.001448 3678 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.001552 3678 leader_election.cc:302] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.002007 3678 leader_election.cc:334] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.002219 3678 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.002728 5443 raft_consensus.cc:2726] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.002303 3678 leader_election.cc:302] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.003621 3678 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.003721 3678 leader_election.cc:302] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.003867 3678 raft_consensus.cc:2670] T fe203387bc9c45ba82afe6b058aa5e5f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.004455 3678 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.004555 3678 leader_election.cc:302] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.004688 3678 raft_consensus.cc:2670] T 221d0d8d69f142a29ec7065a82b3f769 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.005153 3678 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.005247 3678 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.005357 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 05cba6b392b642e59321570aac3541d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.005442 5407 raft_consensus.cc:2726] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.005625 5407 raft_consensus.cc:2726] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.005548 3678 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.006203 3678 leader_election.cc:302] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.006786 3678 leader_election.cc:334] T 515d5b5afbd74ec5ab90a503aef2f4fb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.006868 5407 raft_consensus.cc:2726] T e902efbb774244d18e3ba165bfb7c695 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.005843 3720 tablet_service.cc:1467] Processing CreateTablet for tablet f86ba123808f41669c4e400da7c639ce (DEFAULT_TABLE table=default.Table_a4e3a3cb2f24408ba3e2e8510459eee4 [id=dff3f101d29e4ce180b8afa24d7b248d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:34.007324 3678 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.007481 3678 leader_election.cc:302] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.007593 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86ba123808f41669c4e400da7c639ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.007671 3678 raft_consensus.cc:2670] T 6f3060822021431584a494e6941be756 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.008836 3678 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.008864 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 101994cc708a4fadb46fc8c5915a5587
I20260811 18:57:34.009023 3678 leader_election.cc:302] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.009397 3678 raft_consensus.cc:2670] T 867259e37fe44c33a35823ec2d896754 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.010532 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 101994cc708a4fadb46fc8c5915a5587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.013255 3824 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.013716 3824 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.014002 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 101994cc708a4fadb46fc8c5915a5587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.014113 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_897c0f5693354801afbbaa4de2ad5252" } new_table_name: "default.Table_667271a3af6a4ab489e6ef9a2a2769a0" modify_external_catalogs: true
W20260811 18:57:34.014514 3824 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.016453 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:34.017345 3824 leader_election.cc:334] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.017562 3824 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.017711 3824 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.019729 3824 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020193 3824 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020608 3824 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.020814 3824 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.022132 3541 connection.cc:535] client connection to 127.3.72.67:36027 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.022424 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 3 similar messages]
I20260811 18:57:34.030720 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7
I20260811 18:57:34.031137 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 deleted and replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8
I20260811 18:57:34.031486 3709 tablet_service.cc:1467] Processing CreateTablet for tablet dae10844d9574b1da443358b37c7ab38 (DEFAULT_TABLE table=default.Table_73bf9f47862a4f03ac5e35afacfc2463 [id=685d668b3f3f42549c9e228ea9cdb981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.032970 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28350d4a6baf4232bef35569f82e0d4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.033665 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28350d4a6baf4232bef35569f82e0d4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.039458 4163 tablet_bootstrap.cc:492] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:34.039459 3678 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.039616 3678 leader_election.cc:302] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.039655 4163 ts_tablet_manager.cc:1397] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.119s user 0.003s sys 0.000s
W20260811 18:57:34.039851 3678 raft_consensus.cc:2670] T 06b8f2a97c0f46ea8c56b3de2b97e60c P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.040618 4163 raft_consensus.cc:348] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.040930 4163 raft_consensus.cc:374] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.041005 4163 raft_consensus.cc:729] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.041215 4163 consensus_queue.cc:260] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.047186 4163 ts_tablet_manager.cc:1428] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
W20260811 18:57:34.047189 3824 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.048488 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7216c095cc03424bbbe97a52a9600ebb
I20260811 18:57:34.049083 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 40898ceb18cc4f2d9104818798049ae2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.050402 4162 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.050733 4162 ts_tablet_manager.cc:1397] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.143s user 0.004s sys 0.001s
I20260811 18:57:34.051579 4162 raft_consensus.cc:348] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.051880 4162 raft_consensus.cc:374] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.051985 4162 raft_consensus.cc:729] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.052261 4162 consensus_queue.cc:260] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.052703 4163 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.052987 4162 ts_tablet_manager.cc:1428] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:34.054888 4163 tablet_bootstrap.cc:654] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.058983 3824 leader_election.cc:334] T 06b8f2a97c0f46ea8c56b3de2b97e60c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.059950 3824 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.060909 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae10844d9574b1da443358b37c7ab38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.063287 4162 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.064960 4162 tablet_bootstrap.cc:654] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.066095 3824 leader_election.cc:334] T ad424c8843ff434ba0449a86849bb5a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.067482 4167 ts_tablet_manager.cc:1918] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.067624 4167 log.cc:1196] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fbba1923ce0049448fe4e28f2dd258dc
W20260811 18:57:34.070537 3678 leader_election.cc:334] T 460e03cc14444c83b6c08ca194ca5a8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.071558 3824 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.072353 3678 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.070602 4167 ts_tablet_manager.cc:1939] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.073510 3824 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.073588 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 810) took 4543 ms. Trace:
I20260811 18:57:34.073673 4167 rpcz_store.cc:276] 0811 18:57:29.529948 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.530006 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:33.952814 (+4422808us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:33.952828 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:33.967830 (+ 15002us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.045734 (+ 77904us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.073571 (+ 27837us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4422630}
W20260811 18:57:34.074270 3678 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.074673 3824 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.075001 3678 leader_election.cc:302] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.079843 5407 raft_consensus.cc:482] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:34.080016 3678 raft_consensus.cc:2670] T b6f36ab6bb774257b5c2692c64eff63d P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.079995 5407 raft_consensus.cc:504] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.080368 5417 raft_consensus.cc:482] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.080526 5407 leader_election.cc:288] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.080518 5417 raft_consensus.cc:504] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:34.080873 3824 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.081100 3824 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081151 5417 leader_election.cc:288] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:34.080535 3678 leader_election.cc:334] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081632 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968fa8b8912b40a8bdba77db1d39c5c0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.081753 3678 leader_election.cc:334] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.081847 3678 leader_election.cc:302] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.081984 3678 raft_consensus.cc:2670] T 799ba9a044e145ec89d0025c03f391ea P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.082628 3824 leader_election.cc:334] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.082904 3824 leader_election.cc:334] T ef393c341bb54baf87176db5b5254ec7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.082224 3676 leader_election.cc:341] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.088044 3678 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.088289 3824 leader_election.cc:334] T d2d7d986900244ddbc05dbe73a708dcf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.088451 3678 leader_election.cc:302] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.089039 3824 leader_election.cc:334] T b6f36ab6bb774257b5c2692c64eff63d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.090539 4167 tablet_replica.cc:330] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.091290 4167 raft_consensus.cc:2229] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.091522 4167 raft_consensus.cc:2258] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.092108 4167 ts_tablet_manager.cc:1905] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.095772 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7be6d9578e7847c2822278486119390c
I20260811 18:57:34.096614 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet a25b1eddd4a64847b51ad3cdc0238ff0 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:34.098519 3678 raft_consensus.cc:2670] T d2d7d986900244ddbc05dbe73a708dcf P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.099215 4162 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.099994 4162 ts_tablet_manager.cc:1397] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.001s
I20260811 18:57:34.100843 4162 raft_consensus.cc:348] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.101145 4162 raft_consensus.cc:374] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.101390 4162 raft_consensus.cc:729] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.101742 4162 consensus_queue.cc:260] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.102499 4162 ts_tablet_manager.cc:1428] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:57:34.103363 3824 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.103809 4162 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:34.105906 3678 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.106139 3678 leader_election.cc:302] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.106386 3678 raft_consensus.cc:2670] T a902ad70a25546be926ff8f7bf2ed64f P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.107769 3824 leader_election.cc:334] T 0ba6246e6d29418cbca5f751b34f4ef1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.108655 3824 leader_election.cc:334] T ff1be026ca404eada53f60f257ea367c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.109414 4162 tablet_bootstrap.cc:654] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.109885 3678 leader_election.cc:334] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.111543 3678 leader_election.cc:302] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.111616 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
W20260811 18:57:34.111780 3678 raft_consensus.cc:2670] T cbf6ce167fb348be8d61372791f0cbca P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.112298 3678 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.113413 3824 leader_election.cc:334] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.114423 3678 leader_election.cc:302] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.117547 3678 raft_consensus.cc:2670] T b0b199f4c9e14b468cb26cbd61fc8586 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.117699 3824 leader_election.cc:302] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.118384 3678 leader_election.cc:334] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.118837 3824 raft_consensus.cc:2670] T c2247194db704f9abd54a8a0df1ca566 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.122826 3824 leader_election.cc:334] T 67e63ee9b9fd4b12ae791b9d5bc2e648 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.123062 3678 leader_election.cc:302] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.123482 3678 raft_consensus.cc:2670] T c08436f33db043b1bc6395cd92102e59 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.123307 4167 ts_tablet_manager.cc:1918] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.124264 4167 log.cc:1196] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:34.124714 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c031c1fe18014450aa4cffbc9cd0f573"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.125291 4167 ts_tablet_manager.cc:1939] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.125393 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c031c1fe18014450aa4cffbc9cd0f573 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.125667 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 807) took 4596 ms. Trace:
I20260811 18:57:34.125754 4167 rpcz_store.cc:276] 0811 18:57:29.528947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.530011 (+ 1064us) service_pool.cc:227] Handling call
0811 18:57:34.090477 (+4560466us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:34.090494 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.095328 (+ 4834us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.098290 (+ 2962us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.125643 (+ 27353us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4560219}
I20260811 18:57:34.126811 4163 tablet_bootstrap.cc:492] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.127003 4163 ts_tablet_manager.cc:1397] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.002s
I20260811 18:57:34.126999 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.127604 3678 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.127763 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.127840 4163 raft_consensus.cc:348] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.128094 4163 raft_consensus.cc:374] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.128180 4163 raft_consensus.cc:729] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.128422 4163 consensus_queue.cc:260] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.128711 3678 leader_election.cc:302] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.128549 3824 leader_election.cc:334] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:34.129037 3678 raft_consensus.cc:2670] T 02808605ee6c4634af47fc3fd57e9cd8 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.129177 3824 leader_election.cc:302] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.129364 3824 raft_consensus.cc:2670] T f61d3822fc2d4b569f223238a6464e0a P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.130038 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 277a245243d14de3be26ce3962e1a9b1 (DEFAULT_TABLE table=default.Table_1dc240fd92f941489de07a83cdafb797 [id=3921e9add8ac4b0caa95a18563adb2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.129547 4163 ts_tablet_manager.cc:1428] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260811 18:57:34.131271 3678 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.131743 3678 leader_election.cc:302] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.132373 3678 raft_consensus.cc:2670] T f335e50ae57547ed80b144129faaffbb P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.133229 3678 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.135840 3678 leader_election.cc:302] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.136637 5407 raft_consensus.cc:2726] T 65234acc3d5a462683edbe8f223eeefc P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.137327 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 816) took 4553 ms. Trace:
I20260811 18:57:34.137488 4167 rpcz_store.cc:276] 0811 18:57:29.583723 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.583806 (+ 83us) service_pool.cc:227] Handling call
0811 18:57:34.129862 (+4546056us) ts_tablet_manager.cc:1145] Deleting tablet f3670f8908d54ebb82f698cdd207e48f
0811 18:57:34.137143 (+ 7281us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.137303 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4537805}
I20260811 18:57:34.138247 4163 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.139971 4163 tablet_bootstrap.cc:654] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.138049 4162 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.142944 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277a245243d14de3be26ce3962e1a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.144315 4162 ts_tablet_manager.cc:1397] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.000s
I20260811 18:57:34.145123 4162 raft_consensus.cc:348] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.145339 4162 raft_consensus.cc:374] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.145395 4162 raft_consensus.cc:729] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.145602 4162 consensus_queue.cc:260] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.146224 4167 tablet_replica.cc:330] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.146219 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dd80633f926942e88f54b32d9a03ea3a
I20260811 18:57:34.146294 4162 ts_tablet_manager.cc:1428] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:34.146441 4167 raft_consensus.cc:2229] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.146566 4162 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.146580 4167 raft_consensus.cc:2258] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.149001 4167 ts_tablet_manager.cc:1905] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.149277 5417 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.149427 5417 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.150054 5417 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.152066 4162 tablet_bootstrap.cc:654] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.153363 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
W20260811 18:57:34.158098 3678 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.159353 3678 leader_election.cc:302] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.160220 5407 raft_consensus.cc:2726] T 71df3738a5cf4b12820dda131ced62e1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.161211 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.161279 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 084b60eeb6b64e1da7336203d75770f4 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.161610 3719 tablet_service.cc:1467] Processing CreateTablet for tablet eb43127dda814a718b338906fc2a3c47 (DEFAULT_TABLE table=default.Table_91bea743ec0b491299f45f3ac23cb734 [id=630cc439da6f48acb379cff1ba6d003a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.161744 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b60eeb6b64e1da7336203d75770f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.161996 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb43127dda814a718b338906fc2a3c47. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.162602 3678 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.162719 3678 leader_election.cc:302] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.162848 3678 raft_consensus.cc:2670] T 490f34d7a403416fbab64e46973746de P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.163519 3678 leader_election.cc:334] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.163615 3678 leader_election.cc:302] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.166424 5407 raft_consensus.cc:2726] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.166900 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 20da7ea64e3d49e08b3886693625acad (DEFAULT_TABLE table=default.Table_2e691dcf4a764665aa9275bfa441105a [id=fda127e467d449368d65942e5e5b98f7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:34.167124 3678 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.167229 3678 leader_election.cc:302] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.167322 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20da7ea64e3d49e08b3886693625acad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.167371 3678 raft_consensus.cc:2670] T fbba1923ce0049448fe4e28f2dd258dc P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.168826 4163 tablet_bootstrap.cc:492] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.168977 4163 ts_tablet_manager.cc:1397] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.003s
I20260811 18:57:34.170255 4163 raft_consensus.cc:348] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.170531 4163 raft_consensus.cc:374] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.170639 4163 raft_consensus.cc:729] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.170680 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 9390e8e7967643ab8decce41dde094b2 (DEFAULT_TABLE table=default.Table_9a13d9e49bc748e6b820a614e55ba4c0 [id=d1b8f780910b4d5bb2eeb196d7ff9b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.171047 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9390e8e7967643ab8decce41dde094b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.170924 4163 consensus_queue.cc:260] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.170909 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 7e288f51b09f4c79bb78a9c52a33e16f (DEFAULT_TABLE table=default.Table_79f5b2394f094ecc85733e272dcfe369 [id=f35c0335cfe241b7971283ad3b09f2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.171571 4163 ts_tablet_manager.cc:1428] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:34.172441 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e288f51b09f4c79bb78a9c52a33e16f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:34.174674 3678 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.174865 3678 leader_election.cc:302] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.175063 3678 raft_consensus.cc:2670] T 0b3ec38f9b55489ba00ed76f9e4456f5 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.175676 3678 leader_election.cc:334] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:34.175846 3678 leader_election.cc:302] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.176033 3678 raft_consensus.cc:2670] T 43ad6993b63e4bbd98c8cc52f1a53451 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.178174 3724 tablet_service.cc:1467] Processing CreateTablet for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5 (DEFAULT_TABLE table=default.Table_8f1d0ca0ad934be687840ccd25db713a [id=c7d39b2c17b54f489fc99c48e6554637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178150 3715 tablet_service.cc:1467] Processing CreateTablet for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2 (DEFAULT_TABLE table=default.Table_4e35d031a62148d582b7bdb0240f63da [id=b48dc34f86424368ab050aa368f8759b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178295 3705 tablet_service.cc:1467] Processing CreateTablet for tablet e861f6e4e3f645de84a67ff87124f057 (DEFAULT_TABLE table=default.Table_4dc767eb645648aebd0aacf2d46f9c3b [id=56302f9edb93485cb3dc030323072103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.178591 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.178563 5417 raft_consensus.cc:482] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.178764 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.178704 5417 raft_consensus.cc:504] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.179401 5417 leader_election.cc:288] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.179818 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e861f6e4e3f645de84a67ff87124f057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.193686 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 3b540ae350254346a675b54118c5cfcb (DEFAULT_TABLE table=default.Table_1bfaa8ab4fcd40efb60e1718e424bccf [id=bbe3afd3838c4c2ebf633f308d3de7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.194257 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b540ae350254346a675b54118c5cfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.196224 4167 ts_tablet_manager.cc:1918] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.196375 4167 log.cc:1196] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:34.196707 4167 ts_tablet_manager.cc:1939] T 122b7898fea44ef78f269b1505d8ddc4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.196823 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:34.197490 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 817) took 4613 ms. Trace:
I20260811 18:57:34.197602 4167 rpcz_store.cc:276] 0811 18:57:29.584153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:29.584218 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:34.146173 (+4561955us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:34.146185 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.157083 (+ 10898us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.196121 (+ 39038us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.197470 (+ 1349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4527748}
I20260811 18:57:34.197948 4167 tablet_replica.cc:330] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.198192 4167 raft_consensus.cc:2229] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.201167 4163 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.203199 4163 tablet_bootstrap.cc:654] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.210745 4167 raft_consensus.cc:2258] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.210644 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7009400051004543ade835c61177ac98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.211337 4167 ts_tablet_manager.cc:1905] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:34.211944 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7009400051004543ade835c61177ac98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.203364 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 7216c095cc03424bbbe97a52a9600ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.209270 3678 leader_election.cc:334] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.214601 3678 leader_election.cc:302] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.214993 3678 raft_consensus.cc:2670] T 4b6ec419ed7747c3b24dc11992518e88 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.215710 3678 leader_election.cc:334] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.216162 3678 leader_election.cc:302] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.216504 3678 raft_consensus.cc:2670] T 1e1374a80d15444cb0faff339275aa8d P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.217228 3678 leader_election.cc:334] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.218101 3678 leader_election.cc:302] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.218879 5407 raft_consensus.cc:2726] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.218914 3678 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.219432 3678 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.219877 5407 raft_consensus.cc:2726] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.219924 3678 leader_election.cc:334] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.220389 3678 leader_election.cc:302] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.220896 5407 raft_consensus.cc:2726] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.220942 3678 leader_election.cc:334] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.221421 3678 leader_election.cc:302] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.221762 5407 raft_consensus.cc:2726] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.221819 3678 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.222975 3678 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.222656 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.222795 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.222525 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.223448 5407 raft_consensus.cc:2726] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.223526 3678 leader_election.cc:334] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.224437 3678 leader_election.cc:302] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.224556 4169 ts_tablet_manager.cc:1918] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.224709 4169 log.cc:1196] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f335e50ae57547ed80b144129faaffbb
I20260811 18:57:34.225025 5407 raft_consensus.cc:2726] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.225131 3678 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.225250 3678 leader_election.cc:302] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.225561 4169 ts_tablet_manager.cc:1939] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.226051 5407 raft_consensus.cc:2726] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.226100 3678 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.226199 3678 leader_election.cc:302] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.231174 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.231483 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fe203387bc9c45ba82afe6b058aa5e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.230708 3678 raft_consensus.cc:2670] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:34.232854 3678 consensus_peers.cc:487] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.233359 3678 consensus_peers.cc:487] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.233729 3678 consensus_peers.cc:487] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234043 3678 consensus_peers.cc:487] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234485 3678 consensus_peers.cc:487] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.234782 3678 consensus_peers.cc:487] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.235232 3678 consensus_peers.cc:487] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.235831 3678 consensus_peers.cc:487] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.235998 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 43ad6993b63e4bbd98c8cc52f1a53451 (table default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]) successfully deleted
W20260811 18:57:34.236367 3678 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.236723 3678 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.237165 3678 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.237619 3678 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.237803 5407 raft_consensus.cc:2726] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.238058 5443 raft_consensus.cc:2726] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.238421 3678 leader_election.cc:334] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.238783 3678 leader_election.cc:302] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.239161 5407 raft_consensus.cc:2726] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.239416 3678 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.239722 3678 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.240097 5407 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.240372 3678 leader_election.cc:334] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.240676 3678 leader_election.cc:302] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.242643 5407 raft_consensus.cc:2726] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.243021 3678 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.243386 3678 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.243835 5407 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.244251 3678 consensus_peers.cc:487] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.227365 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be6d9578e7847c2822278486119390c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:34.245126 3678 consensus_peers.cc:487] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.252080 3678 leader_election.cc:334] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.252650 3678 leader_election.cc:302] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.253189 5407 raft_consensus.cc:2726] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.254556 5407 raft_consensus.cc:482] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.254747 5407 raft_consensus.cc:504] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.255360 5407 leader_election.cc:288] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.258857 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0db19ab70f764e00b17ee2e0f4742b8b" } new_table_name: "default.Table_d85b163d70284446be67f85eddf05383" modify_external_catalogs: true
I20260811 18:57:34.259342 4162 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.259485 4162 ts_tablet_manager.cc:1397] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.113s user 0.003s sys 0.010s
I20260811 18:57:34.260342 4162 raft_consensus.cc:348] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.260550 4162 raft_consensus.cc:374] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.260601 4162 raft_consensus.cc:729] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.260787 4162 consensus_queue.cc:260] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.261137 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a79b27c0bd524af985c2e6167624551a" } new_table_name: "default.Table_5fe41c188c7b4726bbc18f5bcf86b3c2" modify_external_catalogs: true
I20260811 18:57:34.261482 4162 ts_tablet_manager.cc:1428] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:57:34.237653 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f3670f8908d54ebb82f698cdd207e48f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3670f8908d54ebb82f698cdd207e48f already in progress: creating tablet
I20260811 18:57:34.262353 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f4e1fc5686944dfa3eb57702c810eb8) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.262753 4162 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.262703 5417 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.262851 5417 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.263465 5417 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.264813 3853 tablet_service.cc:1467] Processing CreateTablet for tablet eb43127dda814a718b338906fc2a3c47 (DEFAULT_TABLE table=default.Table_91bea743ec0b491299f45f3ac23cb734 [id=630cc439da6f48acb379cff1ba6d003a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.265031 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 122b7898fea44ef78f269b1505d8ddc4 (table default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b [id=1bcc678c540844b38d761cec1618c852]) successfully deleted
W20260811 18:57:34.262521 3678 leader_election.cc:334] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.265748 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 20da7ea64e3d49e08b3886693625acad (DEFAULT_TABLE table=default.Table_2e691dcf4a764665aa9275bfa441105a [id=fda127e467d449368d65942e5e5b98f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.265859 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 277a245243d14de3be26ce3962e1a9b1 (DEFAULT_TABLE table=default.Table_1dc240fd92f941489de07a83cdafb797 [id=3921e9add8ac4b0caa95a18563adb2c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.266330 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 7e288f51b09f4c79bb78a9c52a33e16f (DEFAULT_TABLE table=default.Table_79f5b2394f094ecc85733e272dcfe369 [id=f35c0335cfe241b7971283ad3b09f2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.266754 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 084b60eeb6b64e1da7336203d75770f4 (DEFAULT_TABLE table=default.Table_47c0e5602f6b4e1793b88727c8bfdaee [id=6435e5701c3848dd8e321d24e24ab12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.267899 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aee6beb2e70421083841520733087f8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.268357 4162 tablet_bootstrap.cc:654] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.268774 3676 leader_election.cc:341] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.268940 3676 leader_election.cc:302] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.269420 5407 raft_consensus.cc:2726] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.235378 4167 ts_tablet_manager.cc:1918] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.269675 4167 log.cc:1196] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:34.269644 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 839) took 2906 ms. Trace:
I20260811 18:57:34.270046 4167 ts_tablet_manager.cc:1939] T c26851245cab4a2d80e94d4fe278fd50 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.269831 4169 rpcz_store.cc:276] 0811 18:57:31.363218 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.363296 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:33.950472 (+2587176us) ts_tablet_manager.cc:1145] Deleting tablet f335e50ae57547ed80b144129faaffbb
0811 18:57:33.950482 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.059351 (+108869us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.224422 (+165071us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.269525 (+ 45103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2586915}
I20260811 18:57:34.270434 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.270634 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.270598 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 838) took 2907 ms. Trace:
I20260811 18:57:34.270752 4169 rpcz_store.cc:276] 0811 18:57:31.362814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.364000 (+ 1186us) service_pool.cc:227] Handling call
0811 18:57:34.270404 (+2906404us) ts_tablet_manager.cc:1145] Deleting tablet 67e63ee9b9fd4b12ae791b9d5bc2e648
0811 18:57:34.270418 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.270552 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2906100}
I20260811 18:57:34.271445 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 841) took 2884 ms. Trace:
I20260811 18:57:34.271641 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 9390e8e7967643ab8decce41dde094b2 (DEFAULT_TABLE table=default.Table_9a13d9e49bc748e6b820a614e55ba4c0 [id=d1b8f780910b4d5bb2eeb196d7ff9b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.271857 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 826) took 4338 ms. Trace:
I20260811 18:57:34.271634 4169 rpcz_store.cc:276] 0811 18:57:31.387004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.387071 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:34.271212 (+2884141us) ts_tablet_manager.cc:1145] Deleting tablet 99e9474ef8814514832f0e48c2b3495f
0811 18:57:34.271226 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.271424 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2883791}
I20260811 18:57:34.272595 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb43127dda814a718b338906fc2a3c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.271925 4167 rpcz_store.cc:276] 0811 18:57:29.933454 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.070108 (+136654us) service_pool.cc:227] Handling call
0811 18:57:34.197906 (+4127798us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:34.197917 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.225852 (+ 27935us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.235266 (+ 9414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.271836 (+ 36570us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4125317}
I20260811 18:57:34.273139 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dae19af473234626ac121fc618e92a4b" } new_table_name: "default.Table_c48fbb6941794b85becdf643c4d21775" modify_external_catalogs: true
I20260811 18:57:34.274636 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7216c095cc03424bbbe97a52a9600ebb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.274910 5443 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.275024 5443 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.275480 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.275574 5443 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.275839 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.276177 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.276722 3676 leader_election.cc:341] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.277145 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20da7ea64e3d49e08b3886693625acad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.277681 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 828) took 4120 ms. Trace:
I20260811 18:57:34.277769 4167 rpcz_store.cc:276] 0811 18:57:30.157532 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.161642 (+ 4110us) service_pool.cc:227] Handling call
0811 18:57:34.277571 (+4115929us) ts_tablet_manager.cc:1145] Deleting tablet 98ad733726594d2a8087f857f155dbe2
0811 18:57:34.277583 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.277665 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4115662}
I20260811 18:57:34.277823 3864 tablet_service.cc:1467] Processing CreateTablet for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5 (DEFAULT_TABLE table=default.Table_8f1d0ca0ad934be687840ccd25db713a [id=c7d39b2c17b54f489fc99c48e6554637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.278173 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fd5a64e2ac47e7b2b2209eb7f23ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278249 4163 tablet_bootstrap.cc:492] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.278411 4163 ts_tablet_manager.cc:1397] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.077s user 0.001s sys 0.003s
I20260811 18:57:34.278765 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277a245243d14de3be26ce3962e1a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278872 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 829) took 4119 ms. Trace:
I20260811 18:57:34.279035 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084b60eeb6b64e1da7336203d75770f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.278954 4167 rpcz_store.cc:276] 0811 18:57:30.159306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.163887 (+ 4581us) service_pool.cc:227] Handling call
0811 18:57:34.278214 (+4114327us) ts_tablet_manager.cc:1145] Deleting tablet 85140a3800f34a23b716703fd4e73d17
0811 18:57:34.278753 (+ 539us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.278853 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4113992}
W20260811 18:57:34.279958 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet f335e50ae57547ed80b144129faaffbb: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.281030 4167 tablet_replica.cc:330] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.281558 4167 raft_consensus.cc:2229] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.283038 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e288f51b09f4c79bb78a9c52a33e16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.285298 3864 tablet_service.cc:1467] Processing CreateTablet for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2 (DEFAULT_TABLE table=default.Table_4e35d031a62148d582b7bdb0240f63da [id=b48dc34f86424368ab050aa368f8759b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.285769 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce05ffc7b32e4d2f8edde7ad9755c4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.287113 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9390e8e7967643ab8decce41dde094b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.287235 4167 raft_consensus.cc:2258] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.289011 4167 ts_tablet_manager.cc:1905] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.290911 3855 tablet_service.cc:1467] Processing CreateTablet for tablet e861f6e4e3f645de84a67ff87124f057 (DEFAULT_TABLE table=default.Table_4dc767eb645648aebd0aacf2d46f9c3b [id=56302f9edb93485cb3dc030323072103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.291586 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:34.291801 3743 raft_consensus.cc:2454] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:34.292894 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 3b540ae350254346a675b54118c5cfcb (DEFAULT_TABLE table=default.Table_1bfaa8ab4fcd40efb60e1718e424bccf [id=bbe3afd3838c4c2ebf633f308d3de7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.295517 4162 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.295746 4162 ts_tablet_manager.cc:1397] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.002s
I20260811 18:57:34.296669 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 43ad6993b63e4bbd98c8cc52f1a53451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.296612 4162 raft_consensus.cc:348] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.296833 4162 raft_consensus.cc:374] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.296897 4162 raft_consensus.cc:729] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.297113 4162 consensus_queue.cc:260] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.297261 4163 raft_consensus.cc:348] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.297538 4163 raft_consensus.cc:374] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.297724 4162 ts_tablet_manager.cc:1428] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:34.297637 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 840) took 2910 ms. Trace:
I20260811 18:57:34.297741 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be6d9578e7847c2822278486119390c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.297984 4162 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.297837 4169 rpcz_store.cc:276] 0811 18:57:31.386765 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.387457 (+ 692us) service_pool.cc:227] Handling call
0811 18:57:34.272095 (+2884638us) ts_tablet_manager.cc:1145] Deleting tablet 761902f7366d43ffbde226a0ade2e188
0811 18:57:34.297362 (+ 25267us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x618000005118. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d090f80fd 00007f4d090f916e 00007f4d1c04dcfd 00007f4d1bfcb883 00007f4d1826b6e6 00007f4d1826ca37 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c
0811 18:57:34.297375 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.297589 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":56460032,"tablet-delete.queue_time_us":2884316}
I20260811 18:57:34.299283 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 849) took 2560 ms. Trace:
I20260811 18:57:34.299423 4169 rpcz_store.cc:276] 0811 18:57:31.738802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.738859 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:34.298486 (+2559627us) ts_tablet_manager.cc:1145] Deleting tablet 35ebc32eb5d5443eaaeba245b566f98a
0811 18:57:34.298528 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.299237 (+ 709us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2559397}
I20260811 18:57:34.299825 4162 tablet_bootstrap.cc:654] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.300091 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 847) took 2564 ms. Trace:
I20260811 18:57:34.300318 4169 rpcz_store.cc:276] 0811 18:57:31.735570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.739154 (+ 3584us) service_pool.cc:227] Handling call
0811 18:57:34.299892 (+2560738us) ts_tablet_manager.cc:1145] Deleting tablet 1eb30e64e86040098dcee5fc2e9b7266
0811 18:57:34.299905 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.300048 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2560413}
I20260811 18:57:34.301016 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 221d0d8d69f142a29ec7065a82b3f769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.297623 4163 raft_consensus.cc:729] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.301558 4163 consensus_queue.cc:260] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.303179 4163 ts_tablet_manager.cc:1428] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260811 18:57:34.304968 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e861f6e4e3f645de84a67ff87124f057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.307531 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 490f34d7a403416fbab64e46973746de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:13 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.308581 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet dd80633f926942e88f54b32d9a03ea3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.300989 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 850) took 2544 ms. Trace:
I20260811 18:57:34.310722 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b540ae350254346a675b54118c5cfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.309706 4169 rpcz_store.cc:276] 0811 18:57:31.756579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.785593 (+ 29014us) service_pool.cc:227] Handling call
0811 18:57:34.300793 (+2515200us) ts_tablet_manager.cc:1145] Deleting tablet 3dd4ec30acde4585a1c3451927217cb3
0811 18:57:34.300805 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.300942 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2514936}
I20260811 18:57:34.317293 5443 raft_consensus.cc:482] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.317451 5443 raft_consensus.cc:504] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.318033 5443 leader_election.cc:288] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.321688 4167 ts_tablet_manager.cc:1918] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.323261 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45aff3e8cb0c4f7191e4522c6ef7d156" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.326774 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 859) took 2511 ms. Trace:
I20260811 18:57:34.326905 4169 rpcz_store.cc:276] 0811 18:57:31.815117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.815186 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:34.311743 (+2496557us) ts_tablet_manager.cc:1145] Deleting tablet f2f31331974f42c3a22b3ac21b690f82
0811 18:57:34.326601 (+ 14858us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.326753 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2496216}
I20260811 18:57:34.327245 4169 tablet_replica.cc:330] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.327521 3546 master_service.cc:946] ReplaceTablet: received request to replace tablet 0bfb54175c834517b7af291c942333cd from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.327525 4169 raft_consensus.cc:2229] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.327709 4169 raft_consensus.cc:2258] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.328406 4169 ts_tablet_manager.cc:1905] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.328455 4167 log.cc:1196] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:34.328827 4167 ts_tablet_manager.cc:1939] T 9bc5448a555944dfba8ccae09e25cbe9 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.327462 3676 leader_election.cc:341] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.328320 4162 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.330801 4162 ts_tablet_manager.cc:1397] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20260811 18:57:34.332428 4163 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.332319 4162 raft_consensus.cc:348] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.332618 4162 raft_consensus.cc:374] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.332705 4162 raft_consensus.cc:729] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.332945 4162 consensus_queue.cc:260] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.333839 4162 ts_tablet_manager.cc:1428] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:34.334574 4163 tablet_bootstrap.cc:654] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.336319 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 40898ceb18cc4f2d9104818798049ae2
I20260811 18:57:34.336835 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 40898ceb18cc4f2d9104818798049ae2 deleted and replaced by tablet a62cdd333ad8494897ae8ec5afd5af74
I20260811 18:57:34.338276 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 832) took 3646 ms. Trace:
I20260811 18:57:34.338490 5443 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.338610 5443 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.339187 5443 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.338410 4167 rpcz_store.cc:276] 0811 18:57:30.691456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.691515 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:34.280978 (+3589463us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:34.280992 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.292069 (+ 11077us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.321584 (+ 29515us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.338252 (+ 16668us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3586057}
I20260811 18:57:34.343889 4162 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.346022 4162 tablet_bootstrap.cc:654] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.336017 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet cbf6ce167fb348be8d61372791f0cbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336203 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f3060822021431584a494e6941be756 with delete_type TABLET_DATA_DELETED (replaced by tablet 7abca85e5a6d4b55a38c0f9e50f02f76) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336364 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet dd80633f926942e88f54b32d9a03ea3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:33 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336436 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c07150e31994bdf9a00ac2e7a954691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:14 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336620 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f0cdcee681460084662e01152ad9dd with delete_type TABLET_DATA_DELETED (replaced by tablet 9e5ca37dd77c42a19146a99452ff87b0) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.336773 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 3263da75f0d84528b7c8cc55dcaeaa46 with delete_type TABLET_DATA_DELETED (replaced by tablet 4dd203b7eb8643f493b4642c0396e258) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.346884 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 856) took 3276 ms. Trace:
W20260811 18:57:34.349355 3678 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.354120 3678 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.354023 4167 rpcz_store.cc:276] 0811 18:57:31.070670 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.070817 (+ 147us) service_pool.cc:227] Handling call
0811 18:57:34.346706 (+3275889us) ts_tablet_manager.cc:1145] Deleting tablet 22abc6f8e9ef4c32ab69af31bdaad872
0811 18:57:34.346717 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.346866 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3263483}
I20260811 18:57:34.355110 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 858) took 3276 ms. Trace:
I20260811 18:57:34.355333 5443 raft_consensus.cc:2726] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.356094 4167 rpcz_store.cc:276] 0811 18:57:31.078794 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.083842 (+ 5048us) service_pool.cc:227] Handling call
0811 18:57:34.354974 (+3271132us) ts_tablet_manager.cc:1145] Deleting tablet 3030c31bc2b24ee296f29b5d5aa402ff
0811 18:57:34.354986 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.355092 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3270763}
I20260811 18:57:34.357604 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 859) took 3278 ms. Trace:
I20260811 18:57:34.357771 4167 rpcz_store.cc:276] 0811 18:57:31.079020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.084228 (+ 5208us) service_pool.cc:227] Handling call
0811 18:57:34.357476 (+3273248us) ts_tablet_manager.cc:1145] Deleting tablet 6009c9f6bbb0493ebe125120a1a47f3f
0811 18:57:34.357488 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.357585 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3272912}
W20260811 18:57:34.337523 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.343202 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a7051bc539468aafba30fbaf0669d4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.358493 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.343807 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1374a80d15444cb0faff339275aa8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:29 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:34.359127 3676 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.359606 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.359596 3678 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.359721 3678 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.360078 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 861) took 3280 ms. Trace:
I20260811 18:57:34.360574 4167 rpcz_store.cc:276] 0811 18:57:31.079550 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.095154 (+ 15604us) service_pool.cc:227] Handling call
0811 18:57:34.359963 (+3264809us) ts_tablet_manager.cc:1145] Deleting tablet 7d747f03ad234805a4ea4ef60840d426
0811 18:57:34.359974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.360060 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3264598}
W20260811 18:57:34.360998 3824 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.360939 5443 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.361292 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.361434 3824 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.361672 3824 leader_election.cc:334] T 4687b95728fc485fa0e842bc447eb83f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.361232 3678 leader_election.cc:334] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.362283 3824 leader_election.cc:334] T 867259e37fe44c33a35823ec2d896754 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.362294 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a25b1eddd4a64847b51ad3cdc0238ff0
I20260811 18:57:34.362624 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 860) took 3283 ms. Trace:
I20260811 18:57:34.362799 3550 catalog_manager.cc:6565] ReplaceTablet: tablet a25b1eddd4a64847b51ad3cdc0238ff0 deleted and replaced by tablet 8ed141f91ec548db87ac1e719256931b
I20260811 18:57:34.362778 4167 rpcz_store.cc:276] 0811 18:57:31.079280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.089414 (+ 10134us) service_pool.cc:227] Handling call
0811 18:57:34.362509 (+3273095us) ts_tablet_manager.cc:1145] Deleting tablet 90fde17971ea49aaad212a5836661b08
0811 18:57:34.362522 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.362606 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3224511}
W20260811 18:57:34.362881 3824 leader_election.cc:334] T b0b199f4c9e14b468cb26cbd61fc8586 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.363262 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 865) took 3283 ms. Trace:
I20260811 18:57:34.363343 4167 rpcz_store.cc:276] 0811 18:57:31.080281 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.167681 (+ 87400us) service_pool.cc:227] Handling call
0811 18:57:34.363148 (+3195467us) ts_tablet_manager.cc:1145] Deleting tablet c0f68d33e1b14144b34c17e3c7201408
0811 18:57:34.363160 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.363245 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3195196}
W20260811 18:57:34.363415 3824 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.363643 3824 leader_election.cc:334] T 02808605ee6c4634af47fc3fd57e9cd8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.364203 3824 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.364418 3824 leader_election.cc:334] T f335e50ae57547ed80b144129faaffbb P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.369009 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 866) took 3127 ms. Trace:
I20260811 18:57:34.368952 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0d7c25599688441595b4b8f6cfea4584"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.369201 4167 rpcz_store.cc:276] 0811 18:57:31.241338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.241428 (+ 90us) service_pool.cc:227] Handling call
0811 18:57:34.368850 (+3127422us) ts_tablet_manager.cc:1145] Deleting tablet 01a5fdab68694761b7493b42a0bf7c5b
0811 18:57:34.368865 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.368985 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3108715}
W20260811 18:57:34.369668 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d7c25599688441595b4b8f6cfea4584 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.369843 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 867) took 3109 ms. Trace:
I20260811 18:57:34.369935 4167 rpcz_store.cc:276] 0811 18:57:31.260726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.261417 (+ 691us) service_pool.cc:227] Handling call
0811 18:57:34.369641 (+3108224us) ts_tablet_manager.cc:1145] Deleting tablet c944d5cfd9b3405b8151abfc9f2fd998
0811 18:57:34.369667 (+ 26us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.369757 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3097869}
I20260811 18:57:34.370560 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 868) took 3099 ms. Trace:
I20260811 18:57:34.370673 4167 rpcz_store.cc:276] 0811 18:57:31.271161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.271802 (+ 641us) service_pool.cc:227] Handling call
0811 18:57:34.370444 (+3098642us) ts_tablet_manager.cc:1145] Deleting tablet 4e0747ef88574c68ae074732cb7095b9
0811 18:57:34.370456 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.370543 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3050892}
I20260811 18:57:34.375027 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 871) took 2998 ms. Trace:
I20260811 18:57:34.375590 4167 rpcz_store.cc:276] 0811 18:57:31.376414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.376470 (+ 56us) service_pool.cc:227] Handling call
0811 18:57:34.374867 (+2998397us) ts_tablet_manager.cc:1145] Deleting tablet 99e9474ef8814514832f0e48c2b3495f
0811 18:57:34.374881 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.375004 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2998173}
I20260811 18:57:34.376781 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 870) took 3000 ms. Trace:
I20260811 18:57:34.377156 4167 rpcz_store.cc:276] 0811 18:57:31.376288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.382547 (+ 6259us) service_pool.cc:227] Handling call
0811 18:57:34.376482 (+2993935us) ts_tablet_manager.cc:1145] Deleting tablet 761902f7366d43ffbde226a0ade2e188
0811 18:57:34.376495 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.376760 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2993604}
I20260811 18:57:34.378016 4167 tablet_replica.cc:330] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.378762 4167 raft_consensus.cc:2229] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.379140 4167 raft_consensus.cc:2258] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.380491 4167 ts_tablet_manager.cc:1905] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.381772 5443 raft_consensus.cc:482] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.381906 5443 raft_consensus.cc:504] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.382256 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_495f858f5eec496680f982bccd44415d" } new_table_name: "default.Table_8c4206dfbf6d4214980816f41a2c6db8" modify_external_catalogs: true
I20260811 18:57:34.382582 5443 leader_election.cc:288] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.383499 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7057b8a7bae249caaed304b5c66e6871" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.384275 3676 leader_election.cc:341] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.391660 4162 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.391912 4162 ts_tablet_manager.cc:1397] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.001s
I20260811 18:57:34.392750 4162 raft_consensus.cc:348] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.393060 4162 raft_consensus.cc:374] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.393172 4162 raft_consensus.cc:729] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.393479 4162 consensus_queue.cc:260] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.394448 4162 ts_tablet_manager.cc:1428] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:34.396966 4162 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.401391 4169 ts_tablet_manager.cc:1918] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.402714 4169 log.cc:1196] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c08436f33db043b1bc6395cd92102e59
I20260811 18:57:34.402802 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_53f67a53b6684eaeb56341d3d428f565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.403487 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53f67a53b6684eaeb56341d3d428f565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.405352 4169 ts_tablet_manager.cc:1939] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:34.407186 3824 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.407621 3824 leader_election.cc:334] T 490f34d7a403416fbab64e46973746de P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.408221 5443 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.408413 5443 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.409273 5443 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.409097 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 861) took 2587 ms. Trace:
W20260811 18:57:34.409698 3824 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.409830 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.409981 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.410117 3824 leader_election.cc:334] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.410410 3676 leader_election.cc:341] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.410530 3824 leader_election.cc:334] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.410662 3824 leader_election.cc:302] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:34.411140 3824 raft_consensus.cc:2670] T c08436f33db043b1bc6395cd92102e59 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 18:57:34.411852 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.410410 4162 tablet_bootstrap.cc:654] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.409683 4169 rpcz_store.cc:276] 0811 18:57:31.821554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.822979 (+ 1425us) service_pool.cc:227] Handling call
0811 18:57:34.327202 (+2504223us) ts_tablet_manager.cc:1145] Deleting tablet c08436f33db043b1bc6395cd92102e59
0811 18:57:34.327213 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.350872 (+ 23659us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.400773 (+ 49901us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.408915 (+ 8142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2504031}
I20260811 18:57:34.415861 4169 tablet_replica.cc:330] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.417552 4169 raft_consensus.cc:2229] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:34.417840 3824 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.423262 3824 leader_election.cc:334] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.423723 4169 raft_consensus.cc:2258] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:34.425438 3824 consensus_peers.cc:487] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.426586 3824 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.428589 3824 leader_election.cc:334] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.430476 3824 consensus_peers.cc:487] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:34.431437 3824 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.433507 3824 leader_election.cc:334] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.435842 3824 consensus_peers.cc:487] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.436697 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_58f2aeab56c6446184cfeddcb1134314"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.436929 3824 leader_election.cc:334] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.439080 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.440045 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.440685 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 40898ceb18cc4f2d9104818798049ae2 with delete_type TABLET_DATA_DELETED (replaced by tablet a62cdd333ad8494897ae8ec5afd5af74) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.440824 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ed141f91ec548db87ac1e719256931b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:34.444033 4162 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.444262 4162 ts_tablet_manager.cc:1397] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.001s
W20260811 18:57:34.448830 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10e441b5df3d40fd93cf2dafe33bc2f4 consensus state has no leader
I20260811 18:57:34.449596 4162 raft_consensus.cc:348] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.449839 4162 raft_consensus.cc:374] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.449888 4162 raft_consensus.cc:729] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:34.446403 3824 leader_election.cc:334] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.450052 4162 consensus_queue.cc:260] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.450596 4162 ts_tablet_manager.cc:1428] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
W20260811 18:57:34.450872 3824 consensus_peers.cc:487] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.452574 4167 ts_tablet_manager.cc:1918] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:34.452637 3824 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.453904 3824 leader_election.cc:334] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.454900 3824 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.455395 4167 log.cc:1196] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:34.455878 4167 ts_tablet_manager.cc:1939] T b3f0cdcee681460084662e01152ad9dd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.456147 3824 leader_election.cc:334] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.456807 4162 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.457216 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" } modify_external_catalogs: true
I20260811 18:57:34.457479 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3652770b3e134affa80ff22fdfc1a0a1" } modify_external_catalogs: true
W20260811 18:57:34.457296 3824 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.458379 3824 leader_election.cc:334] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.458902 4162 tablet_bootstrap.cc:654] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.459300 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 869) took 3088 ms. Trace:
W20260811 18:57:34.459661 3824 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.461787 4167 rpcz_store.cc:276] 0811 18:57:31.370640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.392165 (+ 21525us) service_pool.cc:227] Handling call
0811 18:57:34.377963 (+2985798us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:34.377974 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.390179 (+ 12205us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.452471 (+ 62292us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.459271 (+ 6800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2970142}
W20260811 18:57:34.461575 3824 consensus_peers.cc:487] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.462424 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 879) took 2712 ms. Trace:
W20260811 18:57:34.463096 3824 leader_election.cc:334] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.466526 4167 rpcz_store.cc:276] 0811 18:57:31.749510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.749550 (+ 40us) service_pool.cc:227] Handling call
0811 18:57:34.462246 (+2712696us) ts_tablet_manager.cc:1145] Deleting tablet 3dd4ec30acde4585a1c3451927217cb3
0811 18:57:34.462258 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.462401 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2712491}
W20260811 18:57:34.466800 3824 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.468807 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a3466b2364d64b0cad93b68732e0dae2 consensus state has no leader
I20260811 18:57:34.469206 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 876) took 2720 ms. Trace:
I20260811 18:57:34.469733 4167 rpcz_store.cc:276] 0811 18:57:31.749039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.757994 (+ 8955us) service_pool.cc:227] Handling call
0811 18:57:34.469006 (+2711012us) ts_tablet_manager.cc:1145] Deleting tablet 1eb30e64e86040098dcee5fc2e9b7266
0811 18:57:34.469026 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.469181 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2710685}
W20260811 18:57:34.469547 3824 leader_election.cc:334] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.470955 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 878) took 2721 ms. Trace:
I20260811 18:57:34.471062 4167 rpcz_store.cc:276] 0811 18:57:31.749382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.749477 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:34.470821 (+2721344us) ts_tablet_manager.cc:1145] Deleting tablet 35ebc32eb5d5443eaaeba245b566f98a
0811 18:57:34.470833 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.470936 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2711157}
I20260811 18:57:34.471382 4167 tablet_replica.cc:330] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.471623 4167 raft_consensus.cc:2229] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.471825 4167 raft_consensus.cc:2258] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.472313 4169 ts_tablet_manager.cc:1905] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.472421 4167 ts_tablet_manager.cc:1905] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.474647 4162 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.474845 4162 ts_tablet_manager.cc:1397] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260811 18:57:34.475790 4162 raft_consensus.cc:348] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.476261 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58f2aeab56c6446184cfeddcb1134314 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.476859 4162 raft_consensus.cc:374] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.477017 4162 raft_consensus.cc:729] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.477442 4162 consensus_queue.cc:260] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.482391 3824 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.485913 3678 leader_election.cc:302] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.485417 3824 leader_election.cc:334] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.487759 3824 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.489174 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" } modify_external_catalogs: true
I20260811 18:57:34.485733 4162 ts_tablet_manager.cc:1428] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260811 18:57:34.489475 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f7fd9b1d34c3439c997e0341237d5a82" } modify_external_catalogs: true
W20260811 18:57:34.489954 3824 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.492249 4162 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:34.491828 3824 leader_election.cc:334] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.493041 5443 raft_consensus.cc:2726] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.493080 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 131844us
W20260811 18:57:34.494077 3824 leader_election.cc:334] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.495708 3824 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.498972 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ebe3fed5aaf45918dbadfde06b2c24e consensus state has no leader
I20260811 18:57:34.499264 4162 tablet_bootstrap.cc:654] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:34.498728 3824 leader_election.cc:334] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.502767 4167 ts_tablet_manager.cc:1918] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.505183 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_061ae2e2037348cab0a2499a98d273a6" } modify_external_catalogs: true
I20260811 18:57:34.505563 4167 log.cc:1196] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:34.505496 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_061ae2e2037348cab0a2499a98d273a6" } modify_external_catalogs: true
I20260811 18:57:34.506029 4167 ts_tablet_manager.cc:1939] T 8c07150e31994bdf9a00ac2e7a954691 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.507001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 889) took 2687 ms. Trace:
I20260811 18:57:34.510268 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c26851245cab4a2d80e94d4fe278fd50 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
W20260811 18:57:34.510720 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 98ad733726594d2a8087f857f155dbe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98ad733726594d2a8087f857f155dbe2 already in progress: creating tablet
W20260811 18:57:34.511142 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 85140a3800f34a23b716703fd4e73d17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85140a3800f34a23b716703fd4e73d17 already in progress: creating tablet
W20260811 18:57:34.511560 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 22abc6f8e9ef4c32ab69af31bdaad872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22abc6f8e9ef4c32ab69af31bdaad872 already in progress: creating tablet
W20260811 18:57:34.511830 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3030c31bc2b24ee296f29b5d5aa402ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3030c31bc2b24ee296f29b5d5aa402ff already in progress: creating tablet
W20260811 18:57:34.512115 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6009c9f6bbb0493ebe125120a1a47f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6009c9f6bbb0493ebe125120a1a47f3f already in progress: creating tablet
W20260811 18:57:34.512362 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7d747f03ad234805a4ea4ef60840d426 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d747f03ad234805a4ea4ef60840d426 already in progress: creating tablet
I20260811 18:57:34.516027 4163 tablet_bootstrap.cc:492] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.515317 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf22cbb415c485ebc1407d9104b715d" } modify_external_catalogs: true
I20260811 18:57:34.516245 4163 ts_tablet_manager.cc:1397] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.184s user 0.020s sys 0.032s
I20260811 18:57:34.516309 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ddf22cbb415c485ebc1407d9104b715d" } modify_external_catalogs: true
W20260811 18:57:34.517040 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 90fde17971ea49aaad212a5836661b08 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90fde17971ea49aaad212a5836661b08 already in progress: creating tablet
W20260811 18:57:34.517333 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c0f68d33e1b14144b34c17e3c7201408 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f68d33e1b14144b34c17e3c7201408 already in progress: creating tablet
W20260811 18:57:34.517607 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 01a5fdab68694761b7493b42a0bf7c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01a5fdab68694761b7493b42a0bf7c5b already in progress: creating tablet
W20260811 18:57:34.517887 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c944d5cfd9b3405b8151abfc9f2fd998 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c944d5cfd9b3405b8151abfc9f2fd998 already in progress: creating tablet
W20260811 18:57:34.518169 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4e0747ef88574c68ae074732cb7095b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e0747ef88574c68ae074732cb7095b9 already in progress: creating tablet
W20260811 18:57:34.518401 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 531ba05901734d6b93bc2be506e8ec5b consensus state has no leader
W20260811 18:57:34.518590 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 99e9474ef8814514832f0e48c2b3495f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99e9474ef8814514832f0e48c2b3495f already in progress: creating tablet
W20260811 18:57:34.519114 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 761902f7366d43ffbde226a0ade2e188 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761902f7366d43ffbde226a0ade2e188 already in progress: creating tablet
I20260811 18:57:34.519024 4163 raft_consensus.cc:348] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.519522 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b3f0cdcee681460084662e01152ad9dd (table default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]) successfully deleted
W20260811 18:57:34.519848 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3dd4ec30acde4585a1c3451927217cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd4ec30acde4585a1c3451927217cb3 already in progress: creating tablet
W20260811 18:57:34.520323 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1eb30e64e86040098dcee5fc2e9b7266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1eb30e64e86040098dcee5fc2e9b7266 already in progress: creating tablet
W20260811 18:57:34.520617 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 35ebc32eb5d5443eaaeba245b566f98a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35ebc32eb5d5443eaaeba245b566f98a already in progress: creating tablet
I20260811 18:57:34.522102 4163 raft_consensus.cc:374] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.522254 4163 raft_consensus.cc:729] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.522558 4163 consensus_queue.cc:260] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.525287 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67" } modify_external_catalogs: true
I20260811 18:57:34.525521 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5054093f72194ee8a604b8aa7fb6cf67" } modify_external_catalogs: true
I20260811 18:57:34.526262 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_acd269829eb743178d173c6cc3d7301e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.526970 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acd269829eb743178d173c6cc3d7301e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.508095 3678 leader_election.cc:334] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.529417 3678 leader_election.cc:302] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.529376 4167 rpcz_store.cc:276] 0811 18:57:31.819978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823490 (+ 3512us) service_pool.cc:227] Handling call
0811 18:57:34.471330 (+2647840us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:34.471344 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.488050 (+ 16706us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.502653 (+ 14603us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.506976 (+ 4323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2647646}
W20260811 18:57:34.529803 3678 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.529882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 887) took 2710 ms. Trace:
I20260811 18:57:34.529912 3678 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.529958 4167 rpcz_store.cc:276] 0811 18:57:31.819714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823753 (+ 4039us) service_pool.cc:227] Handling call
0811 18:57:34.529737 (+2705984us) ts_tablet_manager.cc:1145] Deleting tablet f1800e9e2b064a6fb546b9d33cefa68b
0811 18:57:34.529751 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.529864 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705831}
I20260811 18:57:34.530309 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 888) took 2710 ms. Trace:
I20260811 18:57:34.530436 5407 raft_consensus.cc:2726] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.530414 4167 rpcz_store.cc:276] 0811 18:57:31.819819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824166 (+ 4347us) service_pool.cc:227] Handling call
0811 18:57:34.530191 (+2706025us) ts_tablet_manager.cc:1145] Deleting tablet f2f31331974f42c3a22b3ac21b690f82
0811 18:57:34.530203 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.530292 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705834}
I20260811 18:57:34.530603 5407 raft_consensus.cc:2726] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.530740 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 886) took 2711 ms. Trace:
I20260811 18:57:34.530827 4167 rpcz_store.cc:276] 0811 18:57:31.819628 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824570 (+ 4942us) service_pool.cc:227] Handling call
0811 18:57:34.530632 (+2706062us) ts_tablet_manager.cc:1145] Deleting tablet b702aef88e124443888002d8ccd47b2f
0811 18:57:34.530643 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.530723 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2705895}
I20260811 18:57:34.534202 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c4f7db8f684b436caef8f1315eadb0ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.534934 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4f7db8f684b436caef8f1315eadb0ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.533785 5443 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.539669 5443 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.540400 5443 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.541050 4163 ts_tablet_manager.cc:1428] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.001s sys 0.001s
I20260811 18:57:34.539463 4162 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.541473 4162 ts_tablet_manager.cc:1397] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
W20260811 18:57:34.542228 3678 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.542470 4162 raft_consensus.cc:348] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.542698 4162 raft_consensus.cc:374] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.542774 4162 raft_consensus.cc:729] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.543053 4162 consensus_queue.cc:260] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.541431 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00d943b778741b4aeec076ba46a9c40" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.545157 3676 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.545284 3676 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.546043 5407 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.549479 4162 ts_tablet_manager.cc:1428] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260811 18:57:34.555362 4162 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.559032 4167 tablet_replica.cc:330] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.559584 4167 raft_consensus.cc:2229] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.559864 4167 raft_consensus.cc:2258] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.560561 4167 ts_tablet_manager.cc:1905] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.563480 4163 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.563735 4162 tablet_bootstrap.cc:654] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.565179 4163 tablet_bootstrap.cc:654] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.571216 5443 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.571386 5443 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.572101 5443 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.573276 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b325df1db4fde9d4b26b559121316" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.574254 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_50769590dc5544e0a551369ba2c9768c" } new_table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" modify_external_catalogs: true
W20260811 18:57:34.574797 3676 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.575471 4167 ts_tablet_manager.cc:1918] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.575610 4167 log.cc:1196] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:34.576011 4167 ts_tablet_manager.cc:1939] T f7d4c26da86b49b8a37f2289f4ef6614 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.574249 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e75355b4dffb4b4992e47c68f868d172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.576890 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 904) took 2460 ms. Trace:
W20260811 18:57:34.577181 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e75355b4dffb4b4992e47c68f868d172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.586040 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 374c94a12e374d0e82180421c80d92d0 consensus state has no leader
I20260811 18:57:34.586515 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet afa57ed239ac4afebe517cc48d879645 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.582391 4162 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:34.586704 3678 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.586789 4162 ts_tablet_manager.cc:1397] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20260811 18:57:34.586843 3678 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.587477 5443 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.587605 4162 raft_consensus.cc:348] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.587930 4162 raft_consensus.cc:374] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.588006 4162 raft_consensus.cc:729] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.588230 4162 consensus_queue.cc:260] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.588847 4162 ts_tablet_manager.cc:1428] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:34.588944 4167 rpcz_store.cc:276] 0811 18:57:32.116369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.116816 (+ 447us) service_pool.cc:227] Handling call
0811 18:57:34.558966 (+2442150us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:34.558982 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.572072 (+ 13090us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.575363 (+ 3291us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.576866 (+ 1503us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2441915}
I20260811 18:57:34.594655 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 968fa8b8912b40a8bdba77db1d39c5c0 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.596254 4162 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.598510 4169 ts_tablet_manager.cc:1918] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.598773 4169 log.cc:1196] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fbba1923ce0049448fe4e28f2dd258dc
I20260811 18:57:34.599339 4169 ts_tablet_manager.cc:1939] T fbba1923ce0049448fe4e28f2dd258dc P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.599807 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 860) took 2778 ms. Trace:
I20260811 18:57:34.603310 5443 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.603472 5443 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.604125 5443 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.606609 5443 raft_consensus.cc:482] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.606758 5443 raft_consensus.cc:504] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.607322 5443 leader_election.cc:288] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:34.607967 3678 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.609200 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_aa3debf8d8774103a45a754a6f08465a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.601878 4167 tablet_replica.cc:330] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.610028 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa3debf8d8774103a45a754a6f08465a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.610139 4167 raft_consensus.cc:2229] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.610582 4167 raft_consensus.cc:2258] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:34.610373 3678 leader_election.cc:334] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.611495 4167 ts_tablet_manager.cc:1905] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.615950 4162 tablet_bootstrap.cc:654] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.622766 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8c07150e31994bdf9a00ac2e7a954691 (table default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]) successfully deleted
W20260811 18:57:34.623220 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1800e9e2b064a6fb546b9d33cefa68b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1800e9e2b064a6fb546b9d33cefa68b already in progress: creating tablet
W20260811 18:57:34.623593 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f2f31331974f42c3a22b3ac21b690f82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f31331974f42c3a22b3ac21b690f82 already in progress: creating tablet
I20260811 18:57:34.623884 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:34.624074 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b702aef88e124443888002d8ccd47b2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b702aef88e124443888002d8ccd47b2f already in progress: creating tablet
I20260811 18:57:34.624485 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f7d4c26da86b49b8a37f2289f4ef6614 (table default.Table_523277d801974e708b0f7e9c401bb36a [id=5600dcaaf83c4f978c3d47f79e01568c]) successfully deleted
I20260811 18:57:34.627728 4167 ts_tablet_manager.cc:1918] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.636221 4167 log.cc:1196] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:34.636984 4167 ts_tablet_manager.cc:1939] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.638154 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_97e3f5df19fc4b04a37f62286b42f7e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.638976 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97e3f5df19fc4b04a37f62286b42f7e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.644762 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 903) took 2530 ms. Trace:
I20260811 18:57:34.644954 4167 rpcz_store.cc:276] 0811 18:57:32.114402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.121504 (+ 7102us) service_pool.cc:227] Handling call
0811 18:57:34.589222 (+2467718us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:34.601819 (+ 12597us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.623870 (+ 22051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.627623 (+ 3753us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.644733 (+ 17110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2463676}
I20260811 18:57:34.645664 4162 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.645823 4162 ts_tablet_manager.cc:1397] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.003s
I20260811 18:57:34.645943 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 905) took 2464 ms. Trace:
I20260811 18:57:34.646414 4167 rpcz_store.cc:276] 0811 18:57:32.181526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181613 (+ 87us) service_pool.cc:227] Handling call
0811 18:57:34.645606 (+2463993us) ts_tablet_manager.cc:1145] Deleting tablet ccc63b46a4e346f79d1e03a82490794c
0811 18:57:34.645617 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.645924 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2463737}
I20260811 18:57:34.646695 4162 raft_consensus.cc:348] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.646987 4162 raft_consensus.cc:374] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.647089 4162 raft_consensus.cc:729] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.647312 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 906) took 2465 ms. Trace:
I20260811 18:57:34.648411 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a5361d65b50b445f9d85f5ddda046f99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.648900 4167 rpcz_store.cc:276] 0811 18:57:32.181678 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181922 (+ 244us) service_pool.cc:227] Handling call
0811 18:57:34.646934 (+2465012us) ts_tablet_manager.cc:1145] Deleting tablet cdb92902c2584751af77d3117f52cb4e
0811 18:57:34.646947 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.647295 (+ 348us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2464835}
W20260811 18:57:34.649176 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5361d65b50b445f9d85f5ddda046f99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.647341 4162 consensus_queue.cc:260] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.652415 4162 ts_tablet_manager.cc:1428] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:57:34.652783 4162 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.654590 4162 tablet_bootstrap.cc:654] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.604847 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.608057 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cccebfc54484a9b9abffcce4e459fb5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:34.658990 5417 raft_consensus.cc:482] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.661927 5417 raft_consensus.cc:504] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:34.662132 3676 leader_election.cc:341] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.662271 3676 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.662722 5417 leader_election.cc:288] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:34.662717 3676 leader_election.cc:341] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.662834 3676 leader_election.cc:302] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.662907 4169 rpcz_store.cc:276] 0811 18:57:31.821447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.823296 (+ 1849us) service_pool.cc:227] Handling call
0811 18:57:34.415573 (+2592277us) ts_tablet_manager.cc:1145] Deleting tablet fbba1923ce0049448fe4e28f2dd258dc
0811 18:57:34.415589 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.523072 (+107483us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.598333 (+ 75261us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.599784 (+ 1451us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2591888}
I20260811 18:57:34.663160 5443 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.663370 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 857) took 2861 ms. Trace:
I20260811 18:57:34.663282 5443 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.663457 4169 rpcz_store.cc:276] 0811 18:57:31.801530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.821842 (+ 20312us) service_pool.cc:227] Handling call
0811 18:57:34.663207 (+2841365us) ts_tablet_manager.cc:1145] Deleting tablet b702aef88e124443888002d8ccd47b2f
0811 18:57:34.663225 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.663349 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839188}
I20260811 18:57:34.663868 5443 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.663908 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 858) took 2849 ms. Trace:
I20260811 18:57:34.664012 5443 raft_consensus.cc:2726] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.663988 4169 rpcz_store.cc:276] 0811 18:57:31.814960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.824460 (+ 9500us) service_pool.cc:227] Handling call
0811 18:57:34.663769 (+2839309us) ts_tablet_manager.cc:1145] Deleting tablet f1800e9e2b064a6fb546b9d33cefa68b
0811 18:57:34.663782 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.663892 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2839148}
I20260811 18:57:34.664162 5443 raft_consensus.cc:2726] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.664309 4169 tablet_replica.cc:330] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.664513 4169 raft_consensus.cc:2229] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.665253 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 907) took 2468 ms. Trace:
W20260811 18:57:34.667845 3678 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.669201 4167 rpcz_store.cc:276] 0811 18:57:32.197315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.197413 (+ 98us) service_pool.cc:227] Handling call
0811 18:57:34.665113 (+2467700us) ts_tablet_manager.cc:1145] Deleting tablet 99de8ce45e514ff291cb9e5c0a4c00d1
0811 18:57:34.665125 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.665232 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2467432}
I20260811 18:57:34.670622 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f335e50ae57547ed80b144129faaffbb (table default.Table_3a2c107af5134adbb9cdfb4b0981da6b [id=05f568b4091f4c238d85df0fa7ed90a4]) successfully deleted
W20260811 18:57:34.671031 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 67e63ee9b9fd4b12ae791b9d5bc2e648 because the tablet was not found. No further retry: Not found: Tablet not found: 67e63ee9b9fd4b12ae791b9d5bc2e648
W20260811 18:57:34.671773 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 99e9474ef8814514832f0e48c2b3495f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99e9474ef8814514832f0e48c2b3495f already in progress: creating tablet
I20260811 18:57:34.672556 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_485cdcb897e6430ba77e994afbd44a64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.669940 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:34.673319 3743 raft_consensus.cc:2454] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:34.673461 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_485cdcb897e6430ba77e994afbd44a64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:34.676028 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 761902f7366d43ffbde226a0ade2e188 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761902f7366d43ffbde226a0ade2e188 already in progress: creating tablet
W20260811 18:57:34.676690 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 35ebc32eb5d5443eaaeba245b566f98a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35ebc32eb5d5443eaaeba245b566f98a already in progress: creating tablet
W20260811 18:57:34.677286 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1eb30e64e86040098dcee5fc2e9b7266 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1eb30e64e86040098dcee5fc2e9b7266 already in progress: creating tablet
W20260811 18:57:34.677759 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3dd4ec30acde4585a1c3451927217cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3dd4ec30acde4585a1c3451927217cb3 already in progress: creating tablet
I20260811 18:57:34.671820 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 908) took 2474 ms. Trace:
W20260811 18:57:34.678928 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f2f31331974f42c3a22b3ac21b690f82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f31331974f42c3a22b3ac21b690f82 already in progress: creating tablet
I20260811 18:57:34.679379 4167 rpcz_store.cc:276] 0811 18:57:32.197483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.197732 (+ 249us) service_pool.cc:227] Handling call
0811 18:57:34.670630 (+2472898us) ts_tablet_manager.cc:1145] Deleting tablet 14799a6f61a946008acee8a09d050c1e
0811 18:57:34.670643 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.671797 (+ 1154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2472750}
I20260811 18:57:34.680956 4167 tablet_replica.cc:330] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.682240 4167 raft_consensus.cc:2229] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.684160 4167 raft_consensus.cc:2258] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.686313 4167 ts_tablet_manager.cc:1905] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.690299 5490 raft_consensus.cc:482] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.692790 5490 raft_consensus.cc:504] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:34.695304 3678 leader_election.cc:334] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.695366 5490 leader_election.cc:288] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.696635 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0bfb54175c834517b7af291c942333cd
I20260811 18:57:34.697198 3546 catalog_manager.cc:6565] ReplaceTablet: tablet 0bfb54175c834517b7af291c942333cd deleted and replaced by tablet 13af4d419ae542cc961c75f28efd0204
I20260811 18:57:34.698865 4169 pending_rounds.cc:62] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.698949 4169 pending_rounds.cc:68] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400173454458880 op_type: NO_OP noop_request { }
I20260811 18:57:34.699209 4169 raft_consensus.cc:2866] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:34.699370 4169 raft_consensus.cc:2258] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.680259 4163 tablet_bootstrap.cc:492] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.683394 5417 consensus_queue.cc:1035] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:34.683650 3824 leader_election.cc:341] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:34.685091 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc3f120758c6405dafd2ea0bf54c81d8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.700596 3824 consensus_peers.cc:487] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 122b7898fea44ef78f269b1505d8ddc4. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.695492 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce94d37efd2245bf8cc71fdd90bd9735" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.701005 3676 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.701125 3676 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.701030 3824 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:34.701352 5490 raft_consensus.cc:2726] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.701864 3676 leader_election.cc:341] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.701973 3676 leader_election.cc:302] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.702311 5490 raft_consensus.cc:2726] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.704167 4162 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.705318 4162 ts_tablet_manager.cc:1397] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.015s
I20260811 18:57:34.706037 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e03268eb26a144ba8b53cbd960628036" } new_table_name: "default.Table_5cf0c22ada454c098b23f5915efe0380" modify_external_catalogs: true
I20260811 18:57:34.712174 4162 raft_consensus.cc:348] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.714032 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1c1896700e724204a37439f134a4d988
I20260811 18:57:34.714243 4162 raft_consensus.cc:374] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.715088 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7de254c88a674097b5df7ea987172a9e" } new_table_name: "default.Table_7f65859fb42941e3a8375f65182f9ec4" modify_external_catalogs: true
I20260811 18:57:34.714994 4162 raft_consensus.cc:729] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.716747 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4dd203b7eb8643f493b4642c0396e258
I20260811 18:57:34.717469 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6953ae59bab4944a8db87e863735f79" } new_table_name: "default.Table_76b86d4f079c45b1a7f34a195fc97dfb" modify_external_catalogs: true
I20260811 18:57:34.717042 4162 consensus_queue.cc:260] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.724790 4162 ts_tablet_manager.cc:1428] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.019s user 0.002s sys 0.001s
I20260811 18:57:34.726403 5417 raft_consensus.cc:2781] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:34.726526 5417 raft_consensus.cc:482] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:34.726595 5417 raft_consensus.cc:3037] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.729628 5417 raft_consensus.cc:504] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.730939 5417 leader_election.cc:288] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.733464 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet afa57ed239ac4afebe517cc48d879645
I20260811 18:57:34.734031 3551 catalog_manager.cc:6565] ReplaceTablet: tablet afa57ed239ac4afebe517cc48d879645 deleted and replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5
I20260811 18:57:34.735067 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b27f9dbb52f34687a33c2ebf20e8f780
I20260811 18:57:34.738492 4167 ts_tablet_manager.cc:1918] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.738652 4167 log.cc:1196] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:34.738991 4167 ts_tablet_manager.cc:1939] T 55f97f37fa7c483cacaeed54f93e7a24 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:34.740938 3824 leader_election.cc:341] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 43ad6993b63e4bbd98c8cc52f1a53451
W20260811 18:57:34.741910 3824 consensus_peers.cc:487] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c26851245cab4a2d80e94d4fe278fd50. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:34.741353 4163 ts_tablet_manager.cc:1397] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.178s user 0.002s sys 0.001s
I20260811 18:57:34.743551 5376 consensus_queue.cc:1035] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:34.744426 4163 raft_consensus.cc:348] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.746709 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 909) took 2548 ms. Trace:
I20260811 18:57:34.746795 4163 raft_consensus.cc:374] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.746906 4167 rpcz_store.cc:276] 0811 18:57:32.198466 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.198523 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:34.680901 (+2482378us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:34.680914 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.735089 (+ 54175us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.738372 (+ 3283us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.746681 (+ 8309us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":226,"tablet-delete.queue_time_us":2482168}
I20260811 18:57:34.747690 4167 tablet_replica.cc:330] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.747639 4163 raft_consensus.cc:729] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.748039 4167 raft_consensus.cc:2229] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.747936 4163 consensus_queue.cc:260] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.700099 4169 ts_tablet_manager.cc:1905] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.748364 4167 raft_consensus.cc:2258] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.749012 4167 ts_tablet_manager.cc:1905] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.745266 3824 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:34.752547 4162 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.754719 5490 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.754878 5490 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.755577 5490 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.756821 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.757468 3676 leader_election.cc:341] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:34.757691 3678 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.757786 3678 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.757988 5376 raft_consensus.cc:2781] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:34.758095 5376 raft_consensus.cc:482] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:34.758172 5376 raft_consensus.cc:3037] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.758419 5490 raft_consensus.cc:2726] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.761323 5376 raft_consensus.cc:504] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.762319 5376 leader_election.cc:288] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.763672 4167 ts_tablet_manager.cc:1918] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.763873 4167 log.cc:1196] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/58c515c153df4317b3fb427983ee15e0
I20260811 18:57:34.764267 4167 ts_tablet_manager.cc:1939] T 58c515c153df4317b3fb427983ee15e0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.765214 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 910) took 2487 ms. Trace:
I20260811 18:57:34.765362 4167 rpcz_store.cc:276] 0811 18:57:32.277478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.278560 (+ 1082us) service_pool.cc:227] Handling call
0811 18:57:34.747633 (+2469073us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:34.747648 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.752462 (+ 4814us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.763550 (+ 11088us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.765186 (+ 1636us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2457641}
I20260811 18:57:34.766060 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 911) took 2488 ms. Trace:
I20260811 18:57:34.766314 4167 rpcz_store.cc:276] 0811 18:57:32.277633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.278678 (+ 1045us) service_pool.cc:227] Handling call
0811 18:57:34.765920 (+2487242us) ts_tablet_manager.cc:1145] Deleting tablet e1ad3860d03f46baadd5ede854f1a909
0811 18:57:34.765933 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.766043 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2475833}
I20260811 18:57:34.766819 4167 tablet_replica.cc:330] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.767089 4167 raft_consensus.cc:2229] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.767459 4167 raft_consensus.cc:2258] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.768321 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c08436f33db043b1bc6395cd92102e59 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
I20260811 18:57:34.768797 4167 ts_tablet_manager.cc:1905] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.776235 4167 ts_tablet_manager.cc:1918] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.776419 4167 log.cc:1196] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:34.776760 4167 ts_tablet_manager.cc:1939] T 6432abd4fc5c4f9ca95971ca4795e1cf P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.777633 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 912) took 2215 ms. Trace:
I20260811 18:57:34.777766 4167 rpcz_store.cc:276] 0811 18:57:32.562219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.562289 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:34.766769 (+2204480us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:34.766781 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.772572 (+ 5791us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.776126 (+ 3554us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.777610 (+ 1484us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2204204}
I20260811 18:57:34.778586 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 921) took 2115 ms. Trace:
I20260811 18:57:34.778702 4167 rpcz_store.cc:276] 0811 18:57:32.662850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.666422 (+ 3572us) service_pool.cc:227] Handling call
0811 18:57:34.778295 (+2111873us) ts_tablet_manager.cc:1145] Deleting tablet 0a5b06fa572045f3a5fc5a0b123d77cc
0811 18:57:34.778308 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.778567 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2111567}
I20260811 18:57:34.779222 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 918) took 2121 ms. Trace:
I20260811 18:57:34.779373 4167 rpcz_store.cc:276] 0811 18:57:32.658006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.668878 (+ 10872us) service_pool.cc:227] Handling call
0811 18:57:34.779105 (+2110227us) ts_tablet_manager.cc:1145] Deleting tablet 844bcd16f06c4788af9c0d7d320464aa
0811 18:57:34.779116 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.779206 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2109944}
I20260811 18:57:34.779844 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 932) took 1909 ms. Trace:
I20260811 18:57:34.779945 4167 rpcz_store.cc:276] 0811 18:57:32.870869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.874715 (+ 3846us) service_pool.cc:227] Handling call
0811 18:57:34.779746 (+1905031us) ts_tablet_manager.cc:1145] Deleting tablet e7d75d8468fc4c938fde0b2951d84562
0811 18:57:34.779757 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.779830 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1904668}
I20260811 18:57:34.780367 4167 tablet_replica.cc:330] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.780628 4167 raft_consensus.cc:2229] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.780815 4167 raft_consensus.cc:2258] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.781430 4167 ts_tablet_manager.cc:1905] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.782358 4163 ts_tablet_manager.cc:1428] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.040s user 0.002s sys 0.000s
W20260811 18:57:34.783121 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b702aef88e124443888002d8ccd47b2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b702aef88e124443888002d8ccd47b2f already in progress: creating tablet
W20260811 18:57:34.783495 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1800e9e2b064a6fb546b9d33cefa68b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1800e9e2b064a6fb546b9d33cefa68b already in progress: creating tablet
W20260811 18:57:34.784664 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 91994.7us
I20260811 18:57:34.792405 4167 ts_tablet_manager.cc:1918] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.792577 4167 log.cc:1196] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7251204325dd424192a6f58e18e71c34
I20260811 18:57:34.792924 4167 ts_tablet_manager.cc:1939] T 7251204325dd424192a6f58e18e71c34 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.793624 4163 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.793912 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 941) took 1676 ms. Trace:
I20260811 18:57:34.794032 4167 rpcz_store.cc:276] 0811 18:57:33.117658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.117733 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:34.780322 (+1662589us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:34.780332 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.785812 (+ 5480us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.792275 (+ 6463us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.793889 (+ 1614us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1662309}
I20260811 18:57:34.794579 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 942) took 1647 ms. Trace:
I20260811 18:57:34.794723 4167 rpcz_store.cc:276] 0811 18:57:33.146654 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.146722 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:34.794434 (+1647712us) ts_tablet_manager.cc:1145] Deleting tablet 6df5545532e346f7abcdc4cd228b84da
0811 18:57:34.794449 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.794558 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1647470}
I20260811 18:57:34.795033 4167 tablet_replica.cc:330] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.795255 4167 raft_consensus.cc:2229] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.795423 4167 raft_consensus.cc:2258] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.796036 4167 ts_tablet_manager.cc:1905] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:34.795830 3824 leader_election.cc:334] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.798460 3824 leader_election.cc:302] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:34.802798 4167 ts_tablet_manager.cc:1918] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.802989 4167 log.cc:1196] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:34.803040 4162 tablet_bootstrap.cc:654] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.804008 4163 tablet_bootstrap.cc:654] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.806311 5490 raft_consensus.cc:482] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.806495 5490 raft_consensus.cc:504] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.806668 4169 ts_tablet_manager.cc:1918] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.806784 4169 log.cc:1196] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a902ad70a25546be926ff8f7bf2ed64f
I20260811 18:57:34.807186 4169 ts_tablet_manager.cc:1939] T a902ad70a25546be926ff8f7bf2ed64f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.807232 5490 leader_election.cc:288] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.809407 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d32bb4eca047e4ba786c04b8bebf8b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.812016 3676 leader_election.cc:341] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.803333 4167 ts_tablet_manager.cc:1939] T b2718cd4a98742dfa8014a8a38c69c45 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.816901 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 943) took 1614 ms. Trace:
I20260811 18:57:34.822567 4167 rpcz_store.cc:276] 0811 18:57:33.202791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.202864 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:34.794984 (+1592120us) ts_tablet_manager.cc:1145] Deleting tablet b2718cd4a98742dfa8014a8a38c69c45
0811 18:57:34.794994 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.799653 (+ 4659us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.802702 (+ 3049us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.816876 (+ 14174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591905}
W20260811 18:57:34.814810 3678 leader_election.cc:334] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.823192 3678 leader_election.cc:302] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.823314 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 944) took 1572 ms. Trace:
I20260811 18:57:34.823414 4167 rpcz_store.cc:276] 0811 18:57:33.251387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.251467 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:34.823160 (+1571693us) ts_tablet_manager.cc:1145] Deleting tablet 4af28c051e8243eaa73d4a5ed1abcea6
0811 18:57:34.823175 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.823295 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1571384}
I20260811 18:57:34.823702 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 945) took 1449 ms. Trace:
I20260811 18:57:34.823776 4167 rpcz_store.cc:276] 0811 18:57:33.374541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.374626 (+ 85us) service_pool.cc:227] Handling call
0811 18:57:34.823618 (+1448992us) ts_tablet_manager.cc:1145] Deleting tablet 9608d6ddcad749cd9e72dd6f265db0da
0811 18:57:34.823627 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.823690 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1448755}
I20260811 18:57:34.823933 5490 raft_consensus.cc:2726] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.823993 4167 tablet_replica.cc:330] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.815939 4162 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.824397 4162 ts_tablet_manager.cc:1397] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.073s user 0.006s sys 0.000s
I20260811 18:57:34.825287 4162 raft_consensus.cc:348] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.825727 4162 raft_consensus.cc:374] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.825991 4162 raft_consensus.cc:729] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.826259 4162 consensus_queue.cc:260] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.828876 4167 raft_consensus.cc:2229] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.829051 4167 raft_consensus.cc:2258] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.829595 4167 ts_tablet_manager.cc:1905] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.831198 4162 ts_tablet_manager.cc:1428] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:57:34.831491 5490 raft_consensus.cc:482] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.831653 5490 raft_consensus.cc:504] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.832238 5490 leader_election.cc:288] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:34.833318 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c8d542ae2489ba466ffadd518db12" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.833981 3676 leader_election.cc:341] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.848726 5376 raft_consensus.cc:2726] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.849236 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 862) took 3027 ms. Trace:
I20260811 18:57:34.849349 4169 rpcz_store.cc:276] 0811 18:57:31.821632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:31.825444 (+ 3812us) service_pool.cc:227] Handling call
0811 18:57:34.664265 (+2838821us) ts_tablet_manager.cc:1145] Deleting tablet a902ad70a25546be926ff8f7bf2ed64f
0811 18:57:34.664275 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.803163 (+138888us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.806569 (+ 3406us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.849209 (+ 42640us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41070,"tablet-delete.queue_time_us":2838648}
I20260811 18:57:34.849840 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 875) took 2668 ms. Trace:
I20260811 18:57:34.849918 4169 rpcz_store.cc:276] 0811 18:57:32.181388 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.181581 (+ 193us) service_pool.cc:227] Handling call
0811 18:57:34.849684 (+2668103us) ts_tablet_manager.cc:1145] Deleting tablet cdb92902c2584751af77d3117f52cb4e
0811 18:57:34.849697 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.849824 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2640023}
I20260811 18:57:34.850267 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 876) took 2636 ms. Trace:
I20260811 18:57:34.850411 4169 rpcz_store.cc:276] 0811 18:57:32.213886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.215511 (+ 1625us) service_pool.cc:227] Handling call
0811 18:57:34.850134 (+2634623us) ts_tablet_manager.cc:1145] Deleting tablet ccc63b46a4e346f79d1e03a82490794c
0811 18:57:34.850157 (+ 23us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.850252 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2634335}
I20260811 18:57:34.850828 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 878) took 2633 ms. Trace:
I20260811 18:57:34.850912 4169 rpcz_store.cc:276] 0811 18:57:32.216998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.217086 (+ 88us) service_pool.cc:227] Handling call
0811 18:57:34.850687 (+2633601us) ts_tablet_manager.cc:1145] Deleting tablet 14799a6f61a946008acee8a09d050c1e
0811 18:57:34.850700 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.850809 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633338}
I20260811 18:57:34.851256 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 877) took 2634 ms. Trace:
I20260811 18:57:34.851334 4169 rpcz_store.cc:276] 0811 18:57:32.216870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.220032 (+ 3162us) service_pool.cc:227] Handling call
0811 18:57:34.851128 (+2631096us) ts_tablet_manager.cc:1145] Deleting tablet 99de8ce45e514ff291cb9e5c0a4c00d1
0811 18:57:34.851139 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.851239 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2630876}
I20260811 18:57:34.851648 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 879) took 2616 ms. Trace:
I20260811 18:57:34.851724 4169 rpcz_store.cc:276] 0811 18:57:32.235277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.236166 (+ 889us) service_pool.cc:227] Handling call
0811 18:57:34.851538 (+2615372us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:34.851549 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.851632 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2615174}
I20260811 18:57:34.852083 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 880) took 2578 ms. Trace:
I20260811 18:57:34.852164 4169 rpcz_store.cc:276] 0811 18:57:32.274128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.274201 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:34.851954 (+2577753us) ts_tablet_manager.cc:1145] Deleting tablet e1ad3860d03f46baadd5ede854f1a909
0811 18:57:34.851965 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.852067 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2577468}
I20260811 18:57:34.852416 4169 tablet_replica.cc:330] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.852674 4169 raft_consensus.cc:2229] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.852943 4169 pending_rounds.cc:62] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.852994 4169 pending_rounds.cc:68] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400178813452288 op_type: NO_OP noop_request { }
I20260811 18:57:34.853134 4169 raft_consensus.cc:2866] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:34.853276 4169 raft_consensus.cc:2258] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.854584 4169 ts_tablet_manager.cc:1905] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.861030 5376 raft_consensus.cc:482] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.861207 5376 raft_consensus.cc:504] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:34.861845 5376 leader_election.cc:288] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:34.862911 4169 ts_tablet_manager.cc:1918] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:34.864864 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d37e81d376e14907b13746cca831a96b
I20260811 18:57:34.866585 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f86bfa65a8774b51bc7e7fe59442bf2f
W20260811 18:57:34.835907 3678 leader_election.cc:334] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.845440 4167 ts_tablet_manager.cc:1918] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.872424 4167 log.cc:1196] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:34.872817 4167 ts_tablet_manager.cc:1939] T 3263da75f0d84528b7c8cc55dcaeaa46 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.873189 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 946) took 1130 ms. Trace:
I20260811 18:57:34.873281 4167 rpcz_store.cc:276] 0811 18:57:33.742841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.743076 (+ 235us) service_pool.cc:227] Handling call
0811 18:57:34.823954 (+1080878us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:34.823962 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.833054 (+ 9092us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.845311 (+ 12257us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.873172 (+ 27861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1080605}
I20260811 18:57:34.873340 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d9a2caa47da74f95bd2276d3b1547505"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.874195 4167 tablet_replica.cc:330] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.874353 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9a2caa47da74f95bd2276d3b1547505 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.874581 3551 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_5627fd661e4441d8a677d35bc0f16cf0 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 3 got 0
I20260811 18:57:34.874836 4167 raft_consensus.cc:2229] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.875005 4167 raft_consensus.cc:2258] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.875546 4167 ts_tablet_manager.cc:1905] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.875555 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b2be35bff506498281ab88f9cbb55de8" } new_table_name: "default.Table_0cde3a66762e4c0ea53da6ce0999bf3c" modify_external_catalogs: true
I20260811 18:57:34.835273 4162 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:34.877939 4162 tablet_bootstrap.cc:654] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.878720 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cf1083177b984dbc8f987dc5575ae11a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.879323 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf1083177b984dbc8f987dc5575ae11a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.880829 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.881424 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f4984467f654126ae2d4a7fe64c94f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.872399 3678 leader_election.cc:302] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:34.882436 5490 raft_consensus.cc:2726] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.882704 3678 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 46803.8us
I20260811 18:57:34.882840 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ae1b2ddf92b411f84f56e6a92130fa8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.883468 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.888016 4167 ts_tablet_manager.cc:1918] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.888175 4167 log.cc:1196] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:34.888530 4167 ts_tablet_manager.cc:1939] T 968fa8b8912b40a8bdba77db1d39c5c0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.889868 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_5627fd661e4441d8a677d35bc0f16cf0 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 3 got 0
I20260811 18:57:34.891690 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fab5512bddad41239e285f5b77ef1958"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.892555 4167 tablet_replica.cc:330] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:34.892628 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fab5512bddad41239e285f5b77ef1958 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.884410 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_567110d2149d4cbfbfcf11fbbaddd0fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:34.894239 4167 raft_consensus.cc:2229] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:34.894590 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_567110d2149d4cbfbfcf11fbbaddd0fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.894616 4167 raft_consensus.cc:2258] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.895227 4167 ts_tablet_manager.cc:1905] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.896243 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8518b7d355d14d14983f2c89aad83f5e" } new_table_name: "default.Table_b493ffc53fa84d9983273e1dabc3fa40" modify_external_catalogs: true
W20260811 18:57:34.899528 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
W20260811 18:57:34.901069 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:34.910450 4169 log.cc:1196] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/122b7898fea44ef78f269b1505d8ddc4
I20260811 18:57:34.911156 4169 ts_tablet_manager.cc:1939] T 122b7898fea44ef78f269b1505d8ddc4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:34.921674 5490 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.921876 5490 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.922571 5490 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:34.924713 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.925510 4167 ts_tablet_manager.cc:1918] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:34.932961 4167 log.cc:1196] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:34.933573 4167 ts_tablet_manager.cc:1939] T 68a7051bc539468aafba30fbaf0669d4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:34.933372 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bd16e9b8092642c896a5be846347ad41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:34.934149 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd16e9b8092642c896a5be846347ad41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:34.935873 4167 tablet_replica.cc:330] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:34.936486 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 46505ba544104458888dbc2a37115dcc from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.936729 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 881) took 2378 ms. Trace:
I20260811 18:57:34.936849 4169 rpcz_store.cc:276] 0811 18:57:32.558479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.558559 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:34.852371 (+2293812us) ts_tablet_manager.cc:1145] Deleting tablet 122b7898fea44ef78f269b1505d8ddc4
0811 18:57:34.852381 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.858296 (+ 5915us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.862816 (+ 4520us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:34.936704 (+ 73888us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2259814}
I20260811 18:57:34.937408 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 887) took 2287 ms. Trace:
I20260811 18:57:34.937498 4169 rpcz_store.cc:276] 0811 18:57:32.650161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.651393 (+ 1232us) service_pool.cc:227] Handling call
0811 18:57:34.937252 (+2285859us) ts_tablet_manager.cc:1145] Deleting tablet 844bcd16f06c4788af9c0d7d320464aa
0811 18:57:34.937267 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.937383 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2285674}
I20260811 18:57:34.937870 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 890) took 2287 ms. Trace:
I20260811 18:57:34.937956 4169 rpcz_store.cc:276] 0811 18:57:32.650535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.650733 (+ 198us) service_pool.cc:227] Handling call
0811 18:57:34.937763 (+2287030us) ts_tablet_manager.cc:1145] Deleting tablet 0a5b06fa572045f3a5fc5a0b123d77cc
0811 18:57:34.937775 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.937852 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2286130}
I20260811 18:57:34.938284 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 902) took 2042 ms. Trace:
I20260811 18:57:34.938331 4167 raft_consensus.cc:2229] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:34.938388 4169 rpcz_store.cc:276] 0811 18:57:32.896031 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.918456 (+ 22425us) service_pool.cc:227] Handling call
0811 18:57:34.938177 (+2019721us) ts_tablet_manager.cc:1145] Deleting tablet e7d75d8468fc4c938fde0b2951d84562
0811 18:57:34.938188 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:34.938265 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2019460}
I20260811 18:57:34.938540 4167 raft_consensus.cc:2258] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.938671 4169 tablet_replica.cc:330] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:34.938895 4169 raft_consensus.cc:2229] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:34.939122 4167 ts_tablet_manager.cc:1905] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.906885 4163 tablet_bootstrap.cc:492] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:34.923656 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:34.850251 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 55173.3us
W20260811 18:57:34.942520 3676 leader_election.cc:341] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:34.942665 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:34.942942 3824 leader_election.cc:334] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:34.943063 3824 leader_election.cc:302] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:34.943066 5490 raft_consensus.cc:2726] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:34.943337 5417 raft_consensus.cc:2726] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:34.944016 3824 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:34.938830 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae02c9b0b066474894546829aed304e3 (table default.Table_76b86d4f079c45b1a7f34a195fc97dfb [id=137cef1d67054dc6be05fcef9b22c375], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:34.946537 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebe3fed5aaf45918dbadfde06b2c24e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:34.946764 3741 raft_consensus.cc:3037] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.947350 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6852aaef1d5486b87e5462eae532719" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:34.947530 3744 raft_consensus.cc:3037] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.949812 3741 raft_consensus.cc:2454] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:34.950126 3744 raft_consensus.cc:2454] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:34.950629 4163 ts_tablet_manager.cc:1397] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.157s user 0.006s sys 0.001s
I20260811 18:57:34.955235 4163 raft_consensus.cc:348] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.957461 4163 raft_consensus.cc:374] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.958292 4163 raft_consensus.cc:729] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.962972 4162 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:34.961867 4163 consensus_queue.cc:260] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.964939 4162 ts_tablet_manager.cc:1397] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.129s user 0.005s sys 0.000s
I20260811 18:57:34.967805 5417 raft_consensus.cc:482] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.967965 5417 raft_consensus.cc:504] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.968619 5417 leader_election.cc:288] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.969379 5417 raft_consensus.cc:482] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.969509 5417 raft_consensus.cc:504] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.970078 5417 leader_election.cc:288] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.970414 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 40898ceb18cc4f2d9104818798049ae2 with delete_type TABLET_DATA_DELETED (replaced by tablet a62cdd333ad8494897ae8ec5afd5af74) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.970842 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet a25b1eddd4a64847b51ad3cdc0238ff0 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ed141f91ec548db87ac1e719256931b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.969846 4169 pending_rounds.cc:62] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:34.970062 4163 ts_tablet_manager.cc:1428] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260811 18:57:34.971737 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d4c26da86b49b8a37f2289f4ef6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.971539 4169 pending_rounds.cc:68] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400180473872384 op_type: NO_OP noop_request { }
I20260811 18:57:34.974300 4163 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:34.973603 4169 raft_consensus.cc:2866] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260811 18:57:34.976393 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:34.975629 4169 raft_consensus.cc:2258] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:34.976583 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 45fde25bcb40493aafb083e161aa6932 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.976796 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 302cf69ea7c14fdaac72f6a9a9aff113 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:34.977417 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 with delete_type TABLET_DATA_DELETED (replaced by tablet 56d24aff9fd34f3eb05fdf942cc7efc7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.978956 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8eaafc7677344cac8d9616996c5f416c (DEFAULT_TABLE table=default.Table_7e50906e16414e8b99c43763a538d6be [id=dab5f1b103eb489fb1bbb15659b424ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.979668 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eaafc7677344cac8d9616996c5f416c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.979809 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 38186aed20134de1bc06e0eb13806d34 (DEFAULT_TABLE table=default.Table_4464846117e74da2b263ff545f9937b3 [id=5c41b99d64b54cdbb7c2424ce5e9483b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.980195 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38186aed20134de1bc06e0eb13806d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.980492 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f1e69da8480425682a8348cad2ffa51" } modify_external_catalogs: true
I20260811 18:57:34.980779 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8f1e69da8480425682a8348cad2ffa51" } modify_external_catalogs: true
I20260811 18:57:34.980510 4162 raft_consensus.cc:348] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:34.985038 4162 raft_consensus.cc:374] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.986029 4169 ts_tablet_manager.cc:1905] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:34.987449 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 (DEFAULT_TABLE table=default.Table_3ee627ac9abf47889b5fed972cecd97e [id=17c62f2fe48546309ce0f54050e43299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.987941 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccd56a8b7cc454ea3f2f08b59374521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.988091 3856 tablet_service.cc:1467] Processing CreateTablet for tablet fae3750e5870411e8fe7dedae2fe3ec0 (DEFAULT_TABLE table=default.Table_be8ef75ca590418486730a869708b637 [id=07e28abb2e494245999c05a44c9107fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.988510 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae3750e5870411e8fe7dedae2fe3ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:34.988883 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.991767 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aa50e46e061b41f79d5d6d048ea15132" } modify_external_catalogs: true
I20260811 18:57:34.991998 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aa50e46e061b41f79d5d6d048ea15132" } modify_external_catalogs: true
I20260811 18:57:34.993060 5417 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:34.993201 5417 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:34.993788 5417 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:34.994947 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1896700e724204a37439f134a4d988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.995570 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dd203b7eb8643f493b4642c0396e258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.996107 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet afa57ed239ac4afebe517cc48d879645 with delete_type TABLET_DATA_DELETED (replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:34.996876 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:34.939157 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 94193.6us
I20260811 18:57:34.997524 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 36861ae786c549d687e687fe853d1801 (DEFAULT_TABLE table=default.Table_aa50e46e061b41f79d5d6d048ea15132 [id=5ef75d1e27a8460e83162dd6e254ab9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:34.997972 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36861ae786c549d687e687fe853d1801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.003962 4163 tablet_bootstrap.cc:654] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.004680 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a" } modify_external_catalogs: true
I20260811 18:57:35.004985 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fbcefa72c3fb48bb8c7d6dbf21c4349a" } modify_external_catalogs: true
I20260811 18:57:34.986721 4162 raft_consensus.cc:729] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.009186 4162 consensus_queue.cc:260] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.009852 4162 ts_tablet_manager.cc:1428] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:35.015811 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 09913224068f4a71bab606dea187daee (DEFAULT_TABLE table=default.Table_1afc7507935345b2a081a64ed01106da [id=9b590a32f284495ca2293a9e3b7e9e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.015936 3852 tablet_service.cc:1467] Processing CreateTablet for tablet ad8f24cdcf1c4973b388303fce0a5c08 (DEFAULT_TABLE table=default.Table_2033e26987554c839ae47bd350d82580 [id=4a215557a1474b22bffc1f4ba8a16025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.015940 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f15b0581b33b4796b527e87caf606eb2 (DEFAULT_TABLE table=default.Table_fae8f41d600f4fbfaed39f695cf6672a [id=a98ff2e2e2c14d4fa18822452154a31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.016847 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 6a70ecf0db7641419513558768d4c882 (DEFAULT_TABLE table=default.Table_58b2359b8659428190fe3cd7688c1d4d [id=df01d48865104be1ba7895f5bed2f809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017025 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 61f4112fcb7445b3803456e59a24ad0c (DEFAULT_TABLE table=default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b [id=6ec509fc5d6449dc8493d9d72c95f279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017060 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa4822d4c1149c29187c9ac880766e3 (DEFAULT_TABLE table=default.Table_2fc380eb763b488e815a72be20c8f563 [id=611c2ca37d194e2a886789d9291ad3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.016801 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 15855cd92875485c96b2eb2082e88a0b (DEFAULT_TABLE table=default.Table_6080085ba7594b18b5594e8868f0b80e [id=6e4857cbdcab4ce59110f567698f8f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.017525 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 762db87ede7a490da344412e0ff2ff98 (DEFAULT_TABLE table=default.Table_f414c31b8cff4e889368e4207afb1b53 [id=0822821331174d16b7f07c61263ae948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.019268 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 727ad79a25364ca2ae9a5246ce8bf641 (DEFAULT_TABLE table=default.Table_1e71d261fb704c1296dfc437453d6f38 [id=0497d50325384d099124f631f89e536a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.019912 4162 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.020295 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8323c8e9cc114d2a99b346afe179a86b (DEFAULT_TABLE table=default.Table_f868e00d47d34ad995459fb265d9157a [id=4ebe1460d84e46e2aede53d51d79a6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.020578 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8488352995394b02b14172ea5eed8290"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.021195 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8488352995394b02b14172ea5eed8290 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.023422 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1f988d05c9984be98e5f1f83939bc620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.023663 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09913224068f4a71bab606dea187daee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.023838 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762db87ede7a490da344412e0ff2ff98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.024032 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8323c8e9cc114d2a99b346afe179a86b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.024093 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f988d05c9984be98e5f1f83939bc620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.024309 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8f24cdcf1c4973b388303fce0a5c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.025157 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f4112fcb7445b3803456e59a24ad0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.028751 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a70ecf0db7641419513558768d4c882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.030459 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa4822d4c1149c29187c9ac880766e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.033208 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727ad79a25364ca2ae9a5246ce8bf641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.034456 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15b0581b33b4796b527e87caf606eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.040326 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.041693 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15855cd92875485c96b2eb2082e88a0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.041736 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114d22f22d3d4dc09563aa1108e1f509 consensus state has no leader
I20260811 18:57:35.042788 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.043184 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4e1fc5686944dfa3eb57702c810eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.042940 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2 (DEFAULT_TABLE table=default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 [id=c321bcbff9b04df6ad72fbe9efc2d0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.043708 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.049816 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.117s user 0.002s sys 0.000s
I20260811 18:57:35.056667 3860 tablet_service.cc:1467] Processing CreateTablet for tablet c1239b67f2784a61b56fc0596121f2e0 (DEFAULT_TABLE table=default.Table_b0759818f6cc4cd8b2747ae532c19ccc [id=afc22fa155de4ba8b573ca5f20cafa0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.057287 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1239b67f2784a61b56fc0596121f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.065750 4162 tablet_bootstrap.cc:654] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.068358 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.068763 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.073802 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc" } modify_external_catalogs: true
I20260811 18:57:35.074100 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f14f93014a8d464ba0da0bab0ceb9ddc" } modify_external_catalogs: true
I20260811 18:57:35.075853 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.076293 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b79839172cff41c3a8235675d44c21fa" } modify_external_catalogs: true
I20260811 18:57:35.076687 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b79839172cff41c3a8235675d44c21fa" } modify_external_catalogs: true
I20260811 18:57:35.078915 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_84be3c943435450c938c21da022a54f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.079490 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84be3c943435450c938c21da022a54f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.081743 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6432abd4fc5c4f9ca95971ca4795e1cf with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7fc4a898cd4e2c9e27d6f5f964412a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.082193 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c515c153df4317b3fb427983ee15e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.083135 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f1342b8cd65b4c31a47b27a79c678d2b
I20260811 18:57:35.083319 4163 tablet_bootstrap.cc:492] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.083451 4163 ts_tablet_manager.cc:1397] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.110s user 0.004s sys 0.000s
I20260811 18:57:35.083428 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a905052284004a77adf260546090e666" } new_table_name: "default.Table_43e653688f9047b2a322519e3f4de13d" modify_external_catalogs: true
I20260811 18:57:35.084229 4163 raft_consensus.cc:348] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.084438 4163 raft_consensus.cc:374] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.084497 4163 raft_consensus.cc:729] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.084703 4163 consensus_queue.cc:260] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.085459 4163 ts_tablet_manager.cc:1428] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:35.086706 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e5f47ccc193d48efa15d60a06b148e00
I20260811 18:57:35.089620 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.089924 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.090010 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.090225 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091485 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 8eaafc7677344cac8d9616996c5f416c (DEFAULT_TABLE table=default.Table_7e50906e16414e8b99c43763a538d6be [id=dab5f1b103eb489fb1bbb15659b424ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.091686 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1896700e724204a37439f134a4d988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091912 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.091967 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet afa57ed239ac4afebe517cc48d879645 with delete_type TABLET_DATA_DELETED (replaced by tablet 09f786a1b5064367ae6d89fc1262a0b5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092132 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dd203b7eb8643f493b4642c0396e258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092342 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet b27f9dbb52f34687a33c2ebf20e8f780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092584 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.092149 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 (DEFAULT_TABLE table=default.Table_3ee627ac9abf47889b5fed972cecd97e [id=17c62f2fe48546309ce0f54050e43299]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.093165 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 38186aed20134de1bc06e0eb13806d34 (DEFAULT_TABLE table=default.Table_4464846117e74da2b263ff545f9937b3 [id=5c41b99d64b54cdbb7c2424ce5e9483b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.091908 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eaafc7677344cac8d9616996c5f416c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.095093 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38186aed20134de1bc06e0eb13806d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.099941 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 36861ae786c549d687e687fe853d1801 (DEFAULT_TABLE table=default.Table_aa50e46e061b41f79d5d6d048ea15132 [id=5ef75d1e27a8460e83162dd6e254ab9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100137 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 15855cd92875485c96b2eb2082e88a0b (DEFAULT_TABLE table=default.Table_6080085ba7594b18b5594e8868f0b80e [id=6e4857cbdcab4ce59110f567698f8f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100330 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36861ae786c549d687e687fe853d1801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100492 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15855cd92875485c96b2eb2082e88a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100416 3712 tablet_service.cc:1467] Processing CreateTablet for tablet f15b0581b33b4796b527e87caf606eb2 (DEFAULT_TABLE table=default.Table_fae8f41d600f4fbfaed39f695cf6672a [id=a98ff2e2e2c14d4fa18822452154a31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100608 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fae3750e5870411e8fe7dedae2fe3ec0 (DEFAULT_TABLE table=default.Table_be8ef75ca590418486730a869708b637 [id=07e28abb2e494245999c05a44c9107fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.100737 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15b0581b33b4796b527e87caf606eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100896 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fae3750e5870411e8fe7dedae2fe3ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.101138 3714 tablet_service.cc:1467] Processing CreateTablet for tablet 8323c8e9cc114d2a99b346afe179a86b (DEFAULT_TABLE table=default.Table_f868e00d47d34ad995459fb265d9157a [id=4ebe1460d84e46e2aede53d51d79a6f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.101423 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8323c8e9cc114d2a99b346afe179a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.102553 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 727ad79a25364ca2ae9a5246ce8bf641 (DEFAULT_TABLE table=default.Table_1e71d261fb704c1296dfc437453d6f38 [id=0497d50325384d099124f631f89e536a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.102841 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727ad79a25364ca2ae9a5246ce8bf641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.106549 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccd56a8b7cc454ea3f2f08b59374521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.106663 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 6a70ecf0db7641419513558768d4c882 (DEFAULT_TABLE table=default.Table_58b2359b8659428190fe3cd7688c1d4d [id=df01d48865104be1ba7895f5bed2f809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.106791 3717 tablet_service.cc:1467] Processing CreateTablet for tablet ad8f24cdcf1c4973b388303fce0a5c08 (DEFAULT_TABLE table=default.Table_2033e26987554c839ae47bd350d82580 [id=4a215557a1474b22bffc1f4ba8a16025]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.106997 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a70ecf0db7641419513558768d4c882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.107139 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8f24cdcf1c4973b388303fce0a5c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.100394 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 09913224068f4a71bab606dea187daee (DEFAULT_TABLE table=default.Table_1afc7507935345b2a081a64ed01106da [id=9b590a32f284495ca2293a9e3b7e9e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.110694 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09913224068f4a71bab606dea187daee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.111121 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 61f4112fcb7445b3803456e59a24ad0c (DEFAULT_TABLE table=default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b [id=6ec509fc5d6449dc8493d9d72c95f279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.111433 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f4112fcb7445b3803456e59a24ad0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.115135 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa4822d4c1149c29187c9ac880766e3 (DEFAULT_TABLE table=default.Table_2fc380eb763b488e815a72be20c8f563 [id=611c2ca37d194e2a886789d9291ad3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.115440 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa4822d4c1149c29187c9ac880766e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.118867 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 762db87ede7a490da344412e0ff2ff98 (DEFAULT_TABLE table=default.Table_f414c31b8cff4e889368e4207afb1b53 [id=0822821331174d16b7f07c61263ae948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.119200 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762db87ede7a490da344412e0ff2ff98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.123011 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.123342 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4e1fc5686944dfa3eb57702c810eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.124886 3705 tablet_service.cc:1467] Processing CreateTablet for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2 (DEFAULT_TABLE table=default.Table_85fe88f9dd3d4bf7892aa7bd9fe91823 [id=c321bcbff9b04df6ad72fbe9efc2d0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.125892 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbecdd3617b4b36b0dc5b09ba64e1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.126966 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.127372 3722 tablet_service.cc:1467] Processing CreateTablet for tablet c1239b67f2784a61b56fc0596121f2e0 (DEFAULT_TABLE table=default.Table_b0759818f6cc4cd8b2747ae532c19ccc [id=afc22fa155de4ba8b573ca5f20cafa0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.127235 3709 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.127710 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1239b67f2784a61b56fc0596121f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.128165 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_98051d104e9e4d9b976577088d6126a2" } new_table_name: "default.Table_59c9cdf2f7424fa7980a53799ff004d3" modify_external_catalogs: true
I20260811 18:57:35.128214 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_315c3cbf08354dae86745d91bbc2cd11" } new_table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" modify_external_catalogs: true
I20260811 18:57:35.130462 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f86bfa65a8774b51bc7e7fe59442bf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.134187 4162 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.134356 4162 ts_tablet_manager.cc:1397] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.115s user 0.003s sys 0.003s
I20260811 18:57:35.135110 4162 raft_consensus.cc:348] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.135330 4162 raft_consensus.cc:374] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.135375 4162 raft_consensus.cc:729] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.135672 4162 consensus_queue.cc:260] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.140739 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a902ad70a25546be926ff8f7bf2ed64f (table default.Table_44def5389adf4d9ca825712dfb862206 [id=1db4b7da907046ad9f7b357c47f8c9ab]) successfully deleted
W20260811 18:57:35.141081 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cdb92902c2584751af77d3117f52cb4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdb92902c2584751af77d3117f52cb4e already in progress: creating tablet
W20260811 18:57:35.141352 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ccc63b46a4e346f79d1e03a82490794c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccc63b46a4e346f79d1e03a82490794c already in progress: creating tablet
W20260811 18:57:35.141815 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 14799a6f61a946008acee8a09d050c1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14799a6f61a946008acee8a09d050c1e already in progress: creating tablet
W20260811 18:57:35.141984 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99de8ce45e514ff291cb9e5c0a4c00d1 already in progress: creating tablet
W20260811 18:57:35.142163 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 55f97f37fa7c483cacaeed54f93e7a24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55f97f37fa7c483cacaeed54f93e7a24 already in progress: creating tablet
W20260811 18:57:35.142400 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e1ad3860d03f46baadd5ede854f1a909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1ad3860d03f46baadd5ede854f1a909 already in progress: creating tablet
I20260811 18:57:35.143620 4162 ts_tablet_manager.cc:1428] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260811 18:57:35.147202 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fcb1c9787d3f4fd181f86deca28ad160
I20260811 18:57:35.149772 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 122b7898fea44ef78f269b1505d8ddc4 (table default.Table_be05a95023fd4cfdbcfba8e0ea6bdf1b [id=1bcc678c540844b38d761cec1618c852]) successfully deleted
W20260811 18:57:35.150051 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 844bcd16f06c4788af9c0d7d320464aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 844bcd16f06c4788af9c0d7d320464aa already in progress: creating tablet
W20260811 18:57:35.150377 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0a5b06fa572045f3a5fc5a0b123d77cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5b06fa572045f3a5fc5a0b123d77cc already in progress: creating tablet
W20260811 18:57:35.150609 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e7d75d8468fc4c938fde0b2951d84562 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7d75d8468fc4c938fde0b2951d84562 already in progress: creating tablet
W20260811 18:57:35.151011 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f24c8db84f04484584ad1ba5d97f96d4 consensus state has no leader
W20260811 18:57:35.151983 3824 consensus_peers.cc:487] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.152113 3824 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.152338 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.156924 5417 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.157085 5417 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.157825 5417 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.146665 4162 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:35.161530 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5bbd3274980470b9fe71457b49bdd62 consensus state has no leader
I20260811 18:57:35.161914 4162 tablet_bootstrap.cc:654] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.162545 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c96a264e017471ea114ca0c6a9500ef consensus state has no leader
W20260811 18:57:35.162948 3824 leader_election.cc:334] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.153110 4167 ts_tablet_manager.cc:1918] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.163221 4167 log.cc:1196] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/45fde25bcb40493aafb083e161aa6932
I20260811 18:57:35.163846 4167 ts_tablet_manager.cc:1939] T 45fde25bcb40493aafb083e161aa6932 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.165692 4167 tablet_replica.cc:330] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.166103 4167 raft_consensus.cc:2229] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.166445 4167 raft_consensus.cc:2258] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.167590 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:35.167824 3740 raft_consensus.cc:2454] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:35.168919 5417 raft_consensus.cc:482] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.169072 5417 raft_consensus.cc:504] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.169631 5417 leader_election.cc:288] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:35.170262 3824 leader_election.cc:341] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:35.170476 3824 leader_election.cc:302] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.170749 5417 raft_consensus.cc:2726] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.174428 4169 ts_tablet_manager.cc:1918] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.174638 4169 log.cc:1196] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c26851245cab4a2d80e94d4fe278fd50
I20260811 18:57:35.175108 4169 ts_tablet_manager.cc:1939] T c26851245cab4a2d80e94d4fe278fd50 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.175504 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 913) took 2199 ms. Trace:
I20260811 18:57:35.175644 4169 rpcz_store.cc:276] 0811 18:57:32.975983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.976595 (+ 612us) service_pool.cc:227] Handling call
0811 18:57:34.938626 (+1962031us) ts_tablet_manager.cc:1145] Deleting tablet c26851245cab4a2d80e94d4fe278fd50
0811 18:57:34.938640 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.040944 (+102304us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.146236 (+105292us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.175488 (+ 29252us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":144,"tablet-delete.queue_time_us":1961695}
I20260811 18:57:35.176162 4169 tablet_replica.cc:330] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.176771 4169 raft_consensus.cc:2229] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.176875 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8e24f37301a145fe9b5029eaa654e2af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.177280 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (table default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]) successfully deleted
W20260811 18:57:35.177561 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e24f37301a145fe9b5029eaa654e2af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:35.177537 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ccc63b46a4e346f79d1e03a82490794c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccc63b46a4e346f79d1e03a82490794c already in progress: creating tablet
W20260811 18:57:35.180197 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cdb92902c2584751af77d3117f52cb4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdb92902c2584751af77d3117f52cb4e already in progress: creating tablet
W20260811 18:57:35.181669 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 99de8ce45e514ff291cb9e5c0a4c00d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99de8ce45e514ff291cb9e5c0a4c00d1 already in progress: creating tablet
I20260811 18:57:35.180629 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0d7c25599688441595b4b8f6cfea4584" } new_table_name: "default.Table_41871fb0db9c472689c3dd40888d4c77" modify_external_catalogs: true
W20260811 18:57:35.182677 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 14799a6f61a946008acee8a09d050c1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14799a6f61a946008acee8a09d050c1e already in progress: creating tablet
I20260811 18:57:35.182906 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 55f97f37fa7c483cacaeed54f93e7a24 (table default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]) successfully deleted
I20260811 18:57:35.183094 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 58c515c153df4317b3fb427983ee15e0 (table default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]) successfully deleted
W20260811 18:57:35.183334 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e1ad3860d03f46baadd5ede854f1a909 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1ad3860d03f46baadd5ede854f1a909 already in progress: creating tablet
I20260811 18:57:35.183568 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6432abd4fc5c4f9ca95971ca4795e1cf (table default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
W20260811 18:57:35.183779 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0a5b06fa572045f3a5fc5a0b123d77cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a5b06fa572045f3a5fc5a0b123d77cc already in progress: creating tablet
W20260811 18:57:35.183992 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 844bcd16f06c4788af9c0d7d320464aa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 844bcd16f06c4788af9c0d7d320464aa already in progress: creating tablet
W20260811 18:57:35.184194 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e7d75d8468fc4c938fde0b2951d84562 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7d75d8468fc4c938fde0b2951d84562 already in progress: creating tablet
I20260811 18:57:35.184389 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7251204325dd424192a6f58e18e71c34 (table default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]) successfully deleted
W20260811 18:57:35.184585 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6df5545532e346f7abcdc4cd228b84da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df5545532e346f7abcdc4cd228b84da already in progress: creating tablet
I20260811 18:57:35.184868 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:35.185089 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4af28c051e8243eaa73d4a5ed1abcea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af28c051e8243eaa73d4a5ed1abcea6 already in progress: creating tablet
W20260811 18:57:35.185375 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9608d6ddcad749cd9e72dd6f265db0da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9608d6ddcad749cd9e72dd6f265db0da already in progress: creating tablet
W20260811 18:57:35.185676 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 05cba6b392b642e59321570aac3541d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05cba6b392b642e59321570aac3541d0 already in progress: creating tablet
W20260811 18:57:35.185951 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 101994cc708a4fadb46fc8c5915a5587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 101994cc708a4fadb46fc8c5915a5587 already in progress: creating tablet
W20260811 18:57:35.186151 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 already in progress: creating tablet
W20260811 18:57:35.186393 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7216c095cc03424bbbe97a52a9600ebb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7216c095cc03424bbbe97a52a9600ebb already in progress: creating tablet
I20260811 18:57:35.187420 4163 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.190160 5510 tablet.cc:1722] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260811 18:57:35.194725 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a25a758955e43598be8e8d9c99b750e has no consensus state
I20260811 18:57:35.194726 4169 raft_consensus.cc:2258] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.195731 4169 ts_tablet_manager.cc:1905] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.196126 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1fed68aa37dc45c4bd9492de217dd73c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.196789 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fed68aa37dc45c4bd9492de217dd73c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.198694 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d6113c39441d47a8a1d07df0fbd2371e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.198997 3824 leader_election.cc:302] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:35.199296 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6113c39441d47a8a1d07df0fbd2371e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.199301 5417 raft_consensus.cc:2781] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.199432 5417 raft_consensus.cc:686] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.199714 5417 consensus_queue.cc:237] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.201915 4163 tablet_bootstrap.cc:654] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.206830 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_44d5558ecc5b4b7cba0c0a120270f3eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.207436 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44d5558ecc5b4b7cba0c0a120270f3eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.211647 4167 ts_tablet_manager.cc:1905] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.213649 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b7a67e56b3f641139862409e95439c6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.214287 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7a67e56b3f641139862409e95439c6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.214728 3824 leader_election.cc:302] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.215121 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:35.215086 3824 consensus_peers.cc:487] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f3060822021431584a494e6941be756. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.221007 5376 raft_consensus.cc:2781] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.222381 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
W20260811 18:57:35.227742 3824 consensus_peers.cc:487] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2718cd4a98742dfa8014a8a38c69c45. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.227954 3824 consensus_peers.cc:487] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3263da75f0d84528b7c8cc55dcaeaa46. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.230018 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 43689.9us
I20260811 18:57:35.230270 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 968fa8b8912b40a8bdba77db1d39c5c0 (table default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]) successfully deleted
I20260811 18:57:35.230492 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 68a7051bc539468aafba30fbaf0669d4 (table default.Table_821a790ed161401bb9758d71a0760916 [id=4a5d04ba5f3942a3942909ebd2bf5e20]) successfully deleted
I20260811 18:57:35.231968 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 45fde25bcb40493aafb083e161aa6932 (table default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c [id=4bc5d2d6894e40ebab164e5ffd6ac600]) successfully deleted
W20260811 18:57:35.239728 3824 leader_election.cc:341] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0ac9bdc606be40558dfa33e1e7a8ebc7
W20260811 18:57:35.240202 3824 consensus_peers.cc:487] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 221d0d8d69f142a29ec7065a82b3f769. This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.240453 3824 leader_election.cc:341] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: f7d4c26da86b49b8a37f2289f4ef6614
W20260811 18:57:35.240697 3824 leader_election.cc:341] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:35.240933 3824 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.241362 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.241978 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f47ccc193d48efa15d60a06b148e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.242216 3824 leader_election.cc:334] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.243093 5511 raft_consensus.cc:2781] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:35.242991 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_45ec6e6d5001405faabf015cd3bd77e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.243245 5511 raft_consensus.cc:482] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:35.243466 5511 raft_consensus.cc:3037] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:35.243722 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45ec6e6d5001405faabf015cd3bd77e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.245424 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.224295 4162 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.246114 4162 ts_tablet_manager.cc:1397] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.100s user 0.002s sys 0.002s
I20260811 18:57:35.247558 4162 raft_consensus.cc:348] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.248041 4162 raft_consensus.cc:374] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.248170 4162 raft_consensus.cc:729] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.248528 4162 consensus_queue.cc:260] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.249461 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1e71d261fb704c1296dfc437453d6f38" } new_table_name: "default.Table_adb037985442440c80c98862194343d9" modify_external_catalogs: true
I20260811 18:57:35.250748 4163 tablet_bootstrap.cc:492] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.250875 4163 ts_tablet_manager.cc:1397] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.000s
I20260811 18:57:35.251652 5490 raft_consensus.cc:482] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.251782 5490 raft_consensus.cc:504] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.252360 5490 leader_election.cc:288] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:35.254638 3678 leader_election.cc:334] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.256886 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.260021 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 727ad79a25364ca2ae9a5246ce8bf641 consensus state has no leader
I20260811 18:57:35.249507 4162 ts_tablet_manager.cc:1428] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:35.252573 4163 raft_consensus.cc:348] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.256026 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cce97bfdb1d41e685e091c8c5cc6946" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.267155 3676 leader_election.cc:341] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.267709 3676 leader_election.cc:302] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.268085 5490 raft_consensus.cc:2726] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.262710 5376 raft_consensus.cc:686] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.268929 5376 consensus_queue.cc:237] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.267604 4163 raft_consensus.cc:374] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.269557 4163 raft_consensus.cc:729] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.271557 4163 consensus_queue.cc:260] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.272912 4163 ts_tablet_manager.cc:1428] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.022s user 0.002s sys 0.001s
I20260811 18:57:35.273604 4163 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.277872 4163 tablet_bootstrap.cc:654] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.279557 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 302cf69ea7c14fdaac72f6a9a9aff113
I20260811 18:57:35.280014 3551 catalog_manager.cc:6565] ReplaceTablet: tablet 302cf69ea7c14fdaac72f6a9a9aff113 deleted and replaced by tablet 9708871030cc4ea6b620fb647403c42a
I20260811 18:57:35.280689 5490 raft_consensus.cc:482] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.280850 5490 raft_consensus.cc:504] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.281555 5490 leader_election.cc:288] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.283948 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a847581412494e34b2c76e605ff11772" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.284672 3676 leader_election.cc:341] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.285956 3678 leader_election.cc:334] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.286065 3678 leader_election.cc:302] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.286758 5490 raft_consensus.cc:2726] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.287014 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_21556170a88148319a2e03775ac1e02f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.287694 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21556170a88148319a2e03775ac1e02f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.290567 4162 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.292500 4162 tablet_bootstrap.cc:654] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.292585 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 with delete_type TABLET_DATA_DELETED (replaced by tablet 9708871030cc4ea6b620fb647403c42a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.293351 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c" } modify_external_catalogs: true
I20260811 18:57:35.293570 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_120c3bf5084e4fd1a967d4564e474f6c" } modify_external_catalogs: true
I20260811 18:57:35.296373 4163 tablet_bootstrap.cc:492] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.296679 4163 ts_tablet_manager.cc:1397] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.006s
I20260811 18:57:35.297240 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e50906e16414e8b99c43763a538d6be" } modify_external_catalogs: true
I20260811 18:57:35.297459 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e50906e16414e8b99c43763a538d6be" } modify_external_catalogs: true
I20260811 18:57:35.298369 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_36c481dceea34aacbb4ad6aabe5e403c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.298453 4163 raft_consensus.cc:348] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:35.298987 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36c481dceea34aacbb4ad6aabe5e403c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.299050 4163 raft_consensus.cc:374] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.299309 4163 raft_consensus.cc:729] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.299801 4163 consensus_queue.cc:260] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.301036 4163 ts_tablet_manager.cc:1428] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:35.301528 4163 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.301563 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ef967930b08e4f7492ffa9993b43f445"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.302179 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef967930b08e4f7492ffa9993b43f445 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.304737 4163 tablet_bootstrap.cc:654] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.305907 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 36861ae786c549d687e687fe853d1801
I20260811 18:57:35.306491 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 46505ba544104458888dbc2a37115dcc
I20260811 18:57:35.306875 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 46505ba544104458888dbc2a37115dcc deleted and replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e
I20260811 18:57:35.307381 4167 ts_tablet_manager.cc:1918] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.307596 4167 log.cc:1196] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:35.308079 4167 ts_tablet_manager.cc:1939] T 45aff3e8cb0c4f7191e4522c6ef7d156 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.311328 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 979) took 1080 ms. Trace:
W20260811 18:57:35.322548 3824 consensus_peers.cc:487] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.322793 3824 leader_election.cc:334] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.323185 3824 leader_election.cc:334] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.323339 3824 leader_election.cc:302] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:35.323664 3824 consensus_peers.cc:487] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:35.326653 4163 tablet_bootstrap.cc:492] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.327955 4163 ts_tablet_manager.cc:1397] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.003s
I20260811 18:57:35.328087 4169 ts_tablet_manager.cc:1918] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.329530 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c26851245cab4a2d80e94d4fe278fd50 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
I20260811 18:57:35.331485 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 18c599d945044e0088bc66c8472d2495
W20260811 18:57:35.340058 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2f31331974f42c3a22b3ac21b690f82: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.322836 5407 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.341651 4167 rpcz_store.cc:276] 0811 18:57:34.231069 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.231120 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:35.165587 (+934467us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:35.165627 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.302575 (+136948us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.307242 (+ 4667us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.311297 (+ 4055us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":934263}
I20260811 18:57:35.336782 4162 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.342193 4162 ts_tablet_manager.cc:1397] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.001s
I20260811 18:57:35.343137 4162 raft_consensus.cc:348] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.343372 4162 raft_consensus.cc:374] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.343426 4162 raft_consensus.cc:729] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.343600 4162 consensus_queue.cc:260] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.344108 5407 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.344640 5407 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.346472 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 36861ae786c549d687e687fe853d1801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.346514 4169 log.cc:1196] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/9bc5448a555944dfba8ccae09e25cbe9
I20260811 18:57:35.346710 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 46505ba544104458888dbc2a37115dcc with delete_type TABLET_DATA_DELETED (replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.347054 4169 ts_tablet_manager.cc:1939] T 9bc5448a555944dfba8ccae09e25cbe9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.347469 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 912) took 2376 ms. Trace:
I20260811 18:57:35.347586 4169 rpcz_store.cc:276] 0811 18:57:32.971263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:32.979785 (+ 8522us) service_pool.cc:227] Handling call
0811 18:57:35.176115 (+2196330us) ts_tablet_manager.cc:1145] Deleting tablet 9bc5448a555944dfba8ccae09e25cbe9
0811 18:57:35.176127 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.270940 (+ 94813us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.327958 (+ 57018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.347452 (+ 19494us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17477,"tablet-delete.queue_time_us":2192760}
I20260811 18:57:35.343598 5417 raft_consensus.cc:2726] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.343753 5376 raft_consensus.cc:482] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.344528 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7251204325dd424192a6f58e18e71c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345019 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb1c9787d3f4fd181f86deca28ad160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345065 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 302cf69ea7c14fdaac72f6a9a9aff113 with delete_type TABLET_DATA_DELETED (replaced by tablet 9708871030cc4ea6b620fb647403c42a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.345739 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 18c599d945044e0088bc66c8472d2495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.341192 5526 tablet.cc:1722] T 04eb81bf6f454fbfadd27393af15ab17 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.345741 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:35.346086 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f1342b8cd65b4c31a47b27a79c678d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.346220 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet e5f47ccc193d48efa15d60a06b148e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.348363 5376 raft_consensus.cc:504] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.348801 4162 ts_tablet_manager.cc:1428] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20260811 18:57:35.348861 3676 leader_election.cc:341] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.348970 5376 leader_election.cc:288] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.349086 4162 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:35.349306 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.051s user 0.004s sys 0.000s
W20260811 18:57:35.351804 3678 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.351926 3678 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.352353 5407 raft_consensus.cc:2726] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.353606 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 978) took 1122 ms. Trace:
I20260811 18:57:35.354599 4169 tablet_replica.cc:330] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.354992 4169 raft_consensus.cc:2229] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.355511 4169 pending_rounds.cc:62] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:35.355651 4169 pending_rounds.cc:68] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400176040996864 op_type: NO_OP noop_request { }
I20260811 18:57:35.355888 4169 raft_consensus.cc:2866] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:35.355851 4163 raft_consensus.cc:348] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.246291 5511 raft_consensus.cc:504] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.359305 5511 leader_election.cc:288] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.360001 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.360100 4163 raft_consensus.cc:374] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.360294 4163 raft_consensus.cc:729] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.360847 4167 rpcz_store.cc:276] 0811 18:57:34.230925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.231435 (+ 510us) service_pool.cc:227] Handling call
0811 18:57:35.353484 (+1122049us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:35.353498 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.353585 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1121794}
I20260811 18:57:35.360741 4163 consensus_queue.cc:260] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.361435 3546 catalog_manager.cc:5495] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.361928 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
I20260811 18:57:35.356094 4169 raft_consensus.cc:2258] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.363319 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 974) took 1169 ms. Trace:
I20260811 18:57:35.363426 4167 rpcz_store.cc:276] 0811 18:57:34.193699 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.227298 (+ 33599us) service_pool.cc:227] Handling call
0811 18:57:35.363151 (+1135853us) ts_tablet_manager.cc:1145] Deleting tablet 7be6d9578e7847c2822278486119390c
0811 18:57:35.363169 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.363293 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1118132}
I20260811 18:57:35.364429 4162 tablet_bootstrap.cc:654] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.346310 3824 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.366194 4167 tablet_replica.cc:330] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.367228 4167 raft_consensus.cc:2229] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.369361 4167 raft_consensus.cc:2258] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:35.366981 3824 leader_election.cc:334] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.369601 3824 leader_election.cc:302] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.370513 5511 raft_consensus.cc:2726] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.371346 3824 leader_election.cc:334] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.372121 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" } modify_external_catalogs: true
I20260811 18:57:35.372334 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c5780ef51fc41098318ac636d2c7c7b" } modify_external_catalogs: true
I20260811 18:57:35.373944 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_df08bd98f2ef430eaf0e10876b9e265e" } new_table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" modify_external_catalogs: true
I20260811 18:57:35.371824 3824 leader_election.cc:302] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.374747 4167 ts_tablet_manager.cc:1905] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.375571 5511 raft_consensus.cc:2726] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.375427 4163 ts_tablet_manager.cc:1428] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260811 18:57:35.378203 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dc1e05602be443528f758fba1ac0195f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.378847 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc1e05602be443528f758fba1ac0195f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.380167 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e863da40303e45bda8d9464f8ec9b8a0" } new_table_name: "default.Table_0184bf33514f4ff7bdb551628147f29d" modify_external_catalogs: true
I20260811 18:57:35.381621 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e2635923a5784be1839e861c2277e900" } new_table_name: "default.Table_2afc6c2e3a254c57840c4e891d796bf5" modify_external_catalogs: true
I20260811 18:57:35.382778 4163 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.384897 4169 ts_tablet_manager.cc:1905] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:35.385588 3824 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:35.389057 3824 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:35.390179 3824 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.391324 5407 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.391481 5407 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.392156 5407 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.393035 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0f6df7a291456d80538e2b76417873" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.393648 3676 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.394526 3678 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.394649 3678 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.394883 5407 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.395177 5511 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.398118 3824 leader_election.cc:341] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:35.404587 4163 tablet_bootstrap.cc:654] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.407037 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 36861ae786c549d687e687fe853d1801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.407696 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 9bc5448a555944dfba8ccae09e25cbe9 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:35.409843 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8eaafc7677344cac8d9616996c5f416c
I20260811 18:57:35.410558 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_91eff2208ee74ec7b44c4908dcdbe335"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.410920 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 46505ba544104458888dbc2a37115dcc with delete_type TABLET_DATA_DELETED (replaced by tablet b7ce2b43b2134defb14d45112e4f5e5e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.411008 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 18c599d945044e0088bc66c8472d2495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:35.411247 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91eff2208ee74ec7b44c4908dcdbe335 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.416239 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da (DEFAULT_TABLE table=default.Table_28350d4a6baf4232bef35569f82e0d4c [id=27d6b1acb551467eb6ca07fff162c310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.416697 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a15e7a0c0f4283ac5e575d3c15c9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.417049 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40311cd1f327461e94eb443d83a16480" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:35.417268 3745 raft_consensus.cc:3037] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.419896 3745 raft_consensus.cc:2454] T 40311cd1f327461e94eb443d83a16480 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:35.420984 4167 ts_tablet_manager.cc:1918] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.422657 4167 log.cc:1196] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:35.423031 4167 ts_tablet_manager.cc:1939] T 48d32bb4eca047e4ba786c04b8bebf8b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.423475 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 982) took 1153 ms. Trace:
I20260811 18:57:35.423576 4167 rpcz_store.cc:276] 0811 18:57:34.270025 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.270376 (+ 351us) service_pool.cc:227] Handling call
0811 18:57:35.366139 (+1095763us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:35.366152 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.384566 (+ 18414us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.420871 (+ 36305us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.423454 (+ 2583us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1095585}
I20260811 18:57:35.424340 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.425364 4167 tablet_replica.cc:330] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.425571 4167 raft_consensus.cc:2229] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.425741 4167 raft_consensus.cc:2258] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.426295 4167 ts_tablet_manager.cc:1905] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.428783 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 4e83123680c74cd187604eb237d588d1 (DEFAULT_TABLE table=default.Table_33ef706171fc47669bf54e9a71d90905 [id=dc8bc36ec82541a787edd3efac919baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.429503 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e83123680c74cd187604eb237d588d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.432983 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b0a6f7cded4c4fffbc45114fdb116623 (DEFAULT_TABLE table=default.Table_97e3f5df19fc4b04a37f62286b42f7e9 [id=e72c856e06ea481ba854aad4d2740689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.433383 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a6f7cded4c4fffbc45114fdb116623. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.440640 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa5a3d1ada554d778d507382c8836133 consensus state has no leader
I20260811 18:57:35.441079 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_99de6d48691743d798f56442d85d5be8" } new_table_name: "default.Table_b3c46b3c07b54286b190d12bff8f1965" modify_external_catalogs: true
I20260811 18:57:35.441633 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a1cb848dc1c1438aa868e39197e05009"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.442255 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1cb848dc1c1438aa868e39197e05009 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.442454 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eaafc7677344cac8d9616996c5f416c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.442783 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e2fb15b72c1d49d2ac4961bc04655a29
I20260811 18:57:35.443475 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e1f8487f0094407ba50ccdaecfebe7ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.443751 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91d19b48908740128914fa5c231dfc04 consensus state has no leader
W20260811 18:57:35.444087 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f8487f0094407ba50ccdaecfebe7ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.444240 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9df97a2d74d040ef8f3dd56e41b8c28b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.444792 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9df97a2d74d040ef8f3dd56e41b8c28b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.444929 3867 tablet_service.cc:1467] Processing CreateTablet for tablet fc8df2ed36764928ab40376ebf0838a7 (DEFAULT_TABLE table=default.Table_53f67a53b6684eaeb56341d3d428f565 [id=8fb2161513954b7880919ffbf7505a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.445418 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc8df2ed36764928ab40376ebf0838a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.446616 3553 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_667271a3af6a4ab489e6ef9a2a2769a0 [id=e98d6386b7fe47a9afba14f90faab197] alter complete (version 1)
I20260811 18:57:35.426420 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da (DEFAULT_TABLE table=default.Table_28350d4a6baf4232bef35569f82e0d4c [id=27d6b1acb551467eb6ca07fff162c310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.447350 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
I20260811 18:57:35.447635 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a15e7a0c0f4283ac5e575d3c15c9da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.447750 3551 catalog_manager.cc:6565] ReplaceTablet: tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae deleted and replaced by tablet dfb6ac49f1d4403dad38a3125264088a
W20260811 18:57:35.448393 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f2f5ce2b1d14a9a988cff644e196088 consensus state has no leader
I20260811 18:57:35.448787 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:35.427325 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 4e83123680c74cd187604eb237d588d1 (DEFAULT_TABLE table=default.Table_33ef706171fc47669bf54e9a71d90905 [id=dc8bc36ec82541a787edd3efac919baf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.450677 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e83123680c74cd187604eb237d588d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.453658 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7811427611da4cd1a27acc552c72c374"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.454267 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7811427611da4cd1a27acc552c72c374 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.454852 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b9e645f8bb1848ce8f2cd0feb6ea191c (DEFAULT_TABLE table=default.Table_7009400051004543ade835c61177ac98 [id=9c96db8126534db6aaeea68052a454cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.455315 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e645f8bb1848ce8f2cd0feb6ea191c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.456996 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.434461 4162 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.461331 4162 ts_tablet_manager.cc:1397] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.112s user 0.007s sys 0.000s
I20260811 18:57:35.462503 4162 raft_consensus.cc:348] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.462878 4162 raft_consensus.cc:374] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.462976 4162 raft_consensus.cc:729] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.463223 4162 consensus_queue.cc:260] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.463841 4162 ts_tablet_manager.cc:1428] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:35.471657 4162 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.471750 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.472406 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae with delete_type TABLET_DATA_DELETED (replaced by tablet dfb6ac49f1d4403dad38a3125264088a) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.472965 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.474066 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (table default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]) successfully deleted
W20260811 18:57:35.474565 5327 log.cc:924] Time spent T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Append to log took a long time: real 0.056s user 0.001s sys 0.003s
I20260811 18:57:35.474701 4162 tablet_bootstrap.cc:654] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.476770 3824 leader_election.cc:334] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.477267 3824 leader_election.cc:302] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.477689 5511 raft_consensus.cc:2726] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:35.478183 3824 consensus_peers.cc:487] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.481630 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 52727.1us
I20260811 18:57:35.486878 5407 raft_consensus.cc:482] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.487044 5407 raft_consensus.cc:504] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.487737 5407 leader_election.cc:288] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.488595 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b404e99e9414c67b3cd5aff12203a53" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.489185 3676 leader_election.cc:341] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:35.490872 3541 catalog_manager.cc:4808] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe203387bc9c45ba82afe6b058aa5e5f because the tablet was not found. No further retry: Not found: Tablet not found: fe203387bc9c45ba82afe6b058aa5e5f
W20260811 18:57:35.491185 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7be6d9578e7847c2822278486119390c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7be6d9578e7847c2822278486119390c already in progress: creating tablet
I20260811 18:57:35.491462 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 48d32bb4eca047e4ba786c04b8bebf8b (table default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]) successfully deleted
W20260811 18:57:35.492498 3678 leader_election.cc:334] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.492610 3678 leader_election.cc:302] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.493076 5407 raft_consensus.cc:2726] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.493319 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eaafc7677344cac8d9616996c5f416c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.495417 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ec98c10d285144029e63c99243d4d9fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.496577 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec98c10d285144029e63c99243d4d9fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.498090 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba1e7c81d1bf4209b7089639928742f1" } new_table_name: "default.Table_14587bf2389240a1b10454cad65b748a" modify_external_catalogs: true
I20260811 18:57:35.500945 4163 tablet_bootstrap.cc:492] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.501312 4163 ts_tablet_manager.cc:1397] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.124s user 0.002s sys 0.004s
I20260811 18:57:35.501982 3551 catalog_manager.cc:5495] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.503438 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:35.503822 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:35.504069 3551 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:35.505438 4163 raft_consensus.cc:348] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.505816 4163 raft_consensus.cc:374] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.505955 4163 raft_consensus.cc:729] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.506283 4163 consensus_queue.cc:260] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.508292 4162 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.508486 4162 ts_tablet_manager.cc:1397] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.000s
I20260811 18:57:35.510278 4163 ts_tablet_manager.cc:1428] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:35.510697 4163 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:35.516179 3824 leader_election.cc:341] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 45fde25bcb40493aafb083e161aa6932
I20260811 18:57:35.517300 3824 leader_election.cc:302] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.518620 5511 raft_consensus.cc:2781] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:35.518795 5511 raft_consensus.cc:686] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:35.519061 5511 consensus_queue.cc:237] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.521121 5510 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:35.523518 4162 raft_consensus.cc:348] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.526859 4162 raft_consensus.cc:374] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.526950 4162 raft_consensus.cc:729] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.527180 4162 consensus_queue.cc:260] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.527869 4162 ts_tablet_manager.cc:1428] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
W20260811 18:57:35.530963 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45d6d76629d440129e4509fad38dde43 consensus state has no leader
I20260811 18:57:35.537204 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28350d4a6baf4232bef35569f82e0d4c" } modify_external_catalogs: true
I20260811 18:57:35.537510 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_28350d4a6baf4232bef35569f82e0d4c" } modify_external_catalogs: true
I20260811 18:57:35.537958 4169 ts_tablet_manager.cc:1918] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:35.538630 4169 log.cc:1196] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0b3ec38f9b55489ba00ed76f9e4456f5
W20260811 18:57:35.538317 3824 leader_election.cc:334] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.539425 3824 leader_election.cc:302] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.541270 4169 ts_tablet_manager.cc:1939] T 0b3ec38f9b55489ba00ed76f9e4456f5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:35.542454 5376 raft_consensus.cc:2726] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.541939 4163 tablet_bootstrap.cc:654] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.543885 3824 leader_election.cc:334] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.545624 3718 tablet_service.cc:1467] Processing CreateTablet for tablet fc8df2ed36764928ab40376ebf0838a7 (DEFAULT_TABLE table=default.Table_53f67a53b6684eaeb56341d3d428f565 [id=8fb2161513954b7880919ffbf7505a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.545715 3705 tablet_service.cc:1467] Processing CreateTablet for tablet b0a6f7cded4c4fffbc45114fdb116623 (DEFAULT_TABLE table=default.Table_97e3f5df19fc4b04a37f62286b42f7e9 [id=e72c856e06ea481ba854aad4d2740689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.548939 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e2fb15b72c1d49d2ac4961bc04655a29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.550478 3853 tablet_service.cc:1467] Processing CreateTablet for tablet fa6509e905694f979191469d20e1de41 (DEFAULT_TABLE table=default.Table_c031c1fe18014450aa4cffbc9cd0f573 [id=b9b4076a00c247e38bbe4223c548d8d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.550590 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5533c71460874591bb7380ba652e299d (DEFAULT_TABLE table=default.Table_a5361d65b50b445f9d85f5ddda046f99 [id=a5a8b9a494464ac280c90a3cbf8b4839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.550927 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6509e905694f979191469d20e1de41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.551810 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5533c71460874591bb7380ba652e299d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.551821 4162 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.558940 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc8df2ed36764928ab40376ebf0838a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.560608 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae with delete_type TABLET_DATA_DELETED (replaced by tablet dfb6ac49f1d4403dad38a3125264088a) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.560901 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a6f7cded4c4fffbc45114fdb116623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.563098 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a62cdd333ad8494897ae8ec5afd5af74 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.563184 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ab2eee21264b1aa5ba00528cae3843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.564270 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62cdd333ad8494897ae8ec5afd5af74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.566581 3715 tablet_service.cc:1467] Processing CreateTablet for tablet b9e645f8bb1848ce8f2cd0feb6ea191c (DEFAULT_TABLE table=default.Table_7009400051004543ade835c61177ac98 [id=9c96db8126534db6aaeea68052a454cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.567003 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9e645f8bb1848ce8f2cd0feb6ea191c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.574127 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 827de91945274026a2c8fd17224876d4 (DEFAULT_TABLE table=default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a [id=982c198aba3c4e94bfd3421d3478bb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.574754 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827de91945274026a2c8fd17224876d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.575685 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7407d7434a3b4290889506a933ae2f05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.576474 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7407d7434a3b4290889506a933ae2f05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.577160 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" } modify_external_catalogs: true
I20260811 18:57:35.579293 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 6a25a758955e43598be8e8d9c99b750e (DEFAULT_TABLE table=default.Table_41871fb0db9c472689c3dd40888d4c77 [id=af21ca7213cb485dbef755cf5fa54fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.579770 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a25a758955e43598be8e8d9c99b750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.580650 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d8bd22de86c94d138ea871d6e957b877 (DEFAULT_TABLE table=default.Table_aa3debf8d8774103a45a754a6f08465a [id=e3f3e088f9c04932895af3dd8eae54f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.580993 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bd22de86c94d138ea871d6e957b877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.583345 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed141f91ec548db87ac1e719256931b (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.583380 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6aacb785fd444621820be42007d55668"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.583693 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed141f91ec548db87ac1e719256931b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:35.584088 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6aacb785fd444621820be42007d55668 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.585711 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 09f786a1b5064367ae6d89fc1262a0b5 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.586104 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f786a1b5064367ae6d89fc1262a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.588135 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8752a36a67ad4fb380132326a813d172"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.588769 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8752a36a67ad4fb380132326a813d172 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.600639 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5533c71460874591bb7380ba652e299d (DEFAULT_TABLE table=default.Table_a5361d65b50b445f9d85f5ddda046f99 [id=a5a8b9a494464ac280c90a3cbf8b4839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.601222 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5533c71460874591bb7380ba652e299d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.603819 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.606586 3715 tablet_service.cc:1467] Processing CreateTablet for tablet fa6509e905694f979191469d20e1de41 (DEFAULT_TABLE table=default.Table_c031c1fe18014450aa4cffbc9cd0f573 [id=b9b4076a00c247e38bbe4223c548d8d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.607265 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa6509e905694f979191469d20e1de41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.619416 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fe20c54096804a88ba68b22d0cbea93c (DEFAULT_TABLE table=default.Table_6b28d478364f4057a8c2619e5d90b850 [id=a7df2477039d47e3aa9a641f0c30e4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.620440 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe20c54096804a88ba68b22d0cbea93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.626652 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 (DEFAULT_TABLE table=default.Table_485cdcb897e6430ba77e994afbd44a64 [id=2a7d0a1f579a41639b10989738a9c8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.625985 4162 tablet_bootstrap.cc:654] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.627292 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1700ee40ac0d40198c5b8a161a90b6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.628903 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_74e0e65fa61543fc939b36aee569344d" } modify_external_catalogs: true
I20260811 18:57:35.630381 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 46a15e7a0c0f4283ac5e575d3c15c9da
I20260811 18:57:35.630736 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet c0f68d33e1b14144b34c17e3c7201408 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.631135 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2" } modify_external_catalogs: true
W20260811 18:57:35.631824 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 114397bea09444479df113552c22e154 consensus state has no leader
I20260811 18:57:35.632316 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01f622bd12ce4b2bb39355d87ed79fb2" } modify_external_catalogs: true
I20260811 18:57:35.633280 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ec27862124b74126a62f8588ec1237fa (DEFAULT_TABLE table=default.Table_acd269829eb743178d173c6cc3d7301e [id=9602f52d737341158fe431549212c038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.633420 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 (DEFAULT_TABLE table=default.Table_58f2aeab56c6446184cfeddcb1134314 [id=e4ed6236205a46d78af24a1e46371aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.633711 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec27862124b74126a62f8588ec1237fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.633934 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e71af5ee8b463eb158ac9ae76ff252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.637728 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f4848d8f28744cc49234e7ccfd8d158c (DEFAULT_TABLE table=default.Table_489a043a9e89488ca2fbcd69855c4e5d [id=05d2ef429d684b01a2e8e8d31eeeccf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.637938 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 13af4d419ae542cc961c75f28efd0204 (DEFAULT_TABLE table=default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.638176 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4848d8f28744cc49234e7ccfd8d158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.638358 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13af4d419ae542cc961c75f28efd0204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.638875 3864 tablet_service.cc:1514] Processing DeleteTablet for tablet 45aff3e8cb0c4f7191e4522c6ef7d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.641970 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 6a25a758955e43598be8e8d9c99b750e (DEFAULT_TABLE table=default.Table_41871fb0db9c472689c3dd40888d4c77 [id=af21ca7213cb485dbef755cf5fa54fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.642446 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a25a758955e43598be8e8d9c99b750e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.637245 3718 tablet_service.cc:1467] Processing CreateTablet for tablet a62cdd333ad8494897ae8ec5afd5af74 (DEFAULT_TABLE table=default.Table_4a65337aa3b94589a51dd41812870aca [id=63c67f7beaf54f96a20f9f87df0ad12f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.638971 5526 tablet.cc:1722] T 3a9efd2f19d74a68a8536fd057377738 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:35.643602 3705 tablet_service.cc:1467] Processing CreateTablet for tablet d8bd22de86c94d138ea871d6e957b877 (DEFAULT_TABLE table=default.Table_aa3debf8d8774103a45a754a6f08465a [id=e3f3e088f9c04932895af3dd8eae54f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.643966 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bd22de86c94d138ea871d6e957b877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.639706 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 827de91945274026a2c8fd17224876d4 (DEFAULT_TABLE table=default.Table_7a06565bdcb04b7fafdbbf0cffa2f74a [id=982c198aba3c4e94bfd3421d3478bb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.639916 5407 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.647384 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827de91945274026a2c8fd17224876d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.647238 5407 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.648042 5407 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:35.653178 3678 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.653528 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62cdd333ad8494897ae8ec5afd5af74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.658506 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 09f786a1b5064367ae6d89fc1262a0b5 (DEFAULT_TABLE table=default.Table_944ef43f4d664e43b89a55f61dffcfb7 [id=6db78bb413274181b39a68fca61db986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.658900 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.660475 3676 leader_election.cc:341] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.660586 3676 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.661114 5407 raft_consensus.cc:2726] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.661612 3713 tablet_service.cc:1467] Processing CreateTablet for tablet fe20c54096804a88ba68b22d0cbea93c (DEFAULT_TABLE table=default.Table_6b28d478364f4057a8c2619e5d90b850 [id=a7df2477039d47e3aa9a641f0c30e4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.661680 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.661906 4163 tablet_bootstrap.cc:492] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.662032 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 (DEFAULT_TABLE table=default.Table_485cdcb897e6430ba77e994afbd44a64 [id=2a7d0a1f579a41639b10989738a9c8cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.658506 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed141f91ec548db87ac1e719256931b (DEFAULT_TABLE table=default.Table_25e47ee54a2742a7839d23fc2adce361 [id=b9bcf023571748f7a107a73a143c9c5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.668287 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:35.669082 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.668699 3546 catalog_manager.cc:5495] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:35.670051 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac558a0e8c943649378539ae2c99654 (DEFAULT_TABLE table=default.Table_e75355b4dffb4b4992e47c68f868d172 [id=c703521c919843518a92cb6008f79401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.670854 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 545610361d354707b143b15d22811911 (DEFAULT_TABLE table=default.Table_c4f7db8f684b436caef8f1315eadb0ee [id=442be0a799fd4e73bfa778f2e6e358be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.671339 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f786a1b5064367ae6d89fc1262a0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.674415 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 (DEFAULT_TABLE table=default.Table_58f2aeab56c6446184cfeddcb1134314 [id=e4ed6236205a46d78af24a1e46371aa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.674852 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe20c54096804a88ba68b22d0cbea93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675022 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac558a0e8c943649378539ae2c99654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675187 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e71af5ee8b463eb158ac9ae76ff252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675448 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1700ee40ac0d40198c5b8a161a90b6b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.675420 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ec27862124b74126a62f8588ec1237fa (DEFAULT_TABLE table=default.Table_acd269829eb743178d173c6cc3d7301e [id=9602f52d737341158fe431549212c038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.675734 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac558a0e8c943649378539ae2c99654 (DEFAULT_TABLE table=default.Table_e75355b4dffb4b4992e47c68f868d172 [id=c703521c919843518a92cb6008f79401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.676175 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec27862124b74126a62f8588ec1237fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.686182 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ef47211aa8864d13a55551da47804c19
I20260811 18:57:35.687121 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 22e1a33348e34e35bab786f8966c408b
I20260811 18:57:35.688453 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f" } modify_external_catalogs: true
I20260811 18:57:35.688716 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbf1868109474dcea3712c10bb4d4f3f" } modify_external_catalogs: true
I20260811 18:57:35.691546 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed141f91ec548db87ac1e719256931b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.691556 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a4f7f1249bb248359c032a4e20f7c9b4" } new_table_name: "default.Table_7b7a6563095e481982b84ec228a04f29" modify_external_catalogs: true
I20260811 18:57:35.693097 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f167c64b15d4d18a642c45b00f75658"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:35.693413 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ebca8dd587b144d7909761c052084855" } modify_external_catalogs: true
I20260811 18:57:35.693652 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ebca8dd587b144d7909761c052084855" } modify_external_catalogs: true
W20260811 18:57:35.693737 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f167c64b15d4d18a642c45b00f75658 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.694516 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545610361d354707b143b15d22811911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.695147 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c031c1fe18014450aa4cffbc9cd0f573" } new_table_name: "default.Table_4ecec4051b794418b73f208e526c9bbb" modify_external_catalogs: true
I20260811 18:57:35.696663 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8c94c9b20d784df0811302a8381a7301" } new_table_name: "default.Table_9049c4cdc3654b26a7b3d0cad713ec01" modify_external_catalogs: true
I20260811 18:57:35.699156 4163 ts_tablet_manager.cc:1397] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.151s user 0.002s sys 0.004s
I20260811 18:57:35.700631 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac558a0e8c943649378539ae2c99654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.641893 4167 ts_tablet_manager.cc:1918] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.702934 4167 log.cc:1196] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:35.703305 4167 ts_tablet_manager.cc:1939] T fc3f120758c6405dafd2ea0bf54c81d8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.704106 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 983) took 1433 ms. Trace:
I20260811 18:57:35.704205 4167 rpcz_store.cc:276] 0811 18:57:34.270140 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.270584 (+ 444us) service_pool.cc:227] Handling call
0811 18:57:35.425313 (+1154729us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:35.425325 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.550562 (+125237us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.641782 (+ 91220us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.704081 (+ 62299us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154583}
I20260811 18:57:35.704567 4167 tablet_replica.cc:330] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.704807 4167 raft_consensus.cc:2229] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.704945 4167 raft_consensus.cc:2258] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.705446 4167 ts_tablet_manager.cc:1905] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.711683 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8edcf621b1064a75906a7ffe63e1a9d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.712379 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8edcf621b1064a75906a7ffe63e1a9d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.712697 4163 raft_consensus.cc:348] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.712930 4163 raft_consensus.cc:374] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.712986 4163 raft_consensus.cc:729] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.718016 4163 consensus_queue.cc:260] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.719458 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_5cf0c22ada454c098b23f5915efe0380 [id=64009634513f46eeae6442cfadb7b0f5] alter complete (version 4)
I20260811 18:57:35.723748 5407 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.723968 5407 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:35.726689 3678 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.731956 4163 ts_tablet_manager.cc:1428] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
W20260811 18:57:35.733458 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 838a9f3025a1466296844c218cedf4fa consensus state has no leader
I20260811 18:57:35.734277 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6f804703e20c4214b8b2a31efeaef59e
W20260811 18:57:35.735441 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 419824ebd14e415f90072b8c335f1ad1 consensus state has no leader
I20260811 18:57:35.735697 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet ef47211aa8864d13a55551da47804c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.735953 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 9b931eca7ba04c11af6571cb418f882e
W20260811 18:57:35.737073 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa6509e905694f979191469d20e1de41 consensus state has no leader
I20260811 18:57:35.737017 5511 raft_consensus.cc:482] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.737178 5511 raft_consensus.cc:504] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.737814 5511 leader_election.cc:288] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.739403 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e1a33348e34e35bab786f8966c408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.744307 4163 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.747031 4163 tablet_bootstrap.cc:654] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.748555 5407 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.749262 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d495354c0994fb4a74c376078e41a26" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.749899 3676 leader_election.cc:341] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.750041 3676 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.752214 3724 tablet_service.cc:1467] Processing CreateTablet for tablet 545610361d354707b143b15d22811911 (DEFAULT_TABLE table=default.Table_c4f7db8f684b436caef8f1315eadb0ee [id=442be0a799fd4e73bfa778f2e6e358be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.752369 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e80925854f7a430c93376f6448db3632" } new_table_name: "default.Table_59b6f528f1f44b1996e11ae2e4763f93" modify_external_catalogs: true
I20260811 18:57:35.752709 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545610361d354707b143b15d22811911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.753537 5407 raft_consensus.cc:2726] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.753764 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f4848d8f28744cc49234e7ccfd8d158c (DEFAULT_TABLE table=default.Table_489a043a9e89488ca2fbcd69855c4e5d [id=05d2ef429d684b01a2e8e8d31eeeccf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.754206 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4848d8f28744cc49234e7ccfd8d158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.755192 4162 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.755366 4162 ts_tablet_manager.cc:1397] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.204s user 0.005s sys 0.002s
I20260811 18:57:35.755743 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 13af4d419ae542cc961c75f28efd0204 (DEFAULT_TABLE table=default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:35.755893 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_458dd09db5894e0fae4cc31deb113bdd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.756502 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_458dd09db5894e0fae4cc31deb113bdd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.756893 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13af4d419ae542cc961c75f28efd0204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:35.759356 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.759928 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0084f6fe30164d4d9dcc8ad20699380e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.760462 4162 raft_consensus.cc:348] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.760673 4162 raft_consensus.cc:374] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.760730 4162 raft_consensus.cc:729] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.760959 4162 consensus_queue.cc:260] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.764948 4162 ts_tablet_manager.cc:1428] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:35.767051 4162 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.780081 3740 raft_consensus.cc:1261] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:35.798623 4162 tablet_bootstrap.cc:654] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.799743 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f804703e20c4214b8b2a31efeaef59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.799880 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b931eca7ba04c11af6571cb418f882e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:35.800091 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 053d923618464b0c919563b4be9bc118 consensus state has no leader
I20260811 18:57:35.801237 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet b57ed7814a1e440cba362613cbab32f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:24 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.809252 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 60091bd891a94cc4aac6ef47673a4ee5 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:35.810775 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet fc3f120758c6405dafd2ea0bf54c81d8 (table default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]) successfully deleted
I20260811 18:57:35.811172 4163 tablet_bootstrap.cc:492] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.811326 4163 ts_tablet_manager.cc:1397] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.006s
I20260811 18:57:35.812124 4163 raft_consensus.cc:348] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.812367 4163 raft_consensus.cc:374] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.812417 4163 raft_consensus.cc:729] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.812824 4163 consensus_queue.cc:260] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.817435 4167 ts_tablet_manager.cc:1918] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.817580 4167 log.cc:1196] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:35.818002 4167 ts_tablet_manager.cc:1939] T d00d943b778741b4aeec076ba46a9c40 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.819664 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 46a15e7a0c0f4283ac5e575d3c15c9da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.820446 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet ef47211aa8864d13a55551da47804c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.820981 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 22e1a33348e34e35bab786f8966c408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.821451 4163 ts_tablet_manager.cc:1428] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:35.821514 5511 consensus_queue.cc:1035] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:35.823685 3824 leader_election.cc:341] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:35.826105 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f804703e20c4214b8b2a31efeaef59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.827364 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:35.827895 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aee6beb2e70421083841520733087f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:26 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:35.828248 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:35.828758 3824 leader_election.cc:334] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.829286 3824 leader_election.cc:302] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.829761 4163 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.830351 5511 raft_consensus.cc:2726] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.832787 4162 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.832934 4162 ts_tablet_manager.cc:1397] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.003s
I20260811 18:57:35.833552 4163 tablet_bootstrap.cc:654] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.839800 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b931eca7ba04c11af6571cb418f882e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.844568 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3f83f2ec5f2847c9820675f2f29f0674"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.845299 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f83f2ec5f2847c9820675f2f29f0674 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.845496 4162 raft_consensus.cc:348] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.845767 4162 raft_consensus.cc:374] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.845829 4162 raft_consensus.cc:729] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.846032 4162 consensus_queue.cc:260] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.846692 4162 ts_tablet_manager.cc:1428] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260811 18:57:35.846997 4162 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.847074 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_26f305827594444882a9b3b84edba403" } new_table_name: "default.Table_c33b8642f3da4f97830feaf6f6a67ea8" modify_external_catalogs: true
I20260811 18:57:35.847641 5511 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.847771 5511 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.848366 5511 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.848462 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e394e4379aae4ac8b877d52273a6bf37" } modify_external_catalogs: true
I20260811 18:57:35.848649 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e394e4379aae4ac8b877d52273a6bf37" } modify_external_catalogs: true
I20260811 18:57:35.848999 4162 tablet_bootstrap.cc:654] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.849866 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5437383c0bbb4384825f578e83c10acc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.850440 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5437383c0bbb4384825f578e83c10acc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.851936 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c421939f1354150bddc1156b40cf5fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.852564 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c421939f1354150bddc1156b40cf5fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.857321 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 984) took 1587 ms. Trace:
I20260811 18:57:35.857445 4167 rpcz_store.cc:276] 0811 18:57:34.270301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.275421 (+ 5120us) service_pool.cc:227] Handling call
0811 18:57:35.704518 (+1429097us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:35.704531 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.799227 (+ 94696us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.817316 (+ 18089us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.857294 (+ 39978us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1428906}
I20260811 18:57:35.860569 4163 tablet_bootstrap.cc:492] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.861995 4162 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.862273 4162 ts_tablet_manager.cc:1397] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260811 18:57:35.863178 4162 raft_consensus.cc:348] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.863468 4162 raft_consensus.cc:374] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.863571 4162 raft_consensus.cc:729] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.863871 4162 consensus_queue.cc:260] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.865943 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.865974 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e3021c51525846e199f279f988248af0
W20260811 18:57:35.867012 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf0594d55844ae49c99247ecd8b810f consensus state has no leader
I20260811 18:57:35.870071 4162 ts_tablet_manager.cc:1428] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:35.871220 4162 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.877878 4163 ts_tablet_manager.cc:1397] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.007s
I20260811 18:57:35.881117 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet e3021c51525846e199f279f988248af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.881279 4163 raft_consensus.cc:348] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.871752 4167 tablet_replica.cc:330] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:35.882280 4167 raft_consensus.cc:2229] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:35.882513 4163 raft_consensus.cc:374] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.882670 4163 raft_consensus.cc:729] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.883170 4167 raft_consensus.cc:2258] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.883172 4163 consensus_queue.cc:260] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.883978 4163 ts_tablet_manager.cc:1428] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:35.884593 4163 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:35.882654 3824 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7251204325dd424192a6f58e18e71c34
I20260811 18:57:35.887473 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet e3021c51525846e199f279f988248af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:35.889703 4162 tablet_bootstrap.cc:654] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.894498 4167 ts_tablet_manager.cc:1905] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.902302 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d3410fcb02604cdabfae4dfa57a0bac0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.903053 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3410fcb02604cdabfae4dfa57a0bac0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:35.903858 3824 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.904062 3824 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:35.904472 5511 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.909616 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" } modify_external_catalogs: true
I20260811 18:57:35.909794 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" } modify_external_catalogs: true
I20260811 18:57:35.909924 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f6cfffe3667844f0981c71c1734a5d1e" } modify_external_catalogs: true
I20260811 18:57:35.910005 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91d46286c2244f7aa1c8fd4259b267fa" } modify_external_catalogs: true
I20260811 18:57:35.910684 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1ec52777b5484eb18531a70c114a32f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.911378 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ec52777b5484eb18531a70c114a32f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.911552 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d00d943b778741b4aeec076ba46a9c40 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:35.913017 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
I20260811 18:57:35.914156 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_def64923a6b24aa0a7b9ac4a66dcace5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.914886 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_def64923a6b24aa0a7b9ac4a66dcace5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.917487 5511 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.917665 5511 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.918406 5511 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.922636 4163 tablet_bootstrap.cc:654] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:35.925982 4162 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.926235 4162 ts_tablet_manager.cc:1397] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.002s
I20260811 18:57:35.927392 4162 raft_consensus.cc:348] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.927656 4162 raft_consensus.cc:374] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.927786 4162 raft_consensus.cc:729] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.928066 4162 consensus_queue.cc:260] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.928678 4162 ts_tablet_manager.cc:1428] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:35.929436 4162 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.939085 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 968fa8b8912b40a8bdba77db1d39c5c0 with delete_type TABLET_DATA_DELETED (replaced by tablet f287040b879642448367abe2b1c73cd9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:35.940296 5407 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.940454 5407 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.941097 5407 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:35.941696 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ceca57cb245f4a3a6cb4b36f1306d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:35.942596 3676 leader_election.cc:341] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:35.943069 4163 tablet_bootstrap.cc:492] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:35.943341 4163 ts_tablet_manager.cc:1397] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.002s
I20260811 18:57:35.945559 5510 tablet.cc:1722] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.947212 4163 raft_consensus.cc:348] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:35.949533 3678 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.949698 3678 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:35.949966 5407 raft_consensus.cc:2726] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:35.950402 4163 raft_consensus.cc:374] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.950263 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:35.950498 4163 raft_consensus.cc:729] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.950559 3740 raft_consensus.cc:2454] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:35.950700 4163 consensus_queue.cc:260] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.953441 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0" } modify_external_catalogs: true
I20260811 18:57:35.953573 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_56b89b2b2c4b45768b2866c74d9f1390" } new_table_name: "default.Table_22bd9bbe66b64e209048c8164f116ba7" modify_external_catalogs: true
I20260811 18:57:35.953753 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f4984467f654126ae2d4a7fe64c94f0" } modify_external_catalogs: true
I20260811 18:57:35.953838 4163 ts_tablet_manager.cc:1428] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260811 18:57:35.955129 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_485cdcb897e6430ba77e994afbd44a64" } modify_external_catalogs: true
I20260811 18:57:35.955314 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_485cdcb897e6430ba77e994afbd44a64" } modify_external_catalogs: true
I20260811 18:57:35.956218 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_cce74eefcdbb44369f1915d3f40f54e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:35.956825 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cce74eefcdbb44369f1915d3f40f54e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:35.958177 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ffb2ae0be8bb46b48f6bcf386ac178f3" } new_table_name: "default.Table_6c8484897dd046ab8499d1069356cdcd" modify_external_catalogs: true
I20260811 18:57:35.957834 4163 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:35.962946 4163 tablet_bootstrap.cc:654] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.941800 3541 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2f2f5ce2b1d14a9a988cff644e196088 (table default.Table_b46915474a4146eebe6a8da8b75a4160 [id=ec725d0625094fc994ee36547333988d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2f2f5ce2b1d14a9a988cff644e196088 consensus state has no leader
I20260811 18:57:35.965509 5526 tablet.cc:1722] T 9ebe3fed5aaf45918dbadfde06b2c24e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:35.966248 3824 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:35.967226 5376 raft_consensus.cc:2781] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:35.967307 5376 raft_consensus.cc:482] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:35.967365 5376 raft_consensus.cc:3037] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.969887 5376 raft_consensus.cc:504] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:35.970577 5376 leader_election.cc:288] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:35.972887 4162 tablet_bootstrap.cc:654] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:35.972462 3824 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:35.975183 4167 ts_tablet_manager.cc:1918] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:35.975343 4167 log.cc:1196] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0aee6beb2e70421083841520733087f8
I20260811 18:57:35.975802 4167 ts_tablet_manager.cc:1939] T 0aee6beb2e70421083841520733087f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:35.979072 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 981) took 1709 ms. Trace:
I20260811 18:57:35.979197 4167 rpcz_store.cc:276] 0811 18:57:34.269930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.275792 (+ 5862us) service_pool.cc:227] Handling call
0811 18:57:35.870936 (+1595144us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:35.871195 (+ 259us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.924446 (+ 53251us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.975060 (+ 50614us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.979048 (+ 3988us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1594082}
I20260811 18:57:35.984977 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 911) took 3013 ms. Trace:
I20260811 18:57:35.986578 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b57ed7814a1e440cba362613cbab32f8" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:35.986804 3740 raft_consensus.cc:3037] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:35.986501 5376 raft_consensus.cc:482] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:35.988350 5376 raft_consensus.cc:504] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:35.989045 5376 leader_election.cc:288] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:35.989856 3740 raft_consensus.cc:2454] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:35.989199 4169 rpcz_store.cc:276] 0811 18:57:32.971100 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.024937 (+ 53837us) service_pool.cc:227] Handling call
0811 18:57:35.354528 (+2329591us) ts_tablet_manager.cc:1145] Deleting tablet 0b3ec38f9b55489ba00ed76f9e4456f5
0811 18:57:35.354547 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.451779 (+ 97232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.527313 (+ 75534us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:35.984954 (+457641us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2283934}
I20260811 18:57:35.992334 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 980) took 1722 ms. Trace:
I20260811 18:57:35.993530 4162 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:35.993738 4162 ts_tablet_manager.cc:1397] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.002s
I20260811 18:57:35.996906 4162 raft_consensus.cc:348] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:35.997169 4162 raft_consensus.cc:374] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:35.997247 4162 raft_consensus.cc:729] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.997529 4162 consensus_queue.cc:260] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:35.998654 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1af33fd98c6486f975368ec90493db7 consensus state has no leader
I20260811 18:57:35.994230 4169 tablet_replica.cc:330] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:35.998930 4169 raft_consensus.cc:2229] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:35.999035 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ed17e76c67444161b978d62bf0bef64f" } new_table_name: "default.Table_d1bad93a76ef4643bf354383513e18b8" modify_external_catalogs: true
I20260811 18:57:35.999306 4169 raft_consensus.cc:2258] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:35.999899 4169 ts_tablet_manager.cc:1905] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:35.998684 3824 leader_election.cc:302] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.000986 5376 raft_consensus.cc:2781] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:36.001258 5376 raft_consensus.cc:686] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:36.001524 5376 consensus_queue.cc:237] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.002123 4162 ts_tablet_manager.cc:1428] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:36.002564 4162 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:35.992449 4167 rpcz_store.cc:276] 0811 18:57:34.269815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.308290 (+ 38475us) service_pool.cc:227] Handling call
0811 18:57:35.992195 (+1683905us) ts_tablet_manager.cc:1145] Deleting tablet dd80633f926942e88f54b32d9a03ea3a
0811 18:57:35.992207 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:35.992313 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1682961}
I20260811 18:57:36.002981 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.003525 4167 tablet_replica.cc:330] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:36.003764 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.003998 4167 raft_consensus.cc:2229] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.004189 4167 raft_consensus.cc:2258] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.006261 5407 raft_consensus.cc:482] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.006456 5407 raft_consensus.cc:504] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.006877 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7249f865dd9d4e0fa204c278006d4cee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.007124 5407 leader_election.cc:288] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.007635 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7249f865dd9d4e0fa204c278006d4cee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.008078 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19243ab3d6744aaa87a133f1462cd2fa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.008770 3676 leader_election.cc:341] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.009044 4167 ts_tablet_manager.cc:1905] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.010483 4162 tablet_bootstrap.cc:654] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:36.010885 3678 leader_election.cc:334] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.011021 3678 leader_election.cc:302] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.012356 5490 raft_consensus.cc:2726] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.013491 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:36.013931 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 71df3738a5cf4b12820dda131ced62e1 (table default.Table_ae88bb870eb143408638ffe3715fbf48 [id=b9face92d47f453ab5de5af21691edb7]). Expected version 1 got 0
I20260811 18:57:36.014258 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 9ebe3fed5aaf45918dbadfde06b2c24e (table default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737]). Expected version 1 got 0
W20260811 18:57:36.015842 3824 leader_election.cc:341] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0aee6beb2e70421083841520733087f8
I20260811 18:57:36.016419 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0b3ec38f9b55489ba00ed76f9e4456f5 (table default.Table_31693dc5078f45c382fb5ed270c7b32d [id=41ead11057b94707a815fa4e3e2e5fd0]) successfully deleted
I20260811 18:57:36.018225 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.018884 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.026127 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.028000 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 1 got 0
I20260811 18:57:36.028470 4162 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.028851 4162 ts_tablet_manager.cc:1397] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.004s
W20260811 18:57:36.029569 3824 leader_election.cc:334] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.029927 4162 raft_consensus.cc:348] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.030470 4162 raft_consensus.cc:374] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.030740 4162 raft_consensus.cc:729] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.038594 4162 consensus_queue.cc:260] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.043979 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 0db48155ac754cd3bbac5ac2e21141e6 (DEFAULT_TABLE table=default.Table_91eff2208ee74ec7b44c4908dcdbe335 [id=017537968f9146dd9eff5b3d180287ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.044499 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db48155ac754cd3bbac5ac2e21141e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.044865 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f6b2ca385374abeb709fccbdf6fa0ba consensus state has no leader
I20260811 18:57:36.047047 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
W20260811 18:57:36.049953 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00645f522efa4076832758b48fe566f3 consensus state has no leader
I20260811 18:57:36.050426 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_767c174a258144229f562c9dda12392b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.050863 4162 ts_tablet_manager.cc:1428] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
W20260811 18:57:36.051069 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_767c174a258144229f562c9dda12392b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.052433 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd (DEFAULT_TABLE table=default.Table_dc1e05602be443528f758fba1ac0195f [id=4986af2a7d044e87ae7468ac0e6abfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.052650 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c2053e3676584227a3ea13c4f912e315" } new_table_name: "default.Table_7e6c466c96e8465c9677dcaddee8819a" modify_external_catalogs: true
I20260811 18:57:36.052861 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76506ad179a9408c9e4cf8e4275ebddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.054677 4162 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.058579 4162 tablet_bootstrap.cc:654] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.059325 4167 ts_tablet_manager.cc:1918] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.060405 4167 log.cc:1196] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:36.060873 4167 ts_tablet_manager.cc:1939] T b57ed7814a1e440cba362613cbab32f8 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.061775 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 985) took 1650 ms. Trace:
I20260811 18:57:36.061866 4167 rpcz_store.cc:276] 0811 18:57:34.411674 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.411771 (+ 97us) service_pool.cc:227] Handling call
0811 18:57:36.003471 (+1591700us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:36.003484 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.015543 (+ 12059us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.059230 (+ 43687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.061756 (+ 2526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1591472}
I20260811 18:57:36.062181 4167 tablet_replica.cc:330] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.062417 4167 raft_consensus.cc:2229] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.062574 4167 raft_consensus.cc:2258] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.063062 4167 ts_tablet_manager.cc:1905] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.063468 5376 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.063591 5376 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.064132 5376 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.072245 4163 tablet_bootstrap.cc:492] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.072450 4163 ts_tablet_manager.cc:1397] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.116s user 0.004s sys 0.000s
I20260811 18:57:36.073302 4162 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.073479 4162 ts_tablet_manager.cc:1397] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260811 18:57:36.074107 4163 raft_consensus.cc:348] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.074400 4163 raft_consensus.cc:374] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.074506 4163 raft_consensus.cc:729] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.074251 4162 raft_consensus.cc:348] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.074623 4162 raft_consensus.cc:374] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.074692 4162 raft_consensus.cc:729] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.074817 4163 consensus_queue.cc:260] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.075105 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90ce2b983a431383edb7a9296520a4 (DEFAULT_TABLE table=default.Table_36c481dceea34aacbb4ad6aabe5e403c [id=eba0b555e52141f1b62443826556b6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.075258 3856 tablet_service.cc:1467] Processing CreateTablet for tablet ff7d3c7a5fcf4c2faec0d41371788b2f (DEFAULT_TABLE table=default.Table_45ec6e6d5001405faabf015cd3bd77e5 [id=89ba458c14f14a2bbe6fe1982c0043c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.075394 4163 ts_tablet_manager.cc:1428] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:36.075567 4162 consensus_queue.cc:260] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.075867 3851 tablet_service.cc:1467] Processing CreateTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af (DEFAULT_TABLE table=default.Table_b7a67e56b3f641139862409e95439c6a [id=05cdc8978385423994bd961054aac04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.076248 4162 ts_tablet_manager.cc:1428] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:57:36.076579 4162 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.076565 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 28aaee10703b4993a701a6eecf52d284 (DEFAULT_TABLE table=default.Table_1fed68aa37dc45c4bd9492de217dd73c [id=5f07f3afc20146048d4e068138204639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.082382 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1700ee40ac0d40198c5b8a161a90b6b6
I20260811 18:57:36.083123 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e89523faca1046669de08be2f6b963dd" } modify_external_catalogs: true
I20260811 18:57:36.083366 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e89523faca1046669de08be2f6b963dd" } modify_external_catalogs: true
I20260811 18:57:36.085134 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5627fd661e4441d8a677d35bc0f16cf0" } new_table_name: "default.Table_2e89aca0564646a5bcc2c6ce3f851910" modify_external_catalogs: true
I20260811 18:57:36.086663 4162 tablet_bootstrap.cc:654] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.095325 3855 tablet_service.cc:1467] Processing CreateTablet for tablet ac48afc7eec4410e8d6055ecdb794a4f (DEFAULT_TABLE table=default.Table_d6113c39441d47a8a1d07df0fbd2371e [id=a14d06dea10b47699e87570920020fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.097420 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 41e4ec7cfec94a1988fc6c950421b48a (DEFAULT_TABLE table=default.Table_8e24f37301a145fe9b5029eaa654e2af [id=9a96027cb9cd4b6c9ec6c42c9d429f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.097779 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90ce2b983a431383edb7a9296520a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.097810 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 56937c88026541e4b70d4f507ce03bc3 (DEFAULT_TABLE table=default.Table_1f988d05c9984be98e5f1f83939bc620 [id=b544982375764fafa02b3af75fec3b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.098008 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ffd3c95b744242ba9dee25d95d8e2b68 (DEFAULT_TABLE table=default.Table_84be3c943435450c938c21da022a54f0 [id=f038b486cea94c9482f0bb57ba9e2784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.098104 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56937c88026541e4b70d4f507ce03bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.098315 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7d3c7a5fcf4c2faec0d41371788b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.098502 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e4ec7cfec94a1988fc6c950421b48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.106372 4167 ts_tablet_manager.cc:1918] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.106508 4167 log.cc:1196] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/059b325df1db4fde9d4b26b559121316
I20260811 18:57:36.106864 4167 ts_tablet_manager.cc:1939] T 059b325df1db4fde9d4b26b559121316 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.108050 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 986) took 1668 ms. Trace:
I20260811 18:57:36.108161 4167 rpcz_store.cc:276] 0811 18:57:34.439848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.439948 (+ 100us) service_pool.cc:227] Handling call
0811 18:57:36.062138 (+1622190us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:36.062149 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.066190 (+ 4041us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.106241 (+ 40051us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.108029 (+ 1788us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1621912}
I20260811 18:57:36.109841 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65cb8d901c04d8ba2b7a40efc7fa3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.110006 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 987) took 1669 ms. Trace:
I20260811 18:57:36.110080 4167 rpcz_store.cc:276] 0811 18:57:34.440538 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.440622 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:36.109883 (+1669261us) ts_tablet_manager.cc:1145] Deleting tablet 40898ceb18cc4f2d9104818798049ae2
0811 18:57:36.109892 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.109992 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1669063}
I20260811 18:57:36.110414 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 988) took 1669 ms. Trace:
I20260811 18:57:36.110484 4167 rpcz_store.cc:276] 0811 18:57:34.440720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.440788 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:36.110296 (+1669508us) ts_tablet_manager.cc:1145] Deleting tablet a25b1eddd4a64847b51ad3cdc0238ff0
0811 18:57:36.110305 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.110399 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1669364}
I20260811 18:57:36.110698 4167 tablet_replica.cc:330] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.109546 4162 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.111438 4162 ts_tablet_manager.cc:1397] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.002s
I20260811 18:57:36.112145 4163 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.112445 4162 raft_consensus.cc:348] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.112689 4162 raft_consensus.cc:374] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.112813 4162 raft_consensus.cc:729] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.113062 4162 consensus_queue.cc:260] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.113834 4162 ts_tablet_manager.cc:1428] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:36.114131 4162 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.114259 4163 tablet_bootstrap.cc:654] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.114625 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28aaee10703b4993a701a6eecf52d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.114995 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd3c95b744242ba9dee25d95d8e2b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.117450 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 38a4783168554053aa953dba5e893b6b
I20260811 18:57:36.118211 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_153744b340b44d8c9e939deedadaced9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.118458 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac48afc7eec4410e8d6055ecdb794a4f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.118887 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153744b340b44d8c9e939deedadaced9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.120101 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 374c94a12e374d0e82180421c80d92d0
I20260811 18:57:36.120690 5490 raft_consensus.cc:482] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.120819 5490 raft_consensus.cc:504] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.121462 5490 leader_election.cc:288] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.122062 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c365f79d4a4201ac323f16bddf3e62" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.122866 3676 leader_election.cc:341] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.123994 3678 leader_election.cc:334] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.124112 3678 leader_election.cc:302] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.124540 5407 raft_consensus.cc:2726] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.127409 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 1811e5f62a714f8388460dfcb1717e81 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:36.129528 4162 tablet_bootstrap.cc:654] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.129793 4167 raft_consensus.cc:2229] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.130007 4167 raft_consensus.cc:2258] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.132752 5490 raft_consensus.cc:482] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.132889 5490 raft_consensus.cc:504] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.133524 5490 leader_election.cc:288] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.133631 4167 ts_tablet_manager.cc:1905] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.134233 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6e0d1eeb63493bbea4cd78dd085a3c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:36.134521 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa57a509ea14c5e935b73663b159ef7 (DEFAULT_TABLE table=default.Table_bd16e9b8092642c896a5be846347ad41 [id=297c5c3b5878432493957c51b96ea17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.134742 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.134951 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a4783168554053aa953dba5e893b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:36.134984 3676 leader_election.cc:341] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.136030 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa57a509ea14c5e935b73663b159ef7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.136124 3678 leader_election.cc:334] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.136240 3678 leader_election.cc:302] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.136629 5490 raft_consensus.cc:2726] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.144506 3855 tablet_service.cc:1467] Processing CreateTablet for tablet b71b6efc42de4200bee281e7e70c6bc1 (DEFAULT_TABLE table=default.Table_567110d2149d4cbfbfcf11fbbaddd0fa [id=9816fe72f79d446588b0ea0b81a79786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.144966 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71b6efc42de4200bee281e7e70c6bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.148311 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 60091bd891a94cc4aac6ef47673a4ee5
I20260811 18:57:36.148849 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 60091bd891a94cc4aac6ef47673a4ee5 deleted and replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7
I20260811 18:57:36.149399 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b807102dd88842e2b6e466a147c5b675"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.149981 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b807102dd88842e2b6e466a147c5b675 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.145313 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5d8c1424a045ee927b79351d236624 (DEFAULT_TABLE table=default.Table_fab5512bddad41239e285f5b77ef1958 [id=dab6b2d17a584bce88a0d6318c810076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.150785 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5d8c1424a045ee927b79351d236624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.150930 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_15068805ee264778ac398274e74d630a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.151441 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15068805ee264778ac398274e74d630a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.144622 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fe080f2071964da6b54f1ee5af98d918 (DEFAULT_TABLE table=default.Table_4f4984467f654126ae2d4a7fe64c94f0 [id=afc91ffd37634c85b9909fcff10ca65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.154985 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe080f2071964da6b54f1ee5af98d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.157119 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fe080f2071964da6b54f1ee5af98d918
I20260811 18:57:36.157869 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fa66e20bff3745ca961cfec0dc3a65c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.158466 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa66e20bff3745ca961cfec0dc3a65c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.162214 3549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c48fbb6941794b85becdf643c4d21775 [id=afadeb62ce0d4ceda9714b891c134737] alter complete (version 1)
I20260811 18:57:36.163089 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6c18f70552714e0f9a63822c3b6fb0a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.163669 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c18f70552714e0f9a63822c3b6fb0a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.169165 3855 tablet_service.cc:1467] Processing CreateTablet for tablet 18a44d23ec2441b784693e1cbbd62ad8 (DEFAULT_TABLE table=default.Table_a1cb848dc1c1438aa868e39197e05009 [id=a42d5dcf61f14b7caecfbcfe5b654ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.169654 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a44d23ec2441b784693e1cbbd62ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.173388 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd (DEFAULT_TABLE table=default.Table_dc1e05602be443528f758fba1ac0195f [id=4986af2a7d044e87ae7468ac0e6abfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.173799 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76506ad179a9408c9e4cf8e4275ebddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.174597 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90ce2b983a431383edb7a9296520a4 (DEFAULT_TABLE table=default.Table_36c481dceea34aacbb4ad6aabe5e403c [id=eba0b555e52141f1b62443826556b6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.175377 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ff7d3c7a5fcf4c2faec0d41371788b2f (DEFAULT_TABLE table=default.Table_45ec6e6d5001405faabf015cd3bd77e5 [id=89ba458c14f14a2bbe6fe1982c0043c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.175642 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:36.173063 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 0db48155ac754cd3bbac5ac2e21141e6 (DEFAULT_TABLE table=default.Table_91eff2208ee74ec7b44c4908dcdbe335 [id=017537968f9146dd9eff5b3d180287ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.182533 4163 tablet_bootstrap.cc:492] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.182698 4163 ts_tablet_manager.cc:1397] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.000s
I20260811 18:57:36.183252 4167 ts_tablet_manager.cc:1918] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.183379 4167 log.cc:1196] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a847581412494e34b2c76e605ff11772
I20260811 18:57:36.183516 4163 raft_consensus.cc:348] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.183693 4167 ts_tablet_manager.cc:1939] T a847581412494e34b2c76e605ff11772 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.183749 4163 raft_consensus.cc:374] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.183805 4163 raft_consensus.cc:729] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.183730 3724 tablet_service.cc:1467] Processing CreateTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af (DEFAULT_TABLE table=default.Table_b7a67e56b3f641139862409e95439c6a [id=05cdc8978385423994bd961054aac04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.183981 4163 consensus_queue.cc:260] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.184327 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 28aaee10703b4993a701a6eecf52d284 (DEFAULT_TABLE table=default.Table_1fed68aa37dc45c4bd9492de217dd73c [id=5f07f3afc20146048d4e068138204639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.184769 4163 ts_tablet_manager.cc:1428] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:36.184806 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 41e4ec7cfec94a1988fc6c950421b48a (DEFAULT_TABLE table=default.Table_8e24f37301a145fe9b5029eaa654e2af [id=9a96027cb9cd4b6c9ec6c42c9d429f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.184350 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 989) took 1108 ms. Trace:
I20260811 18:57:36.185395 4162 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.185525 4162 ts_tablet_manager.cc:1397] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.000s
I20260811 18:57:36.185477 3713 tablet_service.cc:1467] Processing CreateTablet for tablet ac48afc7eec4410e8d6055ecdb794a4f (DEFAULT_TABLE table=default.Table_d6113c39441d47a8a1d07df0fbd2371e [id=a14d06dea10b47699e87570920020fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.186252 4162 raft_consensus.cc:348] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.186419 4163 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.186466 4162 raft_consensus.cc:374] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.186529 4162 raft_consensus.cc:729] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.187525 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 1700ee40ac0d40198c5b8a161a90b6b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.187918 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 56937c88026541e4b70d4f507ce03bc3 (DEFAULT_TABLE table=default.Table_1f988d05c9984be98e5f1f83939bc620 [id=b544982375764fafa02b3af75fec3b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188288 3709 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa57a509ea14c5e935b73663b159ef7 (DEFAULT_TABLE table=default.Table_bd16e9b8092642c896a5be846347ad41 [id=297c5c3b5878432493957c51b96ea17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188395 3710 tablet_service.cc:1467] Processing CreateTablet for tablet b71b6efc42de4200bee281e7e70c6bc1 (DEFAULT_TABLE table=default.Table_567110d2149d4cbfbfcf11fbbaddd0fa [id=9816fe72f79d446588b0ea0b81a79786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.188637 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet 374c94a12e374d0e82180421c80d92d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.188689 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a4783168554053aa953dba5e893b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.190291 5376 raft_consensus.cc:482] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.190465 5376 raft_consensus.cc:504] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.190521 3716 tablet_service.cc:1467] Processing CreateTablet for tablet ffd3c95b744242ba9dee25d95d8e2b68 (DEFAULT_TABLE table=default.Table_84be3c943435450c938c21da022a54f0 [id=f038b486cea94c9482f0bb57ba9e2784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.191061 5376 leader_election.cc:288] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.186755 4162 consensus_queue.cc:260] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.193763 4162 ts_tablet_manager.cc:1428] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260811 18:57:36.194058 4162 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.185148 4167 rpcz_store.cc:276] 0811 18:57:35.075693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.075774 (+ 81us) service_pool.cc:227] Handling call
0811 18:57:36.110660 (+1034886us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:36.110670 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.139947 (+ 29277us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.183139 (+ 43192us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.184329 (+ 1190us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1034638}
I20260811 18:57:36.195297 4167 tablet_replica.cc:330] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.195495 4167 raft_consensus.cc:2229] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.195766 4167 raft_consensus.cc:2258] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:36.197017 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4451cc4d7c5940cdb41c021ae486a5a9 consensus state has no leader
I20260811 18:57:36.202581 4167 ts_tablet_manager.cc:1905] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.204754 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1314176a6f3346d7a808360af2c1cfd2" } new_table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" modify_external_catalogs: true
I20260811 18:57:36.207504 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8f2c4aa71e814a5dbc9684802806666e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.208127 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f2c4aa71e814a5dbc9684802806666e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.209264 5376 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.209419 5376 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.210032 5376 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.206820 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_db7b48638c894fa5ade5118c99ed4f80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.211601 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db7b48638c894fa5ade5118c99ed4f80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.228623 5490 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.228829 5490 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.229550 5490 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.229697 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 03c50dcffac54c7bac210eeaed340ca6
I20260811 18:57:36.231398 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.233515 3676 leader_election.cc:341] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.234926 3678 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.235069 3678 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.235615 5490 raft_consensus.cc:2726] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.238471 4163 tablet_bootstrap.cc:654] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.240818 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b7ce2b43b2134defb14d45112e4f5e5e (DEFAULT_TABLE table=default.Table_deb475a5050c4f3dbaece0eb1e4ac618 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.241310 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7ce2b43b2134defb14d45112e4f5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.249212 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_bc171757977f4404bb736a03a87b9ecd" } new_table_name: "default.Table_f4089a325e0243d28570ad4a523a56b1" modify_external_catalogs: true
W20260811 18:57:36.252555 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e544b4b1592c4d51a2ed0d6461edc75e consensus state has no leader
I20260811 18:57:36.256958 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c284a387f502480c8faa36ea0e40a8f7" } new_table_name: "default.Table_abd75a155c204a429d050f24f1f5f0c4" modify_external_catalogs: true
I20260811 18:57:36.277464 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.278299 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.298070 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8fb6e34f004c466381fe6ab5b780568f" } modify_external_catalogs: true
I20260811 18:57:36.298440 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8fb6e34f004c466381fe6ab5b780568f" } modify_external_catalogs: true
I20260811 18:57:36.300993 3860 tablet_service.cc:1467] Processing CreateTablet for tablet fd099e75e1ca407087ce5e01c9916c10 (DEFAULT_TABLE table=default.Table_d9a2caa47da74f95bd2276d3b1547505 [id=c3a362588c3c46d58351c3358fd65e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.301862 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd099e75e1ca407087ce5e01c9916c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.302809 5490 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.302990 5490 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.303728 5490 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.304890 4163 tablet_bootstrap.cc:492] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.305511 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.306311 3676 leader_election.cc:341] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.306468 4163 ts_tablet_manager.cc:1397] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.121s user 0.006s sys 0.002s
W20260811 18:57:36.307597 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.307739 3678 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.308378 5490 raft_consensus.cc:2726] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.308715 3856 tablet_service.cc:1514] Processing DeleteTablet for tablet 374c94a12e374d0e82180421c80d92d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.314005 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0aee6beb2e70421083841520733087f8 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
W20260811 18:57:36.315431 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dd80633f926942e88f54b32d9a03ea3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd80633f926942e88f54b32d9a03ea3a already in progress: creating tablet
I20260811 18:57:36.316468 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b57ed7814a1e440cba362613cbab32f8 (table default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]) successfully deleted
I20260811 18:57:36.312703 4163 raft_consensus.cc:348] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.317299 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 059b325df1db4fde9d4b26b559121316 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
W20260811 18:57:36.318447 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 40898ceb18cc4f2d9104818798049ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 40898ceb18cc4f2d9104818798049ae2 already in progress: creating tablet
I20260811 18:57:36.318126 4163 raft_consensus.cc:374] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:36.319695 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a25b1eddd4a64847b51ad3cdc0238ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25b1eddd4a64847b51ad3cdc0238ff0 already in progress: creating tablet
I20260811 18:57:36.319468 4163 raft_consensus.cc:729] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.323604 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a847581412494e34b2c76e605ff11772 (table default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]) successfully deleted
I20260811 18:57:36.326602 3720 tablet_service.cc:1467] Processing CreateTablet for tablet fe080f2071964da6b54f1ee5af98d918 (DEFAULT_TABLE table=default.Table_4f4984467f654126ae2d4a7fe64c94f0 [id=afc91ffd37634c85b9909fcff10ca65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.326853 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:36.327808 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ad53b06897e4559b15d34a4d98f4005"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.328415 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ad53b06897e4559b15d34a4d98f4005 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.328828 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 18a44d23ec2441b784693e1cbbd62ad8 (DEFAULT_TABLE table=default.Table_a1cb848dc1c1438aa868e39197e05009 [id=a42d5dcf61f14b7caecfbcfe5b654ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.329360 3707 tablet_service.cc:1467] Processing CreateTablet for tablet b7ce2b43b2134defb14d45112e4f5e5e (DEFAULT_TABLE table=default.Table_deb475a5050c4f3dbaece0eb1e4ac618 [id=bd848b945128429589c5bc047d39c2ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.330004 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5d8c1424a045ee927b79351d236624 (DEFAULT_TABLE table=default.Table_fab5512bddad41239e285f5b77ef1958 [id=dab6b2d17a584bce88a0d6318c810076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.330765 3714 tablet_service.cc:1467] Processing CreateTablet for tablet fd099e75e1ca407087ce5e01c9916c10 (DEFAULT_TABLE table=default.Table_d9a2caa47da74f95bd2276d3b1547505 [id=c3a362588c3c46d58351c3358fd65e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.332155 3708 tablet_service.cc:1467] Processing CreateTablet for tablet ad8b3713cac84544a61e8640c07e876e (DEFAULT_TABLE table=default.Table_8488352995394b02b14172ea5eed8290 [id=fcb0065f413e4e6ab66a4b0861d89ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.331384 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 1b44d53ffcc9440fa7594920a05b39a5 (DEFAULT_TABLE table=default.Table_cf1083177b984dbc8f987dc5575ae11a [id=477264043a4649afb58c7591736c35f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.333652 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b44d53ffcc9440fa7594920a05b39a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.337157 3856 tablet_service.cc:1467] Processing CreateTablet for tablet ad8b3713cac84544a61e8640c07e876e (DEFAULT_TABLE table=default.Table_8488352995394b02b14172ea5eed8290 [id=fcb0065f413e4e6ab66a4b0861d89ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.337564 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8b3713cac84544a61e8640c07e876e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.346494 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.348248 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet fe080f2071964da6b54f1ee5af98d918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.349952 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 60091bd891a94cc4aac6ef47673a4ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 31df6bcee1fb4324b20384bf3d3edee7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.352073 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet fe080f2071964da6b54f1ee5af98d918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:35 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.347873 4163 consensus_queue.cc:260] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.354003 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet fc3f120758c6405dafd2ea0bf54c81d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:36.357250 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8304ac7709ca4392baec4cfa78ef5b41: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.359750 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6648cd60f5214830a53129a2e1cf43f4 (DEFAULT_TABLE table=default.Table_e1f8487f0094407ba50ccdaecfebe7ee [id=3fa1075aa4514eb7a0caf0428fde2faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.360926 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d32bb4eca047e4ba786c04b8bebf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.360810 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.361821 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6648cd60f5214830a53129a2e1cf43f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.363901 4163 ts_tablet_manager.cc:1428] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.057s user 0.002s sys 0.002s
I20260811 18:57:36.365665 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e5981cabc4e7ab53de86bb730c70a (DEFAULT_TABLE table=default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 [id=4ab5b403d47a4b61a8ae7a4748f6dc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.366889 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe879c4960b4e0cbd914f96255eff9d (DEFAULT_TABLE table=default.Table_44d5558ecc5b4b7cba0c0a120270f3eb [id=1b972bac79a94bb5af5833344e38f15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.367071 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 31bb11617f1548f88f534289706edb06 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.367668 3852 tablet_service.cc:1467] Processing CreateTablet for tablet eb52f30af7294caa877b3aa3be275259 (DEFAULT_TABLE table=default.Table_ef967930b08e4f7492ffa9993b43f445 [id=f0849aba28af422ca272f0757878b2dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.371505 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7df549360f014d629938e3e94659af8a" } modify_external_catalogs: true
I20260811 18:57:36.372092 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7df549360f014d629938e3e94659af8a" } modify_external_catalogs: true
I20260811 18:57:36.374293 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e5981cabc4e7ab53de86bb730c70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.375393 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe879c4960b4e0cbd914f96255eff9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.376025 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bb11617f1548f88f534289706edb06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.379521 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb52f30af7294caa877b3aa3be275259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.384557 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 9708871030cc4ea6b620fb647403c42a (DEFAULT_TABLE table=default.Table_744e4342189a4b29b424eebd6ad7542f [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.399227 4163 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:36.402590 3824 leader_election.cc:334] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.402766 3824 leader_election.cc:302] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:36.403061 5511 raft_consensus.cc:2726] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.406728 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.407013 3740 raft_consensus.cc:2454] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:36.421314 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9708871030cc4ea6b620fb647403c42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.421667 5511 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.421819 5511 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.422475 5511 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.427500 4169 ts_tablet_manager.cc:1918] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:36.427654 4169 log.cc:1196] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fe203387bc9c45ba82afe6b058aa5e5f
I20260811 18:57:36.428107 4169 ts_tablet_manager.cc:1939] T fe203387bc9c45ba82afe6b058aa5e5f P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:36.429032 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 915) took 3297 ms. Trace:
I20260811 18:57:36.429152 4169 rpcz_store.cc:276] 0811 18:57:33.131271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.131357 (+ 86us) service_pool.cc:227] Handling call
0811 18:57:35.993486 (+2862129us) ts_tablet_manager.cc:1145] Deleting tablet fe203387bc9c45ba82afe6b058aa5e5f
0811 18:57:35.993500 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.069932 (+ 76432us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.427385 (+357453us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.429007 (+ 1622us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2861480}
I20260811 18:57:36.429621 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 916) took 3236 ms. Trace:
I20260811 18:57:36.429809 4169 rpcz_store.cc:276] 0811 18:57:33.192772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.194177 (+ 1405us) service_pool.cc:227] Handling call
0811 18:57:36.429469 (+3235292us) ts_tablet_manager.cc:1145] Deleting tablet 6df5545532e346f7abcdc4cd228b84da
0811 18:57:36.429484 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.429601 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3234973}
I20260811 18:57:36.430138 4169 tablet_replica.cc:330] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:36.463747 4169 raft_consensus.cc:2229] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:36.476868 5490 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.477063 5490 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.477768 5490 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.479553 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.480830 3676 leader_election.cc:341] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.481215 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.481326 3678 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.482009 5407 raft_consensus.cc:2726] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.487469 4169 pending_rounds.cc:62] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:36.487744 4169 pending_rounds.cc:68] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400181435674624 op_type: NO_OP noop_request { }
I20260811 18:57:36.488283 4169 raft_consensus.cc:2866] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:36.488715 4169 raft_consensus.cc:2258] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.503548 5407 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.503751 5407 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.504016 4163 tablet_bootstrap.cc:654] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.504433 5407 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.505332 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.506361 3676 leader_election.cc:341] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:36.514714 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.515007 3678 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.515555 5407 raft_consensus.cc:2726] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.548856 4169 ts_tablet_manager.cc:1905] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:36.596338 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 134828us
I20260811 18:57:36.605265 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.605551 3745 raft_consensus.cc:2454] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:36.623458 5407 raft_consensus.cc:482] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.623656 5407 raft_consensus.cc:504] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.624279 5407 leader_election.cc:288] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.627104 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09366eccf38a4876840cc5ddddbbb103 consensus state has no leader
I20260811 18:57:36.627429 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452ee450778e4d33adfa7e2745b5d970" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.628885 3676 leader_election.cc:341] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.629536 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
W20260811 18:57:36.630511 3678 leader_election.cc:334] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.630748 3678 leader_election.cc:302] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.631196 5407 raft_consensus.cc:2726] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.631415 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8171ab03efd442a9bf4e1176f04cb24" } modify_external_catalogs: true
I20260811 18:57:36.631711 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d8171ab03efd442a9bf4e1176f04cb24" } modify_external_catalogs: true
I20260811 18:57:36.633728 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9cadd28493e444185170bcd4c314208" } modify_external_catalogs: true
I20260811 18:57:36.633996 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9cadd28493e444185170bcd4c314208" } modify_external_catalogs: true
I20260811 18:57:36.654793 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5c323e78021f457095b0ad2754e21334"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.660002 5376 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.660195 5376 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.660997 5376 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:36.666971 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 67348us
W20260811 18:57:36.669519 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 02bf791926d84522b1099c1242acddcd consensus state has no leader
I20260811 18:57:36.670042 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0cde3a66762e4c0ea53da6ce0999bf3c" } new_table_name: "default.Table_61e3f65a0ee948fab69aa4add687f2bc" modify_external_catalogs: true
W20260811 18:57:36.678821 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b278420f8d2f4be8b973de3eb756f69a consensus state has no leader
I20260811 18:57:36.708385 4169 ts_tablet_manager.cc:1918] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:36.708786 4169 log.cc:1196] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b2718cd4a98742dfa8014a8a38c69c45
I20260811 18:57:36.709833 4169 ts_tablet_manager.cc:1939] T b2718cd4a98742dfa8014a8a38c69c45 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:36.710413 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c323e78021f457095b0ad2754e21334 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.720201 5407 raft_consensus.cc:482] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.720381 5407 raft_consensus.cc:504] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.721038 5407 leader_election.cc:288] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:36.723822 3678 leader_election.cc:334] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.725807 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4de110bd3348f0b0dae21c9f2b6a19" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:36.726557 3676 leader_election.cc:341] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.726708 3676 leader_election.cc:302] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.727368 5376 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.727519 5376 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.728749 5407 raft_consensus.cc:2726] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.738149 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 917) took 3508 ms. Trace:
I20260811 18:57:36.738481 4169 rpcz_store.cc:276] 0811 18:57:33.208421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.217588 (+ 9167us) service_pool.cc:227] Handling call
0811 18:57:36.430093 (+3212505us) ts_tablet_manager.cc:1145] Deleting tablet b2718cd4a98742dfa8014a8a38c69c45
0811 18:57:36.430105 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.612064 (+181959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.708225 (+ 96161us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.717182 (+ 8957us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200632}
I20260811 18:57:36.746029 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet fe203387bc9c45ba82afe6b058aa5e5f (table default.Table_e67c791ed9cd4da88ae7ce53ba362301 [id=14b4d412d33c43fdb3d20c85b82c1670]) successfully deleted
I20260811 18:57:36.746330 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:36.746644 3746 raft_consensus.cc:2454] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:36.747812 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6df5545532e346f7abcdc4cd228b84da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6df5545532e346f7abcdc4cd228b84da already in progress: creating tablet
I20260811 18:57:36.739970 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 918) took 3487 ms. Trace:
I20260811 18:57:36.748778 4169 rpcz_store.cc:276] 0811 18:57:33.252693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.253338 (+ 645us) service_pool.cc:227] Handling call
0811 18:57:36.739281 (+3485943us) ts_tablet_manager.cc:1145] Deleting tablet 4af28c051e8243eaa73d4a5ed1abcea6
0811 18:57:36.739342 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.739703 (+ 361us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3485044}
I20260811 18:57:36.761811 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 919) took 3385 ms. Trace:
I20260811 18:57:36.765934 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18a44d23ec2441b784693e1cbbd62ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.766201 3714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd099e75e1ca407087ce5e01c9916c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.766942 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71b6efc42de4200bee281e7e70c6bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767197 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe080f2071964da6b54f1ee5af98d918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767458 3707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7ce2b43b2134defb14d45112e4f5e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767640 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56937c88026541e4b70d4f507ce03bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.767884 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db48155ac754cd3bbac5ac2e21141e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.769447 4162 tablet_bootstrap.cc:654] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.769713 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac48afc7eec4410e8d6055ecdb794a4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.770493 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8b3713cac84544a61e8640c07e876e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.774480 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5d8c1424a045ee927b79351d236624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.774560 5511 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.767292 4169 rpcz_store.cc:276] 0811 18:57:33.374994 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:33.375058 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:36.756311 (+3381253us) ts_tablet_manager.cc:1145] Deleting tablet 9608d6ddcad749cd9e72dd6f265db0da
0811 18:57:36.756546 (+ 235us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.760458 (+ 3912us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3379861}
I20260811 18:57:36.774712 5511 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.775048 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 934) took 2774 ms. Trace:
I20260811 18:57:36.775161 4169 rpcz_store.cc:276] 0811 18:57:34.000113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.005281 (+ 5168us) service_pool.cc:227] Handling call
0811 18:57:36.774860 (+2769579us) ts_tablet_manager.cc:1145] Deleting tablet 05cba6b392b642e59321570aac3541d0
0811 18:57:36.774880 (+ 20us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.775021 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2769358}
I20260811 18:57:36.775436 5511 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.775597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 935) took 2762 ms. Trace:
I20260811 18:57:36.775694 4169 rpcz_store.cc:276] 0811 18:57:34.012880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.013941 (+ 1061us) service_pool.cc:227] Handling call
0811 18:57:36.775464 (+2761523us) ts_tablet_manager.cc:1145] Deleting tablet 101994cc708a4fadb46fc8c5915a5587
0811 18:57:36.775479 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.775577 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761307}
I20260811 18:57:36.776096 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 939) took 2513 ms. Trace:
I20260811 18:57:36.777621 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd3c95b744242ba9dee25d95d8e2b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.780313 3709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa57a509ea14c5e935b73663b159ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.786739 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65cb8d901c04d8ba2b7a40efc7fa3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.790644 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e4ec7cfec94a1988fc6c950421b48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.796878 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90ce2b983a431383edb7a9296520a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.805383 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2698f49dd4b64fce9a411b1c3307abf2" } new_table_name: "default.Table_cbae90dc1ddd458f9780ee1f8a41c264" modify_external_catalogs: true
I20260811 18:57:36.806536 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff7d3c7a5fcf4c2faec0d41371788b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.807672 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c3df98432314ff2b4cbd7004701b258" } modify_external_catalogs: true
I20260811 18:57:36.807941 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0c3df98432314ff2b4cbd7004701b258" } modify_external_catalogs: true
I20260811 18:57:36.812987 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=398 overwritten=0 applied=397 ignored=0} inserts{seen=369 ignored=0} mutations{seen=440 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:57:36.812882 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c767befd17049f8a118fdbd2a89e55d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.814429 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: Bootstrap complete.
I20260811 18:57:36.814914 5407 raft_consensus.cc:482] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.815048 5407 raft_consensus.cc:504] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.815641 5407 leader_election.cc:288] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:36.816478 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28aaee10703b4993a701a6eecf52d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.818061 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 73c519528f75423195b6bdf6683fd91b
W20260811 18:57:36.818567 3678 leader_election.cc:334] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:36.819967 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
I20260811 18:57:36.834619 4167 ts_tablet_manager.cc:1918] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.834900 4167 log.cc:1196] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7057b8a7bae249caaed304b5c66e6871
W20260811 18:57:36.835057 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c767befd17049f8a118fdbd2a89e55d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.835494 4167 ts_tablet_manager.cc:1939] T 7057b8a7bae249caaed304b5c66e6871 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.836822 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 990) took 1747 ms. Trace:
I20260811 18:57:36.837539 4167 rpcz_store.cc:276] 0811 18:57:35.089474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089549 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:36.194683 (+1105134us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:36.195239 (+ 556us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.803284 (+608045us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.833806 (+ 30522us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.836801 (+ 2995us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1104845}
I20260811 18:57:36.835753 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.838450 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.839022 4167 tablet_replica.cc:330] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:36.839145 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ad9e3991d0f4274a4770a8f264f403e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.839767 4167 raft_consensus.cc:2229] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.840332 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3ad5f39f59b8447cb3f15b9646e4b852
I20260811 18:57:36.841151 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9a431bdf0c984b91935aed8346d63c56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.841778 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a431bdf0c984b91935aed8346d63c56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.842653 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_01425dcfb75d41f6ad6349c65957ebac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.843271 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01425dcfb75d41f6ad6349c65957ebac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.845249 4167 raft_consensus.cc:2258] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.845468 4162 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:36.845614 4162 ts_tablet_manager.cc:1397] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.652s user 0.023s sys 0.026s
I20260811 18:57:36.845918 4167 ts_tablet_manager.cc:1905] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.846370 4162 raft_consensus.cc:348] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:36.846578 4162 raft_consensus.cc:374] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.846630 4162 raft_consensus.cc:729] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.846540 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fccc76d3d4284078bfd4edff47287a40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.846827 4162 consensus_queue.cc:260] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:36.847102 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fccc76d3d4284078bfd4edff47287a40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.849486 4162 ts_tablet_manager.cc:1428] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:36.854533 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0e5981cabc4e7ab53de86bb730c70a (DEFAULT_TABLE table=default.Table_2ae1b2ddf92b411f84f56e6a92130fa8 [id=4ab5b403d47a4b61a8ae7a4748f6dc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.854852 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c50dcffac54c7bac210eeaed340ca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.855206 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 1b44d53ffcc9440fa7594920a05b39a5 (DEFAULT_TABLE table=default.Table_cf1083177b984dbc8f987dc5575ae11a [id=477264043a4649afb58c7591736c35f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.776918 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:36.789129 5511 raft_consensus.cc:482] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.854873 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 6648cd60f5214830a53129a2e1cf43f4 (DEFAULT_TABLE table=default.Table_e1f8487f0094407ba50ccdaecfebe7ee [id=3fa1075aa4514eb7a0caf0428fde2faf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.855969 5511 raft_consensus.cc:504] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.789256 5417 raft_consensus.cc:482] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:36.856685 5417 raft_consensus.cc:504] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.857185 5417 leader_election.cc:288] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.795923 5376 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.857321 4167 ts_tablet_manager.cc:1918] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.857494 4167 log.cc:1196] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/2c0f6df7a291456d80538e2b76417873
I20260811 18:57:36.856593 5511 leader_election.cc:288] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.858016 4167 ts_tablet_manager.cc:1939] T 2c0f6df7a291456d80538e2b76417873 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.859324 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7cb3e997b4454598bdce102e92296887
W20260811 18:57:36.861800 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7bfa85fca4bd41dc83acb02ac610cf97 consensus state has no leader
I20260811 18:57:36.796537 5557 consensus_queue.cc:1035] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:36.864988 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:36.865394 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 31bb11617f1548f88f534289706edb06 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.804478 4163 tablet_bootstrap.cc:492] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.819051 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27cda53b7ca54346a7a22bf26f5db990" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:36.866882 4163 ts_tablet_manager.cc:1397] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.471s user 0.009s sys 0.017s
W20260811 18:57:36.867228 3676 leader_election.cc:341] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:36.867347 3676 leader_election.cc:302] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:36.867707 5407 raft_consensus.cc:2726] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:36.868005 4163 raft_consensus.cc:348] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.868268 4163 raft_consensus.cc:374] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.868341 4163 raft_consensus.cc:729] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:36.855616 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 66614.6us
I20260811 18:57:36.868717 3824 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.868537 4163 consensus_queue.cc:260] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:36.868999 3824 leader_election.cc:341] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:36.869177 5511 raft_consensus.cc:2781] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.869274 5511 raft_consensus.cc:482] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.869417 5511 raft_consensus.cc:3037] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.869441 4163 ts_tablet_manager.cc:1428] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:36.856580 4169 rpcz_store.cc:276] 0811 18:57:34.262216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.262271 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:36.775954 (+2513683us) ts_tablet_manager.cc:1145] Deleting tablet 5ce3f2f6f6ff41a2b02492fae757ffd7
0811 18:57:36.775967 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.776055 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2513445}
I20260811 18:57:36.869287 3824 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.870730 4163 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.871065 3824 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:36.871423 5557 raft_consensus.cc:2781] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.871497 5557 raft_consensus.cc:482] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.871564 5557 raft_consensus.cc:3037] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.872236 5417 raft_consensus.cc:2781] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:36.872313 5417 raft_consensus.cc:482] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:36.872373 5417 raft_consensus.cc:3037] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:36.872378 4163 tablet_bootstrap.cc:654] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.875110 5417 raft_consensus.cc:504] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.875319 5557 raft_consensus.cc:504] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.875754 5417 leader_election.cc:288] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.876025 5557 leader_election.cc:288] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.877074 3721 tablet_service.cc:1467] Processing CreateTablet for tablet eb52f30af7294caa877b3aa3be275259 (DEFAULT_TABLE table=default.Table_ef967930b08e4f7492ffa9993b43f445 [id=f0849aba28af422ca272f0757878b2dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.877518 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe879c4960b4e0cbd914f96255eff9d (DEFAULT_TABLE table=default.Table_44d5558ecc5b4b7cba0c0a120270f3eb [id=1b972bac79a94bb5af5833344e38f15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.877959 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 9708871030cc4ea6b620fb647403c42a (DEFAULT_TABLE table=default.Table_744e4342189a4b29b424eebd6ad7542f [id=5b758aa0da964e4897bcd282f9a000ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.881129 5511 raft_consensus.cc:504] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:36.882021 5511 leader_election.cc:288] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:36.882726 3706 tablet_service.cc:1467] Processing CreateTablet for tablet dfb6ac49f1d4403dad38a3125264088a (DEFAULT_TABLE table=default.Table_c12987db235f4f4fb949580479275ec2 [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.883719 5327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:36.890534 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 943) took 2618 ms. Trace:
I20260811 18:57:36.890720 4169 rpcz_store.cc:276] 0811 18:57:34.272149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.274568 (+ 2419us) service_pool.cc:227] Handling call
0811 18:57:36.870175 (+2595607us) ts_tablet_manager.cc:1145] Deleting tablet 7216c095cc03424bbbe97a52a9600ebb
0811 18:57:36.876304 (+ 6129us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.890490 (+ 14186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2595411}
I20260811 18:57:36.892050 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3b056c714ed947198f75f2c7802ae4dd" } new_table_name: "default.Table_69bcdc9302e4490fb9ff40912f5d05b3" modify_external_catalogs: true
I20260811 18:57:36.894140 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_33c7b91319d04649821da27a7798271a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.895205 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9db0e2b871fd484f82e42660d2a09c55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:36.895812 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9db0e2b871fd484f82e42660d2a09c55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:36.897104 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33c7b91319d04649821da27a7798271a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.899861 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 7f4e1fc5686944dfa3eb57702c810eb8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:36.907940 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 8f9b279c6c7344f1931b81e10248b1b6
I20260811 18:57:36.908969 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2962915ef36940f8b6dee3a420d74c59" } new_table_name: "default.Table_b24e66bb2c8b41a582869a9ee02d6745" modify_external_catalogs: true
I20260811 18:57:36.919188 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0e5981cabc4e7ab53de86bb730c70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.921500 4162 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:36.923043 4162 tablet_bootstrap.cc:654] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:36.923965 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 52935.2us
I20260811 18:57:36.925225 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 992) took 1835 ms. Trace:
I20260811 18:57:36.925335 4167 rpcz_store.cc:276] 0811 18:57:35.089692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089881 (+ 189us) service_pool.cc:227] Handling call
0811 18:57:36.838968 (+1749087us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:36.838981 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.851221 (+ 12240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.857206 (+ 5985us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.925203 (+ 67997us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1748945}
I20260811 18:57:36.925729 4167 tablet_replica.cc:330] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.925977 4167 raft_consensus.cc:2229] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.926137 4167 raft_consensus.cc:2258] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.926762 4167 ts_tablet_manager.cc:1905] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.928128 5417 consensus_queue.cc:1035] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:36.928258 3824 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:36.929114 3860 tablet_service.cc:1467] Processing CreateTablet for tablet dfb6ac49f1d4403dad38a3125264088a (DEFAULT_TABLE table=default.Table_c12987db235f4f4fb949580479275ec2 [id=6c0755efd2154b06961f6388a9452316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:36.929849 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet d00d943b778741b4aeec076ba46a9c40 with delete_type TABLET_DATA_DELETED (replaced by tablet ccc63b46a4e346f79d1e03a82490794c) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.930428 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c519528f75423195b6bdf6683fd91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.930953 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.932406 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.932866 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb3e997b4454598bdce102e92296887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.933328 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a7051bc539468aafba30fbaf0669d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:25 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.933761 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:36.936892 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bb11617f1548f88f534289706edb06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937175 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb52f30af7294caa877b3aa3be275259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937410 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe879c4960b4e0cbd914f96255eff9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.937608 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6648cd60f5214830a53129a2e1cf43f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.940994 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b44d53ffcc9440fa7594920a05b39a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:36.943636 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2523fa07587246d994eca722c7032105 consensus state has no leader
I20260811 18:57:36.944106 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_83c2a159da7a4474bd29c31afe70760e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:36.944958 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9708871030cc4ea6b620fb647403c42a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.945063 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_85a94fea83f84948a14090e2deeac956" } new_table_name: "default.Table_34fae463f93a44a2a142745f939fbf2b" modify_external_catalogs: true
I20260811 18:57:36.948592 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb6ac49f1d4403dad38a3125264088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.954025 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb6ac49f1d4403dad38a3125264088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:36.954200 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c519528f75423195b6bdf6683fd91b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:36.954311 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet fa6509e905694f979191469d20e1de41 (table default.Table_4ecec4051b794418b73f208e526c9bbb [id=b9b4076a00c247e38bbe4223c548d8d2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fa6509e905694f979191469d20e1de41 consensus state has no leader
W20260811 18:57:36.958228 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83c2a159da7a4474bd29c31afe70760e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:36.960975 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.964156 5327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb005c681cf42d4ac278ec73a28e4a9, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.965165 5327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 397, Last appended: 1.398, Last appended by leader: 398, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:36.977943 5327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:36.979192 4167 ts_tablet_manager.cc:1918] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:36.979465 4167 log.cc:1196] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:36.979947 4169 tablet_replica.cc:330] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:36.980099 4167 ts_tablet_manager.cc:1939] T 8cce97bfdb1d41e685e091c8c5cc6946 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:36.980213 4169 raft_consensus.cc:2229] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.975515 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:36.981375 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:36.981559 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 993) took 1891 ms. Trace:
I20260811 18:57:36.981786 4167 rpcz_store.cc:276] 0811 18:57:35.089798 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.089959 (+ 161us) service_pool.cc:227] Handling call
0811 18:57:36.925660 (+1835701us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:36.925675 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.929996 (+ 4321us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.979075 (+ 49079us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.981427 (+ 2352us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1835514}
I20260811 18:57:36.982295 4167 tablet_replica.cc:330] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:36.982595 4167 raft_consensus.cc:2229] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:36.982774 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ad5f39f59b8447cb3f15b9646e4b852 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:36.982838 4167 raft_consensus.cc:2258] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.983559 4167 ts_tablet_manager.cc:1905] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.984694 4163 tablet_bootstrap.cc:492] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:36.984881 4163 ts_tablet_manager.cc:1397] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.114s user 0.022s sys 0.018s
I20260811 18:57:36.985683 4163 raft_consensus.cc:348] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.985909 4163 raft_consensus.cc:374] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:36.985958 4163 raft_consensus.cc:729] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:36.986182 4163 consensus_queue.cc:260] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:36.986857 4163 ts_tablet_manager.cc:1428] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:36.987095 4163 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:36.988881 4163 tablet_bootstrap.cc:654] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:36.980415 4169 raft_consensus.cc:2258] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:36.994213 4169 ts_tablet_manager.cc:1905] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:36.999770 4167 ts_tablet_manager.cc:1918] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.000042 4167 log.cc:1196] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:37.000953 4167 ts_tablet_manager.cc:1939] T ce94d37efd2245bf8cc71fdd90bd9735 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.002008 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 994) took 1912 ms. Trace:
I20260811 18:57:37.002226 4167 rpcz_store.cc:276] 0811 18:57:35.089906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.090187 (+ 281us) service_pool.cc:227] Handling call
0811 18:57:36.982248 (+1892061us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:36.982258 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.990750 (+ 8492us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:36.999632 (+ 8882us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.001981 (+ 2349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1891924}
I20260811 18:57:37.003090 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 999) took 1911 ms. Trace:
I20260811 18:57:37.003250 4167 rpcz_store.cc:276] 0811 18:57:35.091592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091643 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:37.002923 (+1911280us) ts_tablet_manager.cc:1145] Deleting tablet 1c1896700e724204a37439f134a4d988
0811 18:57:37.002937 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.003071 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911119}
I20260811 18:57:37.003782 4167 tablet_replica.cc:330] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.004067 4167 raft_consensus.cc:2229] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.004297 4167 raft_consensus.cc:2258] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.005016 4167 ts_tablet_manager.cc:1905] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.010959 4162 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.011233 4162 ts_tablet_manager.cc:1397] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.090s user 0.003s sys 0.000s
I20260811 18:57:37.012946 4167 ts_tablet_manager.cc:1918] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.013173 4167 log.cc:1196] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:37.013826 4167 ts_tablet_manager.cc:1939] T 0cccebfc54484a9b9abffcce4e459fb5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.015525 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 991) took 1925 ms. Trace:
I20260811 18:57:37.015708 4167 rpcz_store.cc:276] 0811 18:57:35.089591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091874 (+ 2283us) service_pool.cc:227] Handling call
0811 18:57:37.003733 (+1911859us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:37.003745 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.008691 (+ 4946us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.012829 (+ 4138us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.015482 (+ 2653us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1911699}
I20260811 18:57:37.016533 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1001) took 1924 ms. Trace:
I20260811 18:57:37.016709 4167 rpcz_store.cc:276] 0811 18:57:35.091887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.091927 (+ 40us) service_pool.cc:227] Handling call
0811 18:57:37.016373 (+1924446us) ts_tablet_manager.cc:1145] Deleting tablet afa57ed239ac4afebe517cc48d879645
0811 18:57:37.016385 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.016512 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924283}
I20260811 18:57:37.017356 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1000) took 1925 ms. Trace:
I20260811 18:57:37.017508 4167 rpcz_store.cc:276] 0811 18:57:35.091691 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092061 (+ 370us) service_pool.cc:227] Handling call
0811 18:57:37.017217 (+1925156us) ts_tablet_manager.cc:1145] Deleting tablet 4dd203b7eb8643f493b4642c0396e258
0811 18:57:37.017229 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.017339 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924942}
I20260811 18:57:37.018042 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1002) took 1926 ms. Trace:
I20260811 18:57:37.018190 4167 rpcz_store.cc:276] 0811 18:57:35.092046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092310 (+ 264us) service_pool.cc:227] Handling call
0811 18:57:37.017933 (+1925623us) ts_tablet_manager.cc:1145] Deleting tablet b27f9dbb52f34687a33c2ebf20e8f780
0811 18:57:37.017943 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.018028 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1925504}
I20260811 18:57:37.018639 4167 tablet_replica.cc:330] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.018900 4167 raft_consensus.cc:2229] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.019129 4167 raft_consensus.cc:2258] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.020871 4167 ts_tablet_manager.cc:1905] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:37.021548 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 85652.2us
I20260811 18:57:37.022637 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" } modify_external_catalogs: true
I20260811 18:57:37.022678 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_28f0027f9b254195878661731f172f78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.022895 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_498524390fc04da5b952cdc86fe5decd" } modify_external_catalogs: true
W20260811 18:57:37.023331 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28f0027f9b254195878661731f172f78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.024983 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" } modify_external_catalogs: true
I20260811 18:57:37.025220 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b46915474a4146eebe6a8da8b75a4160" } modify_external_catalogs: true
I20260811 18:57:37.025918 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b2718cd4a98742dfa8014a8a38c69c45 (table default.Table_1662a43ab5334f8689e0d7e654519650 [id=cdb9b1499b924d41b8e4a872a6dc20da]) successfully deleted
W20260811 18:57:37.026520 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4af28c051e8243eaa73d4a5ed1abcea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4af28c051e8243eaa73d4a5ed1abcea6 already in progress: creating tablet
W20260811 18:57:37.027271 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5193d57970f2486795256698cdf51e6a consensus state has no leader
I20260811 18:57:37.027786 4163 tablet_bootstrap.cc:492] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.027981 4163 ts_tablet_manager.cc:1397] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.001s
W20260811 18:57:37.031265 3678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 144 similar messages]
I20260811 18:57:37.031352 4167 ts_tablet_manager.cc:1918] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.031287 4163 raft_consensus.cc:348] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.031509 4163 raft_consensus.cc:374] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.031540 4167 log.cc:1196] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0bfb54175c834517b7af291c942333cd
I20260811 18:57:37.031566 4163 raft_consensus.cc:729] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.031777 4163 consensus_queue.cc:260] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.031960 4167 ts_tablet_manager.cc:1939] T 0bfb54175c834517b7af291c942333cd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.032402 4163 ts_tablet_manager.cc:1428] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:37.032722 4163 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.034277 4163 tablet_bootstrap.cc:654] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.038614 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0cdd7e4b15384ec8844f2e31a5496d8a consensus state has no leader
I20260811 18:57:37.040246 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 998) took 1948 ms. Trace:
I20260811 18:57:37.040449 4167 rpcz_store.cc:276] 0811 18:57:35.091498 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.092542 (+ 1044us) service_pool.cc:227] Handling call
0811 18:57:37.018597 (+1926055us) ts_tablet_manager.cc:1145] Deleting tablet 0bfb54175c834517b7af291c942333cd
0811 18:57:37.018607 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.025884 (+ 7277us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.031213 (+ 5329us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.040217 (+ 9004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1925910}
I20260811 18:57:37.040652 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:37.045743 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9608d6ddcad749cd9e72dd6f265db0da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9608d6ddcad749cd9e72dd6f265db0da already in progress: creating tablet
I20260811 18:57:37.044847 4162 raft_consensus.cc:348] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.049124 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 05cba6b392b642e59321570aac3541d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05cba6b392b642e59321570aac3541d0 already in progress: creating tablet
I20260811 18:57:37.050031 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:37.050321 4162 raft_consensus.cc:374] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.050441 4162 raft_consensus.cc:729] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.047686 4167 tablet_replica.cc:330] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.051267 4167 raft_consensus.cc:2229] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.051453 4167 raft_consensus.cc:2258] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.051713 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5aa83c98b49c466cb01cdf92154ad384" } modify_external_catalogs: true
I20260811 18:57:37.051965 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5aa83c98b49c466cb01cdf92154ad384" } modify_external_catalogs: true
I20260811 18:57:37.052119 4167 ts_tablet_manager.cc:1905] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:37.055528 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 101994cc708a4fadb46fc8c5915a5587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 101994cc708a4fadb46fc8c5915a5587 already in progress: creating tablet
I20260811 18:57:37.050695 4162 consensus_queue.cc:260] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.060976 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:37.061220 3745 raft_consensus.cc:2454] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:37.066664 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:37.068222 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb3e997b4454598bdce102e92296887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:37.069195 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ce3f2f6f6ff41a2b02492fae757ffd7 already in progress: creating tablet
I20260811 18:57:37.062675 4162 ts_tablet_manager.cc:1428] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.051s user 0.002s sys 0.001s
I20260811 18:57:37.071342 4162 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.071462 4163 tablet_bootstrap.cc:492] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.071615 4163 ts_tablet_manager.cc:1397] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20260811 18:57:37.072695 4163 raft_consensus.cc:348] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.072906 4163 raft_consensus.cc:374] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.072961 4163 raft_consensus.cc:729] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.072798 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2c977f4c94664fcb88bb338b5c79edfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.072928 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a1cb848dc1c1438aa868e39197e05009" } new_table_name: "default.Table_236ea19297e44d6ea717601dbb1a318b" modify_external_catalogs: true
I20260811 18:57:37.073184 4163 consensus_queue.cc:260] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.073450 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c977f4c94664fcb88bb338b5c79edfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.073865 4163 ts_tablet_manager.cc:1428] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:37.074270 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_772fdfa6598e4b2ca2aa3a4b28746b24" } new_table_name: "default.Table_075181f96b7b42059289018e831e2671" modify_external_catalogs: true
I20260811 18:57:37.074747 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 7f4e1fc5686944dfa3eb57702c810eb8
I20260811 18:57:37.074975 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fccc76d3d4284078bfd4edff47287a40" } modify_external_catalogs: true
I20260811 18:57:37.075356 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fccc76d3d4284078bfd4edff47287a40" } modify_external_catalogs: true
I20260811 18:57:37.075462 4163 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.075520 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ea579c33f8b14e9fa7fa389ce7f45931
I20260811 18:57:37.076016 3548 catalog_manager.cc:6565] ReplaceTablet: tablet 7f4e1fc5686944dfa3eb57702c810eb8 deleted and replaced by tablet b192929b6d154885a0cc44c00f1add8e
I20260811 18:57:37.076206 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7057b8a7bae249caaed304b5c66e6871 (table default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]) successfully deleted
W20260811 18:57:37.076647 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7216c095cc03424bbbe97a52a9600ebb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7216c095cc03424bbbe97a52a9600ebb already in progress: creating tablet
I20260811 18:57:37.071395 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:37.072371 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61cea7b0b85448f5854e8fd99ea5e792" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.078099 3745 raft_consensus.cc:3037] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.072512 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f05e1d5cb947fe9dbcc2e876a23788" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.078264 3740 raft_consensus.cc:3037] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.080873 3740 raft_consensus.cc:2454] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.080994 3745 raft_consensus.cc:2454] T 61cea7b0b85448f5854e8fd99ea5e792 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.075767 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f3c4536fc7440db83851fc8d7b3de10" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.081493 3743 raft_consensus.cc:3037] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.083768 4162 tablet_bootstrap.cc:654] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.086907 4163 tablet_bootstrap.cc:654] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.089295 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 2c0f6df7a291456d80538e2b76417873 (table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]) successfully deleted
I20260811 18:57:37.090605 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8cce97bfdb1d41e685e091c8c5cc6946 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
I20260811 18:57:37.098848 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ce94d37efd2245bf8cc71fdd90bd9735 (table default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]) successfully deleted
W20260811 18:57:37.099189 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1c1896700e724204a37439f134a4d988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c1896700e724204a37439f134a4d988 already in progress: creating tablet
I20260811 18:57:37.099560 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0cccebfc54484a9b9abffcce4e459fb5 (table default.Table_d88f576cc3ca4c6eadc6e071aacc15b2 [id=5fc5ade3f38c438d9b501127ac866b88]) successfully deleted
W20260811 18:57:37.099783 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet afa57ed239ac4afebe517cc48d879645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa57ed239ac4afebe517cc48d879645 already in progress: creating tablet
W20260811 18:57:37.100032 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4dd203b7eb8643f493b4642c0396e258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4dd203b7eb8643f493b4642c0396e258 already in progress: creating tablet
W20260811 18:57:37.100275 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b27f9dbb52f34687a33c2ebf20e8f780 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b27f9dbb52f34687a33c2ebf20e8f780 already in progress: creating tablet
I20260811 18:57:37.100473 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0bfb54175c834517b7af291c942333cd (table default.Table_fe2f572c66cd4b9a8656e0fd439d50c7 [id=675b022008f640a28f26cb57134b70b0]) successfully deleted
I20260811 18:57:37.103860 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:37.104127 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:37.108741 4163 tablet_bootstrap.cc:492] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.108927 4163 ts_tablet_manager.cc:1397] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:37.109790 4163 raft_consensus.cc:348] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.110065 4163 raft_consensus.cc:374] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.110142 4163 raft_consensus.cc:729] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.110369 4163 consensus_queue.cc:260] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.110795 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.084043 3743 raft_consensus.cc:2454] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.101410 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9b279c6c7344f1931b81e10248b1b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:36 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.113791 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e" } modify_external_catalogs: true
I20260811 18:57:37.111059 4163 ts_tablet_manager.cc:1428] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:37.116528 3824 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.118716 3824 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.118856 3824 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.121697 3717 tablet_service.cc:1467] Processing CreateTablet for tablet eb2ce551cae74423ba2a7b521fe1e44f (DEFAULT_TABLE table=default.Table_7249f865dd9d4e0fa204c278006d4cee [id=ce14e61175f7486aa71cabde45e4fc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.121974 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 026245dfb0284d95a1afae43c77074c5 (DEFAULT_TABLE table=default.Table_def64923a6b24aa0a7b9ac4a66dcace5 [id=1e2eeed8d9aa4ce0935705782a7a7e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.122177 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2ce551cae74423ba2a7b521fe1e44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.122423 4163 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.122505 3721 tablet_service.cc:1467] Processing CreateTablet for tablet f5a969dbd34f414ba3a0b45748510632 (DEFAULT_TABLE table=default.Table_1ec52777b5484eb18531a70c114a32f6 [id=0d5ee633d4ab43a58bf9009459ee7699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.122825 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a969dbd34f414ba3a0b45748510632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.124181 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d21973bc8e514a089c2beb24a686060f
I20260811 18:57:37.124931 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 2f2f5ce2b1d14a9a988cff644e196088
I20260811 18:57:37.125514 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0084f6fe30164d4d9dcc8ad20699380e" } modify_external_catalogs: true
I20260811 18:57:37.126631 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 38f64f32463148349a2b0d403ae881e9 (DEFAULT_TABLE table=default.Table_cce74eefcdbb44369f1915d3f40f54e2 [id=43162c1c44f442fb912c654af76eac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.127101 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026245dfb0284d95a1afae43c77074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.127511 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f64f32463148349a2b0d403ae881e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.128603 4163 tablet_bootstrap.cc:654] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.132777 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet a3973dfdaac746a6af7a402277d2711b from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.134048 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 20c4f222b5054527b18f2874dacc5558 (DEFAULT_TABLE table=default.Table_b807102dd88842e2b6e466a147c5b675 [id=c97451803afd4beab9583182c36fbac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.134472 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20c4f222b5054527b18f2874dacc5558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.135169 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 558dc0f1c46840d0991d08d792f903d5 (DEFAULT_TABLE table=default.Table_458dd09db5894e0fae4cc31deb113bdd [id=aa118bfdb3b440a1bc81ca195f425546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135419 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 5baa02ab900a4a8893c8e40d6ae5cf58 (DEFAULT_TABLE table=default.Table_7407d7434a3b4290889506a933ae2f05 [id=fe40a820d5cd401c8fc8393b02de9689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135957 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 3edf59b74ee245be852ece5ba50a1054 (DEFAULT_TABLE table=default.Table_6aacb785fd444621820be42007d55668 [id=996a202868ee4f6dadf4be343005550a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.135515 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558dc0f1c46840d0991d08d792f903d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.136770 4169 ts_tablet_manager.cc:1918] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.136885 4169 log.cc:1196] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/43ad6993b63e4bbd98c8cc52f1a53451
I20260811 18:57:37.137182 4169 ts_tablet_manager.cc:1939] T 43ad6993b63e4bbd98c8cc52f1a53451 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.137578 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 950) took 2841 ms. Trace:
I20260811 18:57:37.137670 4169 rpcz_store.cc:276] 0811 18:57:34.296426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.296581 (+ 155us) service_pool.cc:227] Handling call
0811 18:57:36.891109 (+2594528us) ts_tablet_manager.cc:1145] Deleting tablet 43ad6993b63e4bbd98c8cc52f1a53451
0811 18:57:36.979870 (+ 88761us) spinlock_profiling.cc:243] Waited 84.2 ms on lock 0x618000005118. stack: 00007f4d0910266e 00007f4d09102430 00007f4d081895ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f4d090f80fd 00007f4d090f916e 00007f4d1c04dcfd 00007f4d1bfcb883 00007f4d1826b6e6 00007f4d1826ca37 00007f4d1829f404 00007f4d18280870 00007f4d182805fd 00000000009edb7c
0811 18:57:36.979886 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:36.997662 (+ 17776us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.136659 (+138997us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.137558 (+ 899us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":193548160,"tablet-delete.queue_time_us":2594307}
I20260811 18:57:37.138190 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 948) took 2844 ms. Trace:
I20260811 18:57:37.140309 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a385abaeb70347dcac06a5cedbdddeb4 (DEFAULT_TABLE table=default.Table_5437383c0bbb4384825f578e83c10acc [id=78cd09fbad264f7385b142dcf7ff57b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.140679 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a385abaeb70347dcac06a5cedbdddeb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.142446 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3edf59b74ee245be852ece5ba50a1054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.144925 5557 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.145313 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:37.146224 3824 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.146466 3824 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.146646 3824 leader_election.cc:334] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.146724 3824 leader_election.cc:302] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.149000 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5baa02ab900a4a8893c8e40d6ae5cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.151755 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2afc6c2e3a254c57840c4e891d796bf5" } new_table_name: "default.Table_3f202b05f18d4434bb51caa2cd6d23b8" modify_external_catalogs: true
I20260811 18:57:37.152149 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1ec50daf17974aef9e292bab9fc018cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.152814 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ec50daf17974aef9e292bab9fc018cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.154618 5557 raft_consensus.cc:2726] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.155756 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb6981748ef0423aaaa9a85bacef95be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.156366 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb6981748ef0423aaaa9a85bacef95be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.157218 3553 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f13412c5533e4d9ebb03d46521d325ea
W20260811 18:57:37.157351 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18a44d23ec2441b784693e1cbbd62ad8 consensus state has no leader
I20260811 18:57:37.157639 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c364560f47804860afd5635ea9b6885d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.157784 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_faccbd3f194043708dbbd12a50f037cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.158214 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_faccbd3f194043708dbbd12a50f037cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.158214 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c364560f47804860afd5635ea9b6885d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.158972 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1365655a76d7454c8415102e75960152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.159518 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1365655a76d7454c8415102e75960152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.160760 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:37.161374 5557 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.161514 5557 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.162153 5557 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.163656 3546 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
I20260811 18:57:37.168506 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_606ca53b362a44b4ba72954536b85633"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.169100 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_606ca53b362a44b4ba72954536b85633 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.170614 3824 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:37.172700 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa5a3d1ada554d778d507382c8836133 consensus state has no leader
I20260811 18:57:37.173540 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f2b66196716a4ba3893b6b8bfec2b8c0
I20260811 18:57:37.174399 5557 raft_consensus.cc:2781] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:37.174504 5557 raft_consensus.cc:482] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:37.174595 5557 raft_consensus.cc:3037] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.174940 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9485300898e145ba9645d6e5ede8d7f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.175549 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9485300898e145ba9645d6e5ede8d7f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.138266 4169 rpcz_store.cc:276] 0811 18:57:34.293992 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.297681 (+ 3689us) service_pool.cc:227] Handling call
0811 18:57:37.138054 (+2840373us) ts_tablet_manager.cc:1145] Deleting tablet 7be6d9578e7847c2822278486119390c
0811 18:57:37.138066 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.138173 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2840182}
I20260811 18:57:37.177310 5557 raft_consensus.cc:504] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.178560 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 55f97f37fa7c483cacaeed54f93e7a24 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.177913 5557 leader_election.cc:288] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.177678 4169 tablet_replica.cc:330] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.179741 4163 tablet_bootstrap.cc:492] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.179890 4163 ts_tablet_manager.cc:1397] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.000s
I20260811 18:57:37.180387 4167 ts_tablet_manager.cc:1918] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.179792 4169 raft_consensus.cc:2229] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.180702 4163 raft_consensus.cc:348] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.180891 4163 raft_consensus.cc:374] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.180953 4163 raft_consensus.cc:729] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.181103 4169 pending_rounds.cc:62] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:37.181190 4169 pending_rounds.cc:68] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400179772579840 op_type: NO_OP noop_request { }
W20260811 18:57:37.178697 3824 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:37.181161 4163 consensus_queue.cc:260] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:37.181560 3824 leader_election.cc:341] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 968fa8b8912b40a8bdba77db1d39c5c0
W20260811 18:57:37.181744 3824 leader_election.cc:341] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:37.181900 3824 leader_election.cc:302] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.182271 4163 ts_tablet_manager.cc:1428] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.182667 4163 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.181406 4169 raft_consensus.cc:2866] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:37.184665 4169 raft_consensus.cc:2258] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.185238 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 764f87ff39df4706bc05e496fa2c7f29 (DEFAULT_TABLE table=default.Table_8edcf621b1064a75906a7ffe63e1a9d0 [id=673f9c8b623b4a8f8f34608f0487ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.185616 4163 tablet_bootstrap.cc:654] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.185717 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764f87ff39df4706bc05e496fa2c7f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.187810 3719 tablet_service.cc:1467] Processing CreateTablet for tablet dcad9c8052af4cbda4926c7c9430ea2a (DEFAULT_TABLE table=default.Table_d3410fcb02604cdabfae4dfa57a0bac0 [id=8489eaa51fe9406b94e1a7054e307bbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.188822 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcad9c8052af4cbda4926c7c9430ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.189055 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.189110 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 with delete_type TABLET_DATA_DELETED (replaced by tablet b192929b6d154885a0cc44c00f1add8e) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.189370 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 7671139202af4a17976295f521828650 (DEFAULT_TABLE table=default.Table_15068805ee264778ac398274e74d630a [id=3744324baeba44dab08e66b32215651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.189402 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 31df6bcee1fb4324b20384bf3d3edee7 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190018 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc649a70b624d0cb6839f65000dacf9 (DEFAULT_TABLE table=default.Table_8752a36a67ad4fb380132326a813d172 [id=7259d5d4e7764c1b8d3a9d286f7bbd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190410 3712 tablet_service.cc:1467] Processing CreateTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 (DEFAULT_TABLE table=default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e [id=920721dee68b404bb09a2e453baec9c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190748 3720 tablet_service.cc:1467] Processing CreateTablet for tablet 3cdaa303c70a44a19658664c8c9f9dd5 (DEFAULT_TABLE table=default.Table_ec98c10d285144029e63c99243d4d9fe [id=b73816870ec0459ca8092fcde1a27fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190662 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc84fd57013443ca0a0e7752a213566 (DEFAULT_TABLE table=default.Table_6f167c64b15d4d18a642c45b00f75658 [id=1a44fd5ad046463dbd4d84fb4e154005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190699 3716 tablet_service.cc:1467] Processing CreateTablet for tablet f10e19a2978a4a6799064cb36295c93c (DEFAULT_TABLE table=default.Table_7811427611da4cd1a27acc552c72c374 [id=b92c51f24dec492d8735bfb37ead3523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190699 3708 tablet_service.cc:1467] Processing CreateTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 (DEFAULT_TABLE table=default.Table_0084f6fe30164d4d9dcc8ad20699380e [id=48c5b345af51445eac883afc0ae3704f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.190500 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1e182c4df64e7f9fb1ffb452046f7f (DEFAULT_TABLE table=default.Table_3f83f2ec5f2847c9820675f2f29f0674 [id=0428c361975e441a9d8a4e228aeb0a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.191093 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet d21973bc8e514a089c2beb24a686060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.190148 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 3c55a7b27133454881bcddc544c22faa (DEFAULT_TABLE table=default.Table_9c421939f1354150bddc1156b40cf5fe [id=5063c88916b845fcb99d66fd8a45f042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.191548 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31df6bcee1fb4324b20384bf3d3edee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.191514 3724 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.191850 3712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6710084f21734dea9c7ce1f43d1bf587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.191990 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cdaa303c70a44a19658664c8c9f9dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192139 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc84fd57013443ca0a0e7752a213566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192466 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b66196716a4ba3893b6b8bfec2b8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.192975 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10e19a2978a4a6799064cb36295c93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.193373 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc649a70b624d0cb6839f65000dacf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.194895 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdbe37b812846fdb089ec081000125c (DEFAULT_TABLE table=default.Table_9df97a2d74d040ef8f3dd56e41b8c28b [id=e2a3e504be604e95af18e61f23c07911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.196645 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 10bc2fcd64574327acf04347fa9ca0f4 (DEFAULT_TABLE table=default.Table_fa66e20bff3745ca961cfec0dc3a65c1 [id=f91b4d545c044e87a53411a16e80c42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.187983 4167 log.cc:1196] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cf6c8d542ae2489ba466ffadd518db12
I20260811 18:57:37.199440 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1e182c4df64e7f9fb1ffb452046f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.199635 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7671139202af4a17976295f521828650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.199833 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c55a7b27133454881bcddc544c22faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.200795 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 5143412d3c2349fcb97dc527891b49e4 (DEFAULT_TABLE table=default.Table_153744b340b44d8c9e939deedadaced9 [id=da2277b2b7e94f26a71d0e68075766d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.186627 5557 raft_consensus.cc:2781] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.201195 5557 raft_consensus.cc:686] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.201485 5557 consensus_queue.cc:237] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.204388 4167 ts_tablet_manager.cc:1939] T cf6c8d542ae2489ba466ffadd518db12 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.204500 3712 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.204784 3718 tablet_service.cc:1467] Processing CreateTablet for tablet b9f31daf440647949978feb54b97543d (DEFAULT_TABLE table=default.Table_767c174a258144229f562c9dda12392b [id=4ef58bf17e6b402b96fcd79052cd35dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.207845 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdbe37b812846fdb089ec081000125c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.186826 3824 leader_election.cc:302] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.210718 5511 raft_consensus.cc:2781] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.200917 4169 ts_tablet_manager.cc:1905] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.210883 5511 raft_consensus.cc:686] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.211114 5511 consensus_queue.cc:237] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.210601 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bc2fcd64574327acf04347fa9ca0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.226027 4162 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.226182 4162 ts_tablet_manager.cc:1397] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.155s user 0.004s sys 0.000s
I20260811 18:57:37.227031 4162 raft_consensus.cc:348] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.227252 4162 raft_consensus.cc:374] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.227305 4162 raft_consensus.cc:729] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.227531 4162 consensus_queue.cc:260] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.229245 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0790be2c84c14fb3aeb2add67667646e" } new_table_name: "default.Table_293a5368b1724dea8e5591b378e82995" modify_external_catalogs: true
I20260811 18:57:37.230702 4162 ts_tablet_manager.cc:1428] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260811 18:57:37.231689 3824 consensus_peers.cc:487] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b57ed7814a1e440cba362613cbab32f8. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.232072 3824 leader_election.cc:302] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.232327 5376 raft_consensus.cc:2781] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.239619 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
I20260811 18:57:37.246719 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5143412d3c2349fcb97dc527891b49e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.247170 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.248267 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.257578 4162 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.258368 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1020) took 2131 ms. Trace:
I20260811 18:57:37.258509 4167 rpcz_store.cc:276] 0811 18:57:35.126861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.126903 (+ 42us) service_pool.cc:227] Handling call
0811 18:57:37.041544 (+1914641us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:37.041558 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.107400 (+ 65842us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.180247 (+ 72847us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.258318 (+ 78071us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1914342}
I20260811 18:57:37.259001 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1018) took 2132 ms. Trace:
I20260811 18:57:37.259109 4167 rpcz_store.cc:276] 0811 18:57:35.126705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.126774 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:37.258868 (+2132094us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:37.258878 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.258982 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2131338}
I20260811 18:57:37.259495 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1019) took 2132 ms. Trace:
I20260811 18:57:37.259593 4167 rpcz_store.cc:276] 0811 18:57:35.126797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.130400 (+ 3603us) service_pool.cc:227] Handling call
0811 18:57:37.259372 (+2128972us) ts_tablet_manager.cc:1145] Deleting tablet f86bfa65a8774b51bc7e7fe59442bf2f
0811 18:57:37.259385 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.259480 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2128775}
I20260811 18:57:37.259954 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1022) took 2018 ms. Trace:
I20260811 18:57:37.259986 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b325df1db4fde9d4b26b559121316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.260035 4167 rpcz_store.cc:276] 0811 18:57:35.241228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.241290 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:37.259813 (+2018523us) ts_tablet_manager.cc:1145] Deleting tablet f1342b8cd65b4c31a47b27a79c678d2b
0811 18:57:37.259827 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.259940 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2018287}
I20260811 18:57:37.260357 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1023) took 2018 ms. Trace:
I20260811 18:57:37.260443 4167 rpcz_store.cc:276] 0811 18:57:35.241879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.241932 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.260241 (+2018309us) ts_tablet_manager.cc:1145] Deleting tablet e5f47ccc193d48efa15d60a06b148e00
0811 18:57:37.260249 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.260342 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2018131}
I20260811 18:57:37.260802 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1024) took 2015 ms. Trace:
I20260811 18:57:37.260910 4167 rpcz_store.cc:276] 0811 18:57:35.245320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.245377 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:37.260681 (+2015304us) ts_tablet_manager.cc:1145] Deleting tablet fcb1c9787d3f4fd181f86deca28ad160
0811 18:57:37.260695 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.260786 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2015098}
I20260811 18:57:37.261183 4167 tablet_replica.cc:330] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.261392 4167 raft_consensus.cc:2229] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.261572 4167 raft_consensus.cc:2258] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.262480 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 20c4f222b5054527b18f2874dacc5558 (DEFAULT_TABLE table=default.Table_b807102dd88842e2b6e466a147c5b675 [id=c97451803afd4beab9583182c36fbac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.262948 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20c4f222b5054527b18f2874dacc5558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.265319 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a3973dfdaac746a6af7a402277d2711b
I20260811 18:57:37.266134 3547 catalog_manager.cc:6565] ReplaceTablet: tablet a3973dfdaac746a6af7a402277d2711b deleted and replaced by tablet 16db56987b2c44c9b5d3445073226ab9
I20260811 18:57:37.266388 3555 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:37.266728 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_755a5ecf2290451b8befb8e19c0cda4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.266899 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ace3c7ab7e542c6a3c5a5f1aa1d1f5b" } new_table_name: "default.Table_161c7e0df7b04150bf644353aac4604e" modify_external_catalogs: true
W20260811 18:57:37.267468 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_755a5ecf2290451b8befb8e19c0cda4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.272868 4167 ts_tablet_manager.cc:1905] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.276481 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f31daf440647949978feb54b97543d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.279004 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" } modify_external_catalogs: true
I20260811 18:57:37.279250 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_19742da9decc46818ae2cd5debeee68c" } modify_external_catalogs: true
W20260811 18:57:37.286058 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4f21761b37b42dbb814bd54cca89452 consensus state has no leader
I20260811 18:57:37.286651 5376 raft_consensus.cc:686] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.287096 5376 consensus_queue.cc:237] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.289307 4167 ts_tablet_manager.cc:1918] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:37.294714 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 61f4112fcb7445b3803456e59a24ad0c consensus state has no leader
I20260811 18:57:37.296682 4167 log.cc:1196] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:37.303375 4167 ts_tablet_manager.cc:1939] T 7b404e99e9414c67b3cd5aff12203a53 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.305589 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1021) took 2065 ms. Trace:
I20260811 18:57:37.306528 4167 rpcz_store.cc:276] 0811 18:57:35.240249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.256817 (+ 16568us) service_pool.cc:227] Handling call
0811 18:57:37.261136 (+2004319us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:37.261150 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.276876 (+ 15726us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.289196 (+ 12320us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.305559 (+ 16363us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2004046}
I20260811 18:57:37.309882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1025) took 2017 ms. Trace:
I20260811 18:57:37.310055 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:37.311770 3860 tablet_service.cc:1467] Processing CreateTablet for tablet eb2ce551cae74423ba2a7b521fe1e44f (DEFAULT_TABLE table=default.Table_7249f865dd9d4e0fa204c278006d4cee [id=ce14e61175f7486aa71cabde45e4fc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.312248 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2ce551cae74423ba2a7b521fe1e44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.312963 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 15dd14e2b4bd4a08b5b08f532a55291b
I20260811 18:57:37.314217 4167 rpcz_store.cc:276] 0811 18:57:35.292084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.292497 (+ 413us) service_pool.cc:227] Handling call
0811 18:57:37.309717 (+2017220us) ts_tablet_manager.cc:1145] Deleting tablet 302cf69ea7c14fdaac72f6a9a9aff113
0811 18:57:37.309730 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.309860 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2017004}
I20260811 18:57:37.317173 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1026) took 1910 ms. Trace:
I20260811 18:57:37.317744 4167 rpcz_store.cc:276] 0811 18:57:35.406857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.406949 (+ 92us) service_pool.cc:227] Handling call
0811 18:57:37.316989 (+1910040us) ts_tablet_manager.cc:1145] Deleting tablet 36861ae786c549d687e687fe853d1801
0811 18:57:37.317003 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.317149 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1909778}
I20260811 18:57:37.319607 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 38f64f32463148349a2b0d403ae881e9 (DEFAULT_TABLE table=default.Table_cce74eefcdbb44369f1915d3f40f54e2 [id=43162c1c44f442fb912c654af76eac57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.320168 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f64f32463148349a2b0d403ae881e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.325865 5593 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.326136 5593 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.327049 5593 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.327244 5596 tablet.cc:1722] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:37.328758 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c" } modify_external_catalogs: true
I20260811 18:57:37.328931 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 026245dfb0284d95a1afae43c77074c5 (DEFAULT_TABLE table=default.Table_def64923a6b24aa0a7b9ac4a66dcace5 [id=1e2eeed8d9aa4ce0935705782a7a7e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.328975 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_cafc7915ae7e4280a4d71e3bce4ca57c" } modify_external_catalogs: true
I20260811 18:57:37.329334 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026245dfb0284d95a1afae43c77074c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.330786 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13786cfeafbc4312bf84f93b204add4f" } modify_external_catalogs: true
I20260811 18:57:37.331007 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_13786cfeafbc4312bf84f93b204add4f" } modify_external_catalogs: true
I20260811 18:57:37.332078 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f5a969dbd34f414ba3a0b45748510632 (DEFAULT_TABLE table=default.Table_1ec52777b5484eb18531a70c114a32f6 [id=0d5ee633d4ab43a58bf9009459ee7699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.332500 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a969dbd34f414ba3a0b45748510632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.334934 4163 tablet_bootstrap.cc:492] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.335067 4163 ts_tablet_manager.cc:1397] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.152s user 0.010s sys 0.011s
I20260811 18:57:37.335800 4163 raft_consensus.cc:348] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.336014 4163 raft_consensus.cc:374] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.336141 4163 raft_consensus.cc:729] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.336313 4163 consensus_queue.cc:260] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.336826 4163 ts_tablet_manager.cc:1428] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:37.337061 4163 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.338649 4163 tablet_bootstrap.cc:654] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.345635 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1027) took 1934 ms. Trace:
I20260811 18:57:37.345784 4167 rpcz_store.cc:276] 0811 18:57:35.410785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.410860 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:37.318586 (+1907726us) ts_tablet_manager.cc:1145] Deleting tablet 46505ba544104458888dbc2a37115dcc
0811 18:57:37.318600 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.345585 (+ 26985us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1907513}
I20260811 18:57:37.346241 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1028) took 1935 ms. Trace:
I20260811 18:57:37.346333 4167 rpcz_store.cc:276] 0811 18:57:35.410888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.410953 (+ 65us) service_pool.cc:227] Handling call
0811 18:57:37.346094 (+1935141us) ts_tablet_manager.cc:1145] Deleting tablet 18c599d945044e0088bc66c8472d2495
0811 18:57:37.346108 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.346221 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1934938}
I20260811 18:57:37.346642 4167 tablet_replica.cc:330] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.346851 4167 raft_consensus.cc:2229] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.347007 4167 raft_consensus.cc:2258] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.347584 4167 ts_tablet_manager.cc:1905] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.350383 4163 tablet_bootstrap.cc:492] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.350610 4163 ts_tablet_manager.cc:1397] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260811 18:57:37.351648 4163 raft_consensus.cc:348] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.351970 4163 raft_consensus.cc:374] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.352069 4163 raft_consensus.cc:729] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.352309 4163 consensus_queue.cc:260] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.353163 4163 ts_tablet_manager.cc:1428] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.353485 4163 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.355240 4162 tablet_bootstrap.cc:654] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.357286 5596 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37348 (request call id 1061) took 1029 ms. Trace:
I20260811 18:57:37.358695 3852 tablet_service.cc:1467] Processing CreateTablet for tablet a385abaeb70347dcac06a5cedbdddeb4 (DEFAULT_TABLE table=default.Table_5437383c0bbb4384825f578e83c10acc [id=78cd09fbad264f7385b142dcf7ff57b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.359525 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a385abaeb70347dcac06a5cedbdddeb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.361946 4163 tablet_bootstrap.cc:654] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.359157 5596 rpcz_store.cc:276] 0811 18:57:36.327453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.327506 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.357261 (+1029755us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:36.327978 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:36.328082 (+ 104us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:36.328091 (+ 9us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:36.328098 (+ 7us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:36.328138 (+ 40us) alter_schema_op.cc:126] START. Timestamp: 1786474656328080
0811 18:57:36.328141 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:36.328328 (+ 187us) log.cc:842] Serialized 173 byte log entry
0811 18:57:37.263812 (+935484us) op_driver.cc:444] REPLICATION: finished
0811 18:57:37.295775 (+ 31963us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:37.328011 (+ 32236us) log.cc:842] Serialized 5 byte log entry
0811 18:57:37.333864 (+ 5853us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.334115 (+ 251us) log.cc:842] Serialized 14 byte log entry
0811 18:57:37.334165 (+ 50us) alter_schema_op.cc:71] Released schema lock
0811 18:57:37.334168 (+ 3us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":31897,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":8691,"prepare.run_wall_time_us":62140,"raft.queue_time_us":167,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":223,"replication_time_us":935639,"thread_start_us":93417,"threads_started":2,"wal-append.queue_time_us":59623}]]}
I20260811 18:57:37.366290 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914d7e775ebf4b26beed57cb21140166" } modify_external_catalogs: true
I20260811 18:57:37.366544 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_914d7e775ebf4b26beed57cb21140166" } modify_external_catalogs: true
I20260811 18:57:37.366688 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d6113c39441d47a8a1d07df0fbd2371e" } new_table_name: "default.Table_037a37bbcc9044bdb397875f1abe75a0" modify_external_catalogs: true
I20260811 18:57:37.367110 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3b465c02cbcd4236a62ea1c357497b89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.367599 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b465c02cbcd4236a62ea1c357497b89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.368351 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 558dc0f1c46840d0991d08d792f903d5 (DEFAULT_TABLE table=default.Table_458dd09db5894e0fae4cc31deb113bdd [id=aa118bfdb3b440a1bc81ca195f425546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.368834 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 558dc0f1c46840d0991d08d792f903d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.371512 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b908c7d655a467bbda1f9d2fdc1b99a" } new_table_name: "default.Table_fd1672ab9ee548b499043927ab259054" modify_external_catalogs: true
I20260811 18:57:37.374892 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fe2f572c66cd4b9a8656e0fd439d50c7" } new_table_name: "default.Table_b67fc3dfee4e429c908d07053340f80c" modify_external_catalogs: true
I20260811 18:57:37.360247 4167 ts_tablet_manager.cc:1918] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.375891 4167 log.cc:1196] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:37.376502 4167 ts_tablet_manager.cc:1939] T 0c6e0d1eeb63493bbea4cd78dd085a3c P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.376912 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1029) took 1952 ms. Trace:
I20260811 18:57:37.377010 4167 rpcz_store.cc:276] 0811 18:57:35.424226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.424284 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:37.346595 (+1922311us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:37.346604 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.356860 (+ 10256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.360155 (+ 3295us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.376890 (+ 16735us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1921323}
I20260811 18:57:37.377518 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1032) took 1884 ms. Trace:
I20260811 18:57:37.377614 4167 rpcz_store.cc:276] 0811 18:57:35.492951 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.493260 (+ 309us) service_pool.cc:227] Handling call
0811 18:57:37.377351 (+1884091us) ts_tablet_manager.cc:1145] Deleting tablet 8eaafc7677344cac8d9616996c5f416c
0811 18:57:37.377364 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.377500 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1883850}
I20260811 18:57:37.378016 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1035) took 1829 ms. Trace:
I20260811 18:57:37.378125 4167 rpcz_store.cc:276] 0811 18:57:35.548826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.548885 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:37.377893 (+1829008us) ts_tablet_manager.cc:1145] Deleting tablet e2fb15b72c1d49d2ac4961bc04655a29
0811 18:57:37.377905 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.377996 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1828787}
I20260811 18:57:37.378607 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1036) took 1818 ms. Trace:
I20260811 18:57:37.378688 4167 rpcz_store.cc:276] 0811 18:57:35.560468 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.560534 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:37.378446 (+1817912us) ts_tablet_manager.cc:1145] Deleting tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
0811 18:57:37.378458 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.378589 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1817670}
I20260811 18:57:37.378921 4167 tablet_replica.cc:330] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.379145 4167 raft_consensus.cc:2229] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
W20260811 18:57:37.379539 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d721b0e55384d8a8efd88a0e47b1504: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.379820 4167 raft_consensus.cc:2258] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.380239 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d" } modify_external_catalogs: true
I20260811 18:57:37.380455 4167 ts_tablet_manager.cc:1905] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.380445 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f0a164081b1349d9bc89f20d1ea6425d" } modify_external_catalogs: true
I20260811 18:57:37.383245 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 7fdacc021a974bd981dc30e4915378a3 (DEFAULT_TABLE table=default.Table_83c2a159da7a4474bd29c31afe70760e [id=110ac0c6ffd4495ea578641ae12779cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.383759 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fdacc021a974bd981dc30e4915378a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.386466 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet a3973dfdaac746a6af7a402277d2711b with delete_type TABLET_DATA_DELETED (replaced by tablet 16db56987b2c44c9b5d3445073226ab9) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.387095 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.387487 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92a533c58b43c69fd65a1ec335cb3a (DEFAULT_TABLE table=default.Table_01425dcfb75d41f6ad6349c65957ebac [id=794cbc360aaf4b05a62e609896638744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.388913 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b92a533c58b43c69fd65a1ec335cb3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.392858 4167 ts_tablet_manager.cc:1918] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:37.395269 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5baa02ab900a4a8893c8e40d6ae5cf58 (DEFAULT_TABLE table=default.Table_7407d7434a3b4290889506a933ae2f05 [id=fe40a820d5cd401c8fc8393b02de9689]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.395804 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5baa02ab900a4a8893c8e40d6ae5cf58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.396703 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0495c5b770b34d54afb1379a6e36204c
I20260811 18:57:37.405655 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 3edf59b74ee245be852ece5ba50a1054 (DEFAULT_TABLE table=default.Table_6aacb785fd444621820be42007d55668 [id=996a202868ee4f6dadf4be343005550a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.406075 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3edf59b74ee245be852ece5ba50a1054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.407898 4167 log.cc:1196] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:37.408384 4167 ts_tablet_manager.cc:1939] T b1ab2eee21264b1aa5ba00528cae3843 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.408710 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1037) took 1845 ms. Trace:
I20260811 18:57:37.408785 4167 rpcz_store.cc:276] 0811 18:57:35.563055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.563130 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:37.378879 (+1815749us) ts_tablet_manager.cc:1145] Deleting tablet b1ab2eee21264b1aa5ba00528cae3843
0811 18:57:37.378891 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.389558 (+ 10667us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.392776 (+ 3218us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.408691 (+ 15915us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1815552}
I20260811 18:57:37.410048 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 764f87ff39df4706bc05e496fa2c7f29 (DEFAULT_TABLE table=default.Table_8edcf621b1064a75906a7ffe63e1a9d0 [id=673f9c8b623b4a8f8f34608f0487ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.410486 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764f87ff39df4706bc05e496fa2c7f29. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.411727 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac48afc7eec4410e8d6055ecdb794a4f consensus state has no leader
I20260811 18:57:37.413708 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 75ceba7bfc8240268b16c789f07813d1
W20260811 18:57:37.414462 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 86a57b040ab44812a2001705493f2bcd consensus state has no leader
I20260811 18:57:37.414594 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5a75a480f38047d880b0c1fa51403786
I20260811 18:57:37.387914 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 5665e9b2813b4151981a7d487db9e7cd (DEFAULT_TABLE table=default.Table_9db0e2b871fd484f82e42660d2a09c55 [id=867e36c5f215455f808898586ca092f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.415015 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5665e9b2813b4151981a7d487db9e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.420342 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13af4d419ae542cc961c75f28efd0204 consensus state has no leader
I20260811 18:57:37.388209 3719 tablet_service.cc:1467] Processing CreateTablet for tablet ec4405eb41ff4a7484fcc24648f0e56a (DEFAULT_TABLE table=default.Table_28f0027f9b254195878661731f172f78 [id=6bd6c71ba5e94958a2f5758edfa35d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.421029 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec4405eb41ff4a7484fcc24648f0e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.387596 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf3d47a038f423190b343d94cd2694f (DEFAULT_TABLE table=default.Table_33c7b91319d04649821da27a7798271a [id=0a529f726a2a4bb297cdac716eccbc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.422641 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf3d47a038f423190b343d94cd2694f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.425943 3851 tablet_service.cc:1467] Processing CreateTablet for tablet dcad9c8052af4cbda4926c7c9430ea2a (DEFAULT_TABLE table=default.Table_d3410fcb02604cdabfae4dfa57a0bac0 [id=8489eaa51fe9406b94e1a7054e307bbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.426363 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcad9c8052af4cbda4926c7c9430ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.427591 3551 master_service.cc:946] ReplaceTablet: received request to replace tablet 28af2d2c61254b0ca399dddb05e9ee70 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.429241 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d32212c817c94fad8379462b027e7d3d
I20260811 18:57:37.433231 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b9f31daf440647949978feb54b97543d (DEFAULT_TABLE table=default.Table_767c174a258144229f562c9dda12392b [id=4ef58bf17e6b402b96fcd79052cd35dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.433553 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4e1fc5686944dfa3eb57702c810eb8 with delete_type TABLET_DATA_DELETED (replaced by tablet b192929b6d154885a0cc44c00f1add8e) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.433719 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f31daf440647949978feb54b97543d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.434408 4162 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.434543 4162 ts_tablet_manager.cc:1397] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.177s user 0.001s sys 0.003s
I20260811 18:57:37.435180 4162 raft_consensus.cc:348] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.435407 4162 raft_consensus.cc:374] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.435460 4162 raft_consensus.cc:729] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.436805 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ea579c33f8b14e9fa7fa389ce7f45931 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.437371 4167 tablet_replica.cc:330] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.437563 4167 raft_consensus.cc:2229] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.437700 4167 raft_consensus.cc:2258] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.438210 4167 ts_tablet_manager.cc:1905] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.443656 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0f6df7a291456d80538e2b76417873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:15 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.444672 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdbe37b812846fdb089ec081000125c (DEFAULT_TABLE table=default.Table_9df97a2d74d040ef8f3dd56e41b8c28b [id=e2a3e504be604e95af18e61f23c07911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.445106 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdbe37b812846fdb089ec081000125c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.452689 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 7671139202af4a17976295f521828650 (DEFAULT_TABLE table=default.Table_15068805ee264778ac398274e74d630a [id=3744324baeba44dab08e66b32215651b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.453200 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7671139202af4a17976295f521828650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.464879 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc649a70b624d0cb6839f65000dacf9 (DEFAULT_TABLE table=default.Table_8752a36a67ad4fb380132326a813d172 [id=7259d5d4e7764c1b8d3a9d286f7bbd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.465320 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc649a70b624d0cb6839f65000dacf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.467257 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1e182c4df64e7f9fb1ffb452046f7f (DEFAULT_TABLE table=default.Table_3f83f2ec5f2847c9820675f2f29f0674 [id=0428c361975e441a9d8a4e228aeb0a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.470081 4169 ts_tablet_manager.cc:1918] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:37.470202 4169 log.cc:1196] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/221d0d8d69f142a29ec7065a82b3f769
I20260811 18:57:37.470646 4169 ts_tablet_manager.cc:1939] T 221d0d8d69f142a29ec7065a82b3f769 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.470997 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 5143412d3c2349fcb97dc527891b49e4 (DEFAULT_TABLE table=default.Table_153744b340b44d8c9e939deedadaced9 [id=da2277b2b7e94f26a71d0e68075766d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.471310 4162 consensus_queue.cc:260] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.475890 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cf6c8d542ae2489ba466ffadd518db12 (table default.Table_70493900fc9346389c1d4b71c611b962 [id=97d17499bd684f0abf98e0582fa06f5d]) successfully deleted
W20260811 18:57:37.476186 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d37e81d376e14907b13746cca831a96b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d37e81d376e14907b13746cca831a96b already in progress: creating tablet
W20260811 18:57:37.476392 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f86bfa65a8774b51bc7e7fe59442bf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f86bfa65a8774b51bc7e7fe59442bf2f already in progress: creating tablet
W20260811 18:57:37.476570 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f1342b8cd65b4c31a47b27a79c678d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1342b8cd65b4c31a47b27a79c678d2b already in progress: creating tablet
W20260811 18:57:37.476739 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e5f47ccc193d48efa15d60a06b148e00 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f47ccc193d48efa15d60a06b148e00 already in progress: creating tablet
W20260811 18:57:37.476915 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fcb1c9787d3f4fd181f86deca28ad160 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb1c9787d3f4fd181f86deca28ad160 already in progress: creating tablet
I20260811 18:57:37.477432 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 7b404e99e9414c67b3cd5aff12203a53 (table default.Table_d8397de033b74ce9accaa4085f59d7e6 [id=bebb3efbbc56416597abdf4a0b3952ed]) successfully deleted
I20260811 18:57:37.477542 4162 ts_tablet_manager.cc:1428] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
W20260811 18:57:37.477622 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 302cf69ea7c14fdaac72f6a9a9aff113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302cf69ea7c14fdaac72f6a9a9aff113 already in progress: creating tablet
W20260811 18:57:37.477800 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 36861ae786c549d687e687fe853d1801 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36861ae786c549d687e687fe853d1801 already in progress: creating tablet
I20260811 18:57:37.477864 4162 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.479489 4162 tablet_bootstrap.cc:654] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.480113 3855 tablet_service.cc:1467] Processing CreateTablet for tablet f10e19a2978a4a6799064cb36295c93c (DEFAULT_TABLE table=default.Table_7811427611da4cd1a27acc552c72c374 [id=b92c51f24dec492d8735bfb37ead3523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.483053 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88737be020144b20a2c4df22b9bf7b52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.483420 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_ba12b590ed224215a8ea56022e39e6c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.483669 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88737be020144b20a2c4df22b9bf7b52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.483786 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba12b590ed224215a8ea56022e39e6c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.483003 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e407c33cb78457c831222ccfdc05d9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.484406 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e407c33cb78457c831222ccfdc05d9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.484359 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eb78d7140fc94f75b13bca56099bde32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.484778 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb78d7140fc94f75b13bca56099bde32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.485867 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2f5ce2b1d14a9a988cff644e196088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.485874 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.485975 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc84fd57013443ca0a0e7752a213566 (DEFAULT_TABLE table=default.Table_6f167c64b15d4d18a642c45b00f75658 [id=1a44fd5ad046463dbd4d84fb4e154005]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.486280 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.486440 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet d21973bc8e514a089c2beb24a686060f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486500 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 (DEFAULT_TABLE table=default.Table_0084f6fe30164d4d9dcc8ad20699380e [id=48c5b345af51445eac883afc0ae3704f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.486521 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b66196716a4ba3893b6b8bfec2b8c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486913 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet a3973dfdaac746a6af7a402277d2711b with delete_type TABLET_DATA_DELETED (replaced by tablet 16db56987b2c44c9b5d3445073226ab9) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.486927 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 3cdaa303c70a44a19658664c8c9f9dd5 (DEFAULT_TABLE table=default.Table_ec98c10d285144029e63c99243d4d9fe [id=b73816870ec0459ca8092fcde1a27fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.485531 3857 tablet_service.cc:1467] Processing CreateTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 (DEFAULT_TABLE table=default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e [id=920721dee68b404bb09a2e453baec9c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.487560 3854 tablet_service.cc:1467] Processing CreateTablet for tablet 7fdacc021a974bd981dc30e4915378a3 (DEFAULT_TABLE table=default.Table_83c2a159da7a4474bd29c31afe70760e [id=110ac0c6ffd4495ea578641ae12779cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.488265 3862 tablet_service.cc:1467] Processing CreateTablet for tablet ec4405eb41ff4a7484fcc24648f0e56a (DEFAULT_TABLE table=default.Table_28f0027f9b254195878661731f172f78 [id=6bd6c71ba5e94958a2f5758edfa35d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.488867 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 15dd14e2b4bd4a08b5b08f532a55291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.489048 3858 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92a533c58b43c69fd65a1ec335cb3a (DEFAULT_TABLE table=default.Table_01425dcfb75d41f6ad6349c65957ebac [id=794cbc360aaf4b05a62e609896638744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.489655 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 9cf3d47a038f423190b343d94cd2694f (DEFAULT_TABLE table=default.Table_33c7b91319d04649821da27a7798271a [id=0a529f726a2a4bb297cdac716eccbc6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.489794 3866 tablet_service.cc:1467] Processing CreateTablet for tablet 5665e9b2813b4151981a7d487db9e7cd (DEFAULT_TABLE table=default.Table_9db0e2b871fd484f82e42660d2a09c55 [id=867e36c5f215455f808898586ca092f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.491252 3861 tablet_service.cc:1467] Processing CreateTablet for tablet ba29218db89a4814ac4e2f559884e34e (DEFAULT_TABLE table=default.Table_9a431bdf0c984b91935aed8346d63c56 [id=36e0f6431382481fae88e9cfec7f513c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.491591 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 79bf7d16b6024c9fb853c852d1c44f13 (DEFAULT_TABLE table=default.Table_5c323e78021f457095b0ad2754e21334 [id=dad48d97579f4912953013414d46c23b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.492729 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 31df6bcee1fb4324b20384bf3d3edee7 (DEFAULT_TABLE table=default.Table_bf526916d0504506a3e70cc658d22416 [id=3298c3e4fc0f40d1a4700840f2d9d917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.493160 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 3c55a7b27133454881bcddc544c22faa (DEFAULT_TABLE table=default.Table_9c421939f1354150bddc1156b40cf5fe [id=5063c88916b845fcb99d66fd8a45f042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.493858 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5ef097030b2146bb9df7f7d2bffde6b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.494462 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ef097030b2146bb9df7f7d2bffde6b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.495373 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 10bc2fcd64574327acf04347fa9ca0f4 (DEFAULT_TABLE table=default.Table_fa66e20bff3745ca961cfec0dc3a65c1 [id=f91b4d545c044e87a53411a16e80c42a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.495916 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9db0e2b871fd484f82e42660d2a09c55" } new_table_name: "default.Table_0ff3ccffaec842d2a6057786e0d521d3" modify_external_catalogs: true
W20260811 18:57:37.497931 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 46505ba544104458888dbc2a37115dcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46505ba544104458888dbc2a37115dcc already in progress: creating tablet
W20260811 18:57:37.498260 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 18c599d945044e0088bc66c8472d2495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18c599d945044e0088bc66c8472d2495 already in progress: creating tablet
I20260811 18:57:37.498605 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
W20260811 18:57:37.498787 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8eaafc7677344cac8d9616996c5f416c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8eaafc7677344cac8d9616996c5f416c already in progress: creating tablet
W20260811 18:57:37.498921 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e2fb15b72c1d49d2ac4961bc04655a29 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2fb15b72c1d49d2ac4961bc04655a29 already in progress: creating tablet
W20260811 18:57:37.499060 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae already in progress: creating tablet
I20260811 18:57:37.499442 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:37.500269 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_31dac67fa2c146b0956b5a3b25672da3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.500829 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31dac67fa2c146b0956b5a3b25672da3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.492370 3824 leader_election.cc:334] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.502231 3723 tablet_service.cc:1467] Processing CreateTablet for tablet ba29218db89a4814ac4e2f559884e34e (DEFAULT_TABLE table=default.Table_9a431bdf0c984b91935aed8346d63c56 [id=36e0f6431382481fae88e9cfec7f513c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.502702 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba29218db89a4814ac4e2f559884e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.502848 3711 tablet_service.cc:1467] Processing CreateTablet for tablet bc38e6760754449fbd6209d765cce70c (DEFAULT_TABLE table=default.Table_2ad9e3991d0f4274a4770a8f264f403e [id=9beaec986d304a5493a2d84fe239f803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.503153 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc38e6760754449fbd6209d765cce70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.503859 3716 tablet_service.cc:1467] Processing CreateTablet for tablet 76a3c8bc00124a328d7eb80294e08b40 (DEFAULT_TABLE table=default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e [id=64199234b0924a10b5c42ad4d92c44ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.504217 3716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a3c8bc00124a328d7eb80294e08b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.506671 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b192929b6d154885a0cc44c00f1add8e (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.507191 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b192929b6d154885a0cc44c00f1add8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.507622 5557 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.507783 5557 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:37.508452 5557 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:37.501727 3824 leader_election.cc:334] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.508777 3824 leader_election.cc:334] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.508981 3824 leader_election.cc:334] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.509083 3824 leader_election.cc:302] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.510324 5557 raft_consensus.cc:2726] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.506696 3706 tablet_service.cc:1467] Processing CreateTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 (DEFAULT_TABLE table=default.Table_5ad53b06897e4559b15d34a4d98f4005 [id=99cf45a7b48c49b9b61196f14a84330e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.510649 3824 leader_election.cc:334] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.510751 3824 leader_election.cc:302] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.510852 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.513604 4167 ts_tablet_manager.cc:1918] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.513718 4167 log.cc:1196] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:37.514402 5557 raft_consensus.cc:2726] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.515077 3824 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:37.515305 3824 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.515396 3824 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.515340 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca13a41112144c79ff0612e9c9fd663 (DEFAULT_TABLE table=default.Table_9c767befd17049f8a118fdbd2a89e55d [id=57cfd7ce65924312b163d92e3418bc05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.515729 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca13a41112144c79ff0612e9c9fd663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.515756 4167 ts_tablet_manager.cc:1939] T c99ceca57cb245f4a3a6cb4b36f1306d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.516467 5557 raft_consensus.cc:2726] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.518242 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c96a264e017471ea114ca0c6a9500ef" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:37.518514 3740 raft_consensus.cc:3037] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.519922 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 9bee89926dd74b54aaf40675bbae5d85 (DEFAULT_TABLE table=default.Table_8f2c4aa71e814a5dbc9684802806666e [id=df3f586e76a84ad2a47fb6a4d47ce274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.521392 3740 raft_consensus.cc:2454] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:37.529625 3718 tablet_service.cc:1467] Processing CreateTablet for tablet f86907d0e6cf4cca9b263435070c705e (DEFAULT_TABLE table=default.Table_db7b48638c894fa5ade5118c99ed4f80 [id=681e2d3ad28d47978115767346af16bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.530161 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86907d0e6cf4cca9b263435070c705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.534713 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ceba7bfc8240268b16c789f07813d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.534817 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 0495c5b770b34d54afb1379a6e36204c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.535144 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a75a480f38047d880b0c1fa51403786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.538825 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_35bcb0c44a4840879a1f1d1386bf24c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.539628 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35bcb0c44a4840879a1f1d1386bf24c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.540887 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e" } new_table_name: "default.Table_d24b30f82e744f838b01bb0f37a99644" modify_external_catalogs: true
I20260811 18:57:37.541394 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 43ad6993b63e4bbd98c8cc52f1a53451 (table default.Table_1e85b84e5b0e45bf80321dfb55c1704d [id=bb1f096d4965461a988af1ca355cb89b]) successfully deleted
W20260811 18:57:37.541653 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7be6d9578e7847c2822278486119390c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7be6d9578e7847c2822278486119390c already in progress: creating tablet
I20260811 18:57:37.542467 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet d32212c817c94fad8379462b027e7d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.544103 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1e182c4df64e7f9fb1ffb452046f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.544656 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6710084f21734dea9c7ce1f43d1bf587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.544881 3858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b92a533c58b43c69fd65a1ec335cb3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545094 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc84fd57013443ca0a0e7752a213566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545282 3854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fdacc021a974bd981dc30e4915378a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545526 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cdaa303c70a44a19658664c8c9f9dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.545737 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec4405eb41ff4a7484fcc24648f0e56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.549306 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cf3d47a038f423190b343d94cd2694f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.552603 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bf7d16b6024c9fb853c852d1c44f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.556257 3861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba29218db89a4814ac4e2f559884e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.558210 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 79bf7d16b6024c9fb853c852d1c44f13 (DEFAULT_TABLE table=default.Table_5c323e78021f457095b0ad2754e21334 [id=dad48d97579f4912953013414d46c23b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.559547 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b66196716a4ba3893b6b8bfec2b8c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.562565 3866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5665e9b2813b4151981a7d487db9e7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.563179 3855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10e19a2978a4a6799064cb36295c93c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.568455 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bf7d16b6024c9fb853c852d1c44f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.576347 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 067717418be34f7d9efe1c3be7f98830 (DEFAULT_TABLE table=default.Table_6c18f70552714e0f9a63822c3b6fb0a5 [id=203d5f31364f4e5bbc0413442f419953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.576834 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5143412d3c2349fcb97dc527891b49e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.580201 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bee89926dd74b54aaf40675bbae5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.583666 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c55a7b27133454881bcddc544c22faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.584062 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bc2fcd64574327acf04347fa9ca0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.585193 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 9bee89926dd74b54aaf40675bbae5d85 (DEFAULT_TABLE table=default.Table_8f2c4aa71e814a5dbc9684802806666e [id=df3f586e76a84ad2a47fb6a4d47ce274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.585722 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bee89926dd74b54aaf40675bbae5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.586474 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31df6bcee1fb4324b20384bf3d3edee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.590724 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 067717418be34f7d9efe1c3be7f98830 (DEFAULT_TABLE table=default.Table_6c18f70552714e0f9a63822c3b6fb0a5 [id=203d5f31364f4e5bbc0413442f419953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.590894 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067717418be34f7d9efe1c3be7f98830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.591174 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067717418be34f7d9efe1c3be7f98830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.596189 3522 master_runner.cc:186] Error getting master registration for 127.3.72.126:34821: OK, instance_id { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" instance_seqno: 1786474650422925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:37.596462 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:37.602156 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.604092 5611 tablet.cc:1722] T c6852aaef1d5486b87e5462eae532719 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:37.605310 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1040) took 2001 ms. Trace:
I20260811 18:57:37.606423 4167 rpcz_store.cc:276] 0811 18:57:35.603688 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.603757 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:37.437316 (+1833559us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:37.437329 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.440985 (+ 3656us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.513521 (+ 72536us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.605276 (+ 91755us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1833270}
I20260811 18:57:37.607419 4162 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.612739 4162 ts_tablet_manager.cc:1397] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.135s user 0.004s sys 0.000s
I20260811 18:57:37.613596 4162 raft_consensus.cc:348] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.613945 4162 raft_consensus.cc:374] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.614056 4162 raft_consensus.cc:729] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.614305 4162 consensus_queue.cc:260] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.615134 4162 ts_tablet_manager.cc:1428] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:37.615458 4162 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.617005 4162 tablet_bootstrap.cc:654] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.617254 4167 tablet_replica.cc:330] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:37.621827 3824 consensus_peers.cc:487] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:37.623472 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 951) took 3322 ms. Trace:
I20260811 18:57:37.623575 4169 rpcz_store.cc:276] 0811 18:57:34.300739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.300963 (+ 224us) service_pool.cc:227] Handling call
0811 18:57:37.177631 (+2876668us) ts_tablet_manager.cc:1145] Deleting tablet 221d0d8d69f142a29ec7065a82b3f769
0811 18:57:37.177643 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.224190 (+ 46547us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.470007 (+245817us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.623452 (+153445us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14483,"tablet-delete.queue_time_us":2876413}
I20260811 18:57:37.623931 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 949) took 3329 ms. Trace:
I20260811 18:57:37.624002 4169 rpcz_store.cc:276] 0811 18:57:34.295383 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.307452 (+ 12069us) service_pool.cc:227] Handling call
0811 18:57:37.623839 (+3316387us) ts_tablet_manager.cc:1145] Deleting tablet 490f34d7a403416fbab64e46973746de
0811 18:57:37.623850 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.623916 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3316091}
I20260811 18:57:37.624248 4169 tablet_replica.cc:330] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.624444 4169 raft_consensus.cc:2229] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.624593 4169 raft_consensus.cc:2258] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.625113 4169 ts_tablet_manager.cc:1905] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.631210 4167 raft_consensus.cc:2229] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.631561 4167 raft_consensus.cc:2258] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.632315 4167 ts_tablet_manager.cc:1905] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.635433 5407 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.635610 5407 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.636333 5407 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.637246 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.638396 3676 leader_election.cc:341] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:37.649623 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 76a3c8bc00124a328d7eb80294e08b40 (DEFAULT_TABLE table=default.Table_8ce7bbc3b605485e9ce129d1a1dbaf6e [id=64199234b0924a10b5c42ad4d92c44ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.650111 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76a3c8bc00124a328d7eb80294e08b40. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.651126 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76a3c8bc00124a328d7eb80294e08b40 consensus state has no leader
I20260811 18:57:37.653438 3555 catalog_manager.cc:5495] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.653848 5557 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.653981 5557 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.654616 5557 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.656323 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet c6852aaef1d5486b87e5462eae532719 (table default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca]). Expected version 1 got 0
I20260811 18:57:37.657521 4169 ts_tablet_manager.cc:1918] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:37.659471 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5665e9b2813b4151981a7d487db9e7cd consensus state has no leader
I20260811 18:57:37.659896 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c925e9ac591346b1a5142f338c68bcc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.660514 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c925e9ac591346b1a5142f338c68bcc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.662737 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" } modify_external_catalogs: true
I20260811 18:57:37.662989 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b24d38a66a5f499480e1bd110a992a7a" } modify_external_catalogs: true
I20260811 18:57:37.654117 3555 catalog_manager.cc:5495] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.670305 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:37.670624 4169 log.cc:1196] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cbf6ce167fb348be8d61372791f0cbca
I20260811 18:57:37.671171 4169 ts_tablet_manager.cc:1939] T cbf6ce167fb348be8d61372791f0cbca P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.671609 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 952) took 3335 ms. Trace:
I20260811 18:57:37.671766 4169 rpcz_store.cc:276] 0811 18:57:34.335899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.335958 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:37.624200 (+3288242us) ts_tablet_manager.cc:1145] Deleting tablet cbf6ce167fb348be8d61372791f0cbca
0811 18:57:37.624212 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.653292 (+ 29080us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.657412 (+ 4120us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.671589 (+ 14177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3272828}
I20260811 18:57:37.672248 4169 tablet_replica.cc:330] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:37.672535 4169 raft_consensus.cc:2229] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.672706 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" } modify_external_catalogs: true
I20260811 18:57:37.660971 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca13a41112144c79ff0612e9c9fd663 (DEFAULT_TABLE table=default.Table_9c767befd17049f8a118fdbd2a89e55d [id=57cfd7ce65924312b163d92e3418bc05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.673265 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca13a41112144c79ff0612e9c9fd663. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.645305 3678 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.674422 3678 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.674968 5407 raft_consensus.cc:2726] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.663993 3871 tablet_service.cc:1467] Processing CreateTablet for tablet f86907d0e6cf4cca9b263435070c705e (DEFAULT_TABLE table=default.Table_db7b48638c894fa5ade5118c99ed4f80 [id=681e2d3ad28d47978115767346af16bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.678735 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f86907d0e6cf4cca9b263435070c705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.683063 5557 raft_consensus.cc:482] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.683247 5557 raft_consensus.cc:504] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.683867 5557 leader_election.cc:288] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:37.684128 3824 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:37.685956 3824 leader_election.cc:302] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:37.686197 5557 raft_consensus.cc:2781] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.686353 5557 raft_consensus.cc:686] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:37.686570 5557 consensus_queue.cc:237] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:37.689379 4162 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.689788 4162 ts_tablet_manager.cc:1397] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.074s user 0.003s sys 0.000s
I20260811 18:57:37.690651 4162 raft_consensus.cc:348] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.690992 4162 raft_consensus.cc:374] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.691118 4162 raft_consensus.cc:729] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.691401 4162 consensus_queue.cc:260] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.692291 4162 ts_tablet_manager.cc:1428] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.692659 4162 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.695223 4162 tablet_bootstrap.cc:654] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.695581 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4655d6a166ce46f9b16b42662d325f94" } modify_external_catalogs: true
I20260811 18:57:37.699676 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_52e514412ff342e3b34ba958e863d4ba" } modify_external_catalogs: true
I20260811 18:57:37.700947 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_52e514412ff342e3b34ba958e863d4ba" } modify_external_catalogs: true
I20260811 18:57:37.700181 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24c4140a00244c8d8446d91705a8d4a8" } modify_external_catalogs: true
I20260811 18:57:37.702235 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_24c4140a00244c8d8446d91705a8d4a8" } modify_external_catalogs: true
I20260811 18:57:37.700474 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_75b5605dc2a14c9daf6d68e7094bcb39" } new_table_name: "default.Table_b2e37753037145439b8e47e544940455" modify_external_catalogs: true
I20260811 18:57:37.705478 4162 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.705730 4162 ts_tablet_manager.cc:1397] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260811 18:57:37.706557 4162 raft_consensus.cc:348] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.706933 4162 raft_consensus.cc:374] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.707063 4162 raft_consensus.cc:729] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.707374 4162 consensus_queue.cc:260] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.708312 4162 ts_tablet_manager.cc:1428] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.708673 4162 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.700791 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8076b90bff8d49c08328583af352ae7c" } new_table_name: "default.Table_440c0b981c8a42c28bf5f00f50ad7296" modify_external_catalogs: true
I20260811 18:57:37.710312 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3e8780219db840aeb566b570452e221b" } new_table_name: "default.Table_b373e0051180486da1efff0b123d200e" modify_external_catalogs: true
I20260811 18:57:37.710529 4162 tablet_bootstrap.cc:654] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.716399 3552 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
W20260811 18:57:37.717265 3824 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:37.717851 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 3195082643264e0d90f70e1007ab9b24
W20260811 18:57:37.717841 3824 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 059b325df1db4fde9d4b26b559121316
I20260811 18:57:37.718505 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" } modify_external_catalogs: true
I20260811 18:57:37.718767 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b1a781a4a2fd472180ad02bb0b1f7ee4" } modify_external_catalogs: true
I20260811 18:57:37.719408 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d495354c0994fb4a74c376078e41a26" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:37.719868 4169 pending_rounds.cc:62] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:37.719985 4169 pending_rounds.cc:68] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400181036277760 op_type: NO_OP noop_request { }
I20260811 18:57:37.720201 4169 raft_consensus.cc:2866] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:37.720392 4169 raft_consensus.cc:2258] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.721150 4169 ts_tablet_manager.cc:1905] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.727438 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d2326f1112004b75bdb9d55298314884"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.728170 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2326f1112004b75bdb9d55298314884 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.732092 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 221d0d8d69f142a29ec7065a82b3f769 (table default.Table_919f7bdbd76f4707813e30d37dfc3767 [id=3772979cce2249c2953e67cc206c1e22]) successfully deleted
W20260811 18:57:37.732369 3539 catalog_manager.cc:4808] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 490f34d7a403416fbab64e46973746de because the tablet was not found. No further retry: Not found: Tablet not found: 490f34d7a403416fbab64e46973746de
I20260811 18:57:37.740178 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet dcdbcfcd69a24d3f8a0f8383ba4c88e8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.753002 3871 tablet_service.cc:1467] Processing CreateTablet for tablet bc38e6760754449fbd6209d765cce70c (DEFAULT_TABLE table=default.Table_2ad9e3991d0f4274a4770a8f264f403e [id=9beaec986d304a5493a2d84fe239f803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.755726 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cbf6ce167fb348be8d61372791f0cbca (table default.Table_1558f156b5eb46a1858ca7673e4a9747 [id=dda7fb95dc1549ddbc88d8525ff023e8]) successfully deleted
I20260811 18:57:37.759416 4167 ts_tablet_manager.cc:1918] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:37.760648 4167 log.cc:1196] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/5d495354c0994fb4a74c376078e41a26
I20260811 18:57:37.764215 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b7b38d3b9570467a9128bd7fe73acf2f
I20260811 18:57:37.764810 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 6b4de110bd3348f0b0dae21c9f2b6a19 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:37.765595 4167 ts_tablet_manager.cc:1939] T 5d495354c0994fb4a74c376078e41a26 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:37.767083 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1049) took 2105 ms. Trace:
I20260811 18:57:37.767607 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc38e6760754449fbd6209d765cce70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.767798 4167 rpcz_store.cc:276] 0811 18:57:35.661443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.661634 (+ 191us) service_pool.cc:227] Handling call
0811 18:57:37.617203 (+1955569us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:37.617215 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.645066 (+ 27851us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.759272 (+114206us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.767059 (+ 7787us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9606,"tablet-delete.queue_time_us":1955407}
I20260811 18:57:37.769824 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 429eb5824f3f4b2b8da6e17ba7c69b92
I20260811 18:57:37.758180 3852 tablet_service.cc:1467] Processing CreateTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 (DEFAULT_TABLE table=default.Table_5ad53b06897e4559b15d34a4d98f4005 [id=99cf45a7b48c49b9b61196f14a84330e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.760504 4163 tablet_bootstrap.cc:492] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.773669 4163 ts_tablet_manager.cc:1397] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.420s user 0.004s sys 0.007s
I20260811 18:57:37.775707 4163 raft_consensus.cc:348] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.776067 4163 raft_consensus.cc:374] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.776254 4163 raft_consensus.cc:729] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.776602 4163 consensus_queue.cc:260] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.777442 4163 ts_tablet_manager.cc:1428] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:37.777813 4163 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.780731 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet 0495c5b770b34d54afb1379a6e36204c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.787012 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1057) took 1967 ms. Trace:
I20260811 18:57:37.787123 4167 rpcz_store.cc:276] 0811 18:57:35.819520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.819590 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:37.786849 (+1967259us) ts_tablet_manager.cc:1145] Deleting tablet 46a15e7a0c0f4283ac5e575d3c15c9da
0811 18:57:37.786863 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.786987 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1967018}
I20260811 18:57:37.787484 4167 tablet_replica.cc:330] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:37.787703 4167 raft_consensus.cc:2229] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.788089 4167 raft_consensus.cc:2258] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.788708 4167 ts_tablet_manager.cc:1905] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.794013 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f89eebbc40234794a081734f834294ab
I20260811 18:57:37.768417 4162 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.796281 4162 ts_tablet_manager.cc:1397] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.088s user 0.000s sys 0.009s
I20260811 18:57:37.797200 4162 raft_consensus.cc:348] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.797539 4162 raft_consensus.cc:374] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.797665 4162 raft_consensus.cc:729] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.797976 4162 consensus_queue.cc:260] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.798754 4162 ts_tablet_manager.cc:1428] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:37.804453 5407 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.804620 5407 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.805269 5407 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.805840 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet c99ceca57cb245f4a3a6cb4b36f1306d (table default.Table_8a27cf06565547a39a8eee2420c9a31b [id=0a4e4826c2cf4250a34e7e26eece8e3a]) successfully deleted
I20260811 18:57:37.806586 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:37.807022 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 5d495354c0994fb4a74c376078e41a26 (table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]) successfully deleted
W20260811 18:57:37.807201 3676 leader_election.cc:341] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:37.807292 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 46a15e7a0c0f4283ac5e575d3c15c9da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46a15e7a0c0f4283ac5e575d3c15c9da already in progress: creating tablet
I20260811 18:57:37.809242 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 97e6e6b26a11432eb739e4219d062f7d (DEFAULT_TABLE table=default.Table_1ec50daf17974aef9e292bab9fc018cd [id=c571e0e9444e48859e1ccaf8b4abc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.809825 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 275ec54207504e9f8267925e2224b2ab (DEFAULT_TABLE table=default.Table_1365655a76d7454c8415102e75960152 [id=d634ab6079f74d08ac11be356a004ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.810479 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 961f8adc3ef24257b669288c44fe2511 (DEFAULT_TABLE table=default.Table_fb6981748ef0423aaaa9a85bacef95be [id=b5aed8fa1431445da78c1e9cfb7a19de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.811491 3708 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed4bc2ff0d743bbbe8383d235e1f398 (DEFAULT_TABLE table=default.Table_9485300898e145ba9645d6e5ede8d7f1 [id=089de6d15c3245d1a9af59a8444fbeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.811025 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 87c390ea36844d91a115a4c6a64e6563 (DEFAULT_TABLE table=default.Table_2c977f4c94664fcb88bb338b5c79edfe [id=7d23ddd1558d48488f96456d33f7e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812178 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7 (DEFAULT_TABLE table=default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 [id=9a0911a26573434cb56ca228b312c33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812584 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 1932990f8a014b74999971d07c1cbe84 (DEFAULT_TABLE table=default.Table_606ca53b362a44b4ba72954536b85633 [id=b620868692c840f58e1a970a496e8a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.813021 3718 tablet_service.cc:1467] Processing CreateTablet for tablet e883852aaf6c41c79e87e4286da16a27 (DEFAULT_TABLE table=default.Table_755a5ecf2290451b8befb8e19c0cda4b [id=94f0c49c6b794f85b2a460c8789c6c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.812584 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 03109d96439141228d8c26b491e2dd8c (DEFAULT_TABLE table=default.Table_c364560f47804860afd5635ea9b6885d [id=30094b497eb9441caec9b0296d80858d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.814970 3678 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.815114 3678 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.815631 5407 raft_consensus.cc:2726] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.816679 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 16db56987b2c44c9b5d3445073226ab9 (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.817122 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 75daed89122c406caa4bd9bbaa67d9b8 (DEFAULT_TABLE table=default.Table_faccbd3f194043708dbbd12a50f037cf [id=14137b353d6b409ab3461d37309cdff7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.754408 3824 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.784461 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ceba7bfc8240268b16c789f07813d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.785375 3872 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a75a480f38047d880b0c1fa51403786 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.817670 3824 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.818017 5557 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.818379 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 63979.2us
W20260811 18:57:37.818610 3824 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.818801 3824 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.819072 5557 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.820184 3824 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.820374 3824 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:37.820624 5557 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:37.820962 3824 leader_election.cc:334] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.824334 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 75daed89122c406caa4bd9bbaa67d9b8 (DEFAULT_TABLE table=default.Table_faccbd3f194043708dbbd12a50f037cf [id=14137b353d6b409ab3461d37309cdff7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:37.830133 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
W20260811 18:57:37.830719 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a135bd92ab5b4f76bad9b3eedcfada29 consensus state has no leader
I20260811 18:57:37.831346 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3195082643264e0d90f70e1007ab9b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.765017 3860 tablet_service.cc:1467] Processing CreateTablet for tablet b192929b6d154885a0cc44c00f1add8e (DEFAULT_TABLE table=default.Table_23a9b3e82a454b1a8a3b1f56264c8ee8 [id=fe22365a40174f33ae3ae31dc7eff40e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.786206 4163 tablet_bootstrap.cc:654] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.787927 3865 tablet_service.cc:1514] Processing DeleteTablet for tablet d32212c817c94fad8379462b027e7d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.824334 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 87c390ea36844d91a115a4c6a64e6563 (DEFAULT_TABLE table=default.Table_2c977f4c94664fcb88bb338b5c79edfe [id=7d23ddd1558d48488f96456d33f7e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.786612 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.835459 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75daed89122c406caa4bd9bbaa67d9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.838652 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b192929b6d154885a0cc44c00f1add8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.839010 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a5bbd3274980470b9fe71457b49bdd62
W20260811 18:57:37.839672 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:37.846837 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c390ea36844d91a115a4c6a64e6563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.854897 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 16db56987b2c44c9b5d3445073226ab9 (DEFAULT_TABLE table=default.Table_f3bec93fd84c4da3aa02fcdafd33e7a7 [id=ab388e289912485198523c44e2ed1647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.855742 3859 tablet_service.cc:1467] Processing CreateTablet for tablet e883852aaf6c41c79e87e4286da16a27 (DEFAULT_TABLE table=default.Table_755a5ecf2290451b8befb8e19c0cda4b [id=94f0c49c6b794f85b2a460c8789c6c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.856423 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 03109d96439141228d8c26b491e2dd8c (DEFAULT_TABLE table=default.Table_c364560f47804860afd5635ea9b6885d [id=30094b497eb9441caec9b0296d80858d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.856845 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03109d96439141228d8c26b491e2dd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.860293 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b67fc3dfee4e429c908d07053340f80c" } new_table_name: "default.Table_543e49bc9d8a40e8a06df8d7f48479b7" modify_external_catalogs: true
I20260811 18:57:37.862666 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.863265 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5dde73e565645cfbaf60a60e2d0e71a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.865164 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 3a9efd2f19d74a68a8536fd057377738 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.865245 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9a431bdf0c984b91935aed8346d63c56" } new_table_name: "default.Table_f0da3b5bd2b14fd3961e325d592c86b6" modify_external_catalogs: true
I20260811 18:57:37.865867 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bb1820be09714d0b976eda8e0151671e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.866611 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb1820be09714d0b976eda8e0151671e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.869143 5557 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.869350 5557 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.870122 5557 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.871446 5407 raft_consensus.cc:482] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.871598 5407 raft_consensus.cc:504] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.872257 5407 leader_election.cc:288] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.873103 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37e81d376e14907b13746cca831a96b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.873828 3676 leader_election.cc:341] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:37.875228 3678 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.875416 3678 leader_election.cc:302] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.875701 5407 raft_consensus.cc:2726] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:37.878239 4169 ts_tablet_manager.cc:1918] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:37.878458 4169 log.cc:1196] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6f3060822021431584a494e6941be756
I20260811 18:57:37.878984 4169 ts_tablet_manager.cc:1939] T 6f3060822021431584a494e6941be756 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:37.879438 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 953) took 3543 ms. Trace:
W20260811 18:57:37.884646 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13af4d419ae542cc961c75f28efd0204 consensus state has no leader
I20260811 18:57:37.894330 4162 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.895992 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0c6490a83dec455bac3fe91b50da8a95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.896322 4162 tablet_bootstrap.cc:654] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.896727 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c6490a83dec455bac3fe91b50da8a95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.898893 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e6e6b26a11432eb739e4219d062f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.902622 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275ec54207504e9f8267925e2224b2ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.904897 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_908f52c0075e4f6c8f60b6c84f13e004"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.905141 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b882f1fc510a41558369a5ced806d29a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.905077 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5dc02b59b48d45c8bee6458f21962eeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.905547 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b882f1fc510a41558369a5ced806d29a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.905632 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5dc02b59b48d45c8bee6458f21962eeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:37.905552 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_908f52c0075e4f6c8f60b6c84f13e004 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.906611 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961f8adc3ef24257b669288c44fe2511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.911015 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed4bc2ff0d743bbbe8383d235e1f398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.911329 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.912690 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba29218db89a4814ac4e2f559884e34e consensus state has no leader
I20260811 18:57:37.914598 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1932990f8a014b74999971d07c1cbe84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.918509 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c390ea36844d91a115a4c6a64e6563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.879951 3864 tablet_service.cc:1467] Processing CreateTablet for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7 (DEFAULT_TABLE table=default.Table_3c1f5965d2e54153a2d5b720ce4a0aa3 [id=9a0911a26573434cb56ca228b312c33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.880749 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 275ec54207504e9f8267925e2224b2ab (DEFAULT_TABLE table=default.Table_1365655a76d7454c8415102e75960152 [id=d634ab6079f74d08ac11be356a004ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.880749 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 961f8adc3ef24257b669288c44fe2511 (DEFAULT_TABLE table=default.Table_fb6981748ef0423aaaa9a85bacef95be [id=b5aed8fa1431445da78c1e9cfb7a19de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.881287 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 97e6e6b26a11432eb739e4219d062f7d (DEFAULT_TABLE table=default.Table_1ec50daf17974aef9e292bab9fc018cd [id=c571e0e9444e48859e1ccaf8b4abc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.882372 3855 tablet_service.cc:1514] Processing DeleteTablet for tablet 3195082643264e0d90f70e1007ab9b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.882671 3858 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d495354c0994fb4a74c376078e41a26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.882946 3866 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883167 3854 tablet_service.cc:1514] Processing DeleteTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883442 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet f89eebbc40234794a081734f834294ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.883607 3863 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bbd3274980470b9fe71457b49bdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:37.922127 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e" } modify_external_catalogs: true
I20260811 18:57:37.922406 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dd9429d1d96e4c95b9fcd62b8b3e608e" } modify_external_catalogs: true
I20260811 18:57:37.923509 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e883852aaf6c41c79e87e4286da16a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.923735 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03109d96439141228d8c26b491e2dd8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.928686 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16db56987b2c44c9b5d3445073226ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.930639 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75daed89122c406caa4bd9bbaa67d9b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.942574 4162 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:37.942759 4162 ts_tablet_manager.cc:1397] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.001s
I20260811 18:57:37.943439 4162 raft_consensus.cc:348] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.943674 4162 raft_consensus.cc:374] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.943738 4162 raft_consensus.cc:729] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.881287 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 5ed4bc2ff0d743bbbe8383d235e1f398 (DEFAULT_TABLE table=default.Table_9485300898e145ba9645d6e5ede8d7f1 [id=089de6d15c3245d1a9af59a8444fbeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.943923 4162 consensus_queue.cc:260] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.944489 4162 ts_tablet_manager.cc:1428] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:37.944789 4162 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:37.946717 4162 tablet_bootstrap.cc:654] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:37.956127 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet b4c7f500a5444bae9a015a5e903c0622 on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:37.956415 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 65234acc3d5a462683edbe8f223eeefc on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:37.957391 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 476da7e2681540a89824b58fb7237df9 (table default.Table_2940279efa3f41ec980785fb086dd12d [id=da685ecf591b4f0a986154b12499b125], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 476da7e2681540a89824b58fb7237df9 consensus state has no leader
I20260811 18:57:37.892078 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 1932990f8a014b74999971d07c1cbe84 (DEFAULT_TABLE table=default.Table_606ca53b362a44b4ba72954536b85633 [id=b620868692c840f58e1a970a496e8a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.897558 4163 tablet_bootstrap.cc:492] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:37.960824 4163 ts_tablet_manager.cc:1397] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.183s user 0.004s sys 0.000s
I20260811 18:57:37.961752 4163 raft_consensus.cc:348] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.962019 4163 raft_consensus.cc:374] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.962080 4163 raft_consensus.cc:729] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.961826 3555 catalog_manager.cc:5495] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.962313 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:37.962280 4163 consensus_queue.cc:260] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.962630 3555 catalog_manager.cc:5495] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:37.962924 4163 ts_tablet_manager.cc:1428] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:37.963027 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
I20260811 18:57:37.963243 4163 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:37.906963 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16db56987b2c44c9b5d3445073226ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.963547 3552 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7b329f3d8a5543659bfb5de93453f46b [id=75ea77c989e746f88a94e72de6cd11ca] alter complete (version 1)
I20260811 18:57:37.969794 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961f8adc3ef24257b669288c44fe2511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971791 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ed4bc2ff0d743bbbe8383d235e1f398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971654 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_149093c5dbb2402ebc6afe1a069fb195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:37.971933 5593 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.972062 5593 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
W20260811 18:57:37.972276 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_149093c5dbb2402ebc6afe1a069fb195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.972647 5593 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:37.973845 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e6e6b26a11432eb739e4219d062f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.971670 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb" } modify_external_catalogs: true
I20260811 18:57:37.974563 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a35f3caa50e849abac2fc4d5711ea9fb" } modify_external_catalogs: true
I20260811 18:57:37.976344 4163 tablet_bootstrap.cc:654] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.976713 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.976948 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet f89eebbc40234794a081734f834294ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:37.977135 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e883852aaf6c41c79e87e4286da16a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.982489 5407 raft_consensus.cc:482] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.982656 5407 raft_consensus.cc:504] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:37.983392 5407 leader_election.cc:288] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:37.984050 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8f5b87062349c4b19ec99fa0175f24" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:37.984668 3676 leader_election.cc:341] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:37.989032 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a8cb2d7003c4cf1b836c6b39a6db3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.995959 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 275ec54207504e9f8267925e2224b2ab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:37.999301 3678 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:37.999449 3678 leader_election.cc:302] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:37.999776 5407 raft_consensus.cc:2726] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.002820 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1932990f8a014b74999971d07c1cbe84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.007182 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bbd3274980470b9fe71457b49bdd62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.010516 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b38d3b9570467a9128bd7fe73acf2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.011792 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 6710084f21734dea9c7ce1f43d1bf587
I20260811 18:57:38.012734 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e9f0e443ab84424dac307b745b496986
I20260811 18:57:38.013394 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" } modify_external_catalogs: true
I20260811 18:57:38.013612 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_911b59bd767843d39233f5c65932fb5d" } modify_external_catalogs: true
I20260811 18:57:38.015297 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_788572afacab482590222fc4d084ef0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.015763 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_614252f40e34465080bb490a1a7050ed" } modify_external_catalogs: true
W20260811 18:57:38.015900 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_788572afacab482590222fc4d084ef0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.015928 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_614252f40e34465080bb490a1a7050ed" } modify_external_catalogs: true
I20260811 18:57:38.017884 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fe6c2229e23048eb81c61314102eab38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.018476 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe6c2229e23048eb81c61314102eab38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.921648 4169 rpcz_store.cc:276] 0811 18:57:34.336043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336162 (+ 119us) service_pool.cc:227] Handling call
0811 18:57:37.672190 (+3336028us) ts_tablet_manager.cc:1145] Deleting tablet 6f3060822021431584a494e6941be756
0811 18:57:37.672211 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:37.724559 (+ 52348us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.878120 (+153561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:37.879420 (+ 1300us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3320668}
I20260811 18:57:38.026991 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 954) took 3690 ms. Trace:
I20260811 18:57:38.027138 4169 rpcz_store.cc:276] 0811 18:57:34.336181 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336330 (+ 149us) service_pool.cc:227] Handling call
0811 18:57:38.026786 (+3690456us) ts_tablet_manager.cc:1145] Deleting tablet dd80633f926942e88f54b32d9a03ea3a
0811 18:57:38.026802 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.026962 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675076}
I20260811 18:57:38.027465 4169 tablet_replica.cc:330] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.027691 4169 raft_consensus.cc:2229] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.027870 4169 raft_consensus.cc:2258] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:37.913493 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.032150 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bfea0cd619024204b9bff1e2dd62b7b0
W20260811 18:57:38.032105 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 118631us
I20260811 18:57:38.033958 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 69059a0667234a408cc1c09585df63bc
I20260811 18:57:38.036274 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2e407c33cb78457c831222ccfdc05d9d" } new_table_name: "default.Table_cb4ec5ebc85e4562a65d10436234b4c2" modify_external_catalogs: true
I20260811 18:57:38.036890 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b373e0051180486da1efff0b123d200e" } new_table_name: "default.Table_144f54709c324231aab49a10d3daab83" modify_external_catalogs: true
I20260811 18:57:38.041230 5407 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.041393 5407 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.041949 5407 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.043035 3678 consensus_peers.cc:487] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.043247 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_be8ef75ca590418486730a869708b637" } new_table_name: "default.Table_f747b7645d4643a5bd6215c8ff94bb1e" modify_external_catalogs: true
I20260811 18:57:38.043084 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:38.043507 3888 raft_consensus.cc:2454] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:38.043344 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1afb343a7aa84568ad8ad65cab128123"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.044051 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1afb343a7aa84568ad8ad65cab128123 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.044016 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.044296 5407 raft_consensus.cc:2781] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.044363 5407 raft_consensus.cc:482] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.044415 5407 raft_consensus.cc:3037] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.048430 5407 raft_consensus.cc:504] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.049606 5407 leader_election.cc:288] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.051025 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.051859 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:38.052096 3888 raft_consensus.cc:3037] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:38.053393 3678 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.054664 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a" } modify_external_catalogs: true
I20260811 18:57:38.054883 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c5dde73e565645cfbaf60a60e2d0e71a" } modify_external_catalogs: true
I20260811 18:57:38.056417 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e544b4b1592c4d51a2ed0d6461edc75e" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.056757 3745 raft_consensus.cc:2379] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in current term 1: Already voted for candidate bd8c4bdbd811417ab011a7131f17d283 in this term.
I20260811 18:57:38.064256 4169 ts_tablet_manager.cc:1905] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.066097 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet 82b04e37fc264fc08e0ed9517356a17d from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:38.068427 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fae3750e5870411e8fe7dedae2fe3ec0 consensus state has no leader
W20260811 18:57:38.069460 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b48a82a7182b4bf89fb9fd3830829909 has no consensus state
W20260811 18:57:38.070034 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet be9288a76eab4606ba189d7de5813973 consensus state has no leader
I20260811 18:57:38.079357 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 82b04e37fc264fc08e0ed9517356a17d
I20260811 18:57:38.079733 3549 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 8ab3b72a62db4781806808f3af2ce46e
I20260811 18:57:38.079933 3555 catalog_manager.cc:6565] ReplaceTablet: tablet 82b04e37fc264fc08e0ed9517356a17d deleted and replaced by tablet 94857ef729eb435c97d089b8ab108cea
I20260811 18:57:38.087909 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bfaa8ab4fcd40efb60e1718e424bccf" } new_table_name: "default.Table_8b0f40c58a1c4f028c865253aba93296" modify_external_catalogs: true
I20260811 18:57:38.088680 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.089140 3715 tablet_service.cc:1467] Processing CreateTablet for tablet e6562b8db6d54b42a89d19df04cab07c (DEFAULT_TABLE table=default.Table_88737be020144b20a2c4df22b9bf7b52 [id=2ef647fe25634721afac1f7c2d4a359a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.089584 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6562b8db6d54b42a89d19df04cab07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.093039 3555 master_service.cc:946] ReplaceTablet: received request to replace tablet d11b889e9d044a049d0586382c263951 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.072790 3888 raft_consensus.cc:2454] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:38.102594 3676 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.103048 5407 raft_consensus.cc:2781] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 18:57:38.101482 3824 leader_election.cc:341] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Illegal state: Tablet not RUNNING: STOPPED
W20260811 18:57:38.104046 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b540ae350254346a675b54118c5cfcb consensus state has no leader
I20260811 18:57:38.103906 3824 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.107209 4163 tablet_bootstrap.cc:492] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.107493 4163 ts_tablet_manager.cc:1397] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.144s user 0.006s sys 0.000s
I20260811 18:57:38.110123 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d11b889e9d044a049d0586382c263951
I20260811 18:57:38.110711 3555 catalog_manager.cc:6565] ReplaceTablet: tablet d11b889e9d044a049d0586382c263951 deleted and replaced by tablet e4441c724b394e8092fb19c0d2427233
I20260811 18:57:38.108707 4163 raft_consensus.cc:348] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.111124 4163 raft_consensus.cc:374] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.111196 4163 raft_consensus.cc:729] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.111613 4163 consensus_queue.cc:260] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.112296 5407 raft_consensus.cc:686] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:38.112381 4163 ts_tablet_manager.cc:1428] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:38.112601 5407 consensus_queue.cc:237] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:38.117178 3824 leader_election.cc:341] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:38.117766 5557 raft_consensus.cc:2726] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.118477 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6f3060822021431584a494e6941be756 (table default.Table_e5be012fe5c34e9bb842907f4b154b8e [id=aa84878356db499591a61cc81550307c]) successfully deleted
I20260811 18:57:38.119063 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.119263 3745 raft_consensus.cc:2454] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:38.121982 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9914a933914f4377ba70e0b57c70edff" } new_table_name: "default.Table_c3327dac0fcc458d9b96d9eeb6098f22" modify_external_catalogs: true
I20260811 18:57:38.124890 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_80140471fa194b22bf8347836f95fdca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.125450 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80140471fa194b22bf8347836f95fdca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.126811 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0765ca548a9649e88a2ad253c392107b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.127434 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0765ca548a9649e88a2ad253c392107b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.129698 5407 raft_consensus.cc:482] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.129899 5407 raft_consensus.cc:504] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.131803 5407 leader_election.cc:288] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.133719 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18242f762d8407db0e2f8716890feb4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.133864 3678 leader_election.cc:334] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.134460 3676 leader_election.cc:341] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.134594 3676 leader_election.cc:302] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:38.136392 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet dd80633f926942e88f54b32d9a03ea3a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd80633f926942e88f54b32d9a03ea3a already in progress: creating tablet
W20260811 18:57:38.136404 3824 leader_election.cc:334] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.136533 3824 leader_election.cc:302] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.136890 5593 raft_consensus.cc:2726] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.140314 4167 ts_tablet_manager.cc:1918] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:38.140457 4167 log.cc:1196] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ef47211aa8864d13a55551da47804c19
I20260811 18:57:38.140848 4167 ts_tablet_manager.cc:1939] T ef47211aa8864d13a55551da47804c19 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.141681 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1058) took 2321 ms. Trace:
I20260811 18:57:38.141793 4167 rpcz_store.cc:276] 0811 18:57:35.820344 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.820397 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:37.787439 (+1967042us) ts_tablet_manager.cc:1145] Deleting tablet ef47211aa8864d13a55551da47804c19
0811 18:57:37.787450 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.058537 (+271087us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.140193 (+ 81656us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.141659 (+ 1466us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1966867}
I20260811 18:57:38.142239 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1059) took 2321 ms. Trace:
I20260811 18:57:38.142359 4167 rpcz_store.cc:276] 0811 18:57:35.820890 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.820941 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:38.142114 (+2321173us) ts_tablet_manager.cc:1145] Deleting tablet 22e1a33348e34e35bab786f8966c408b
0811 18:57:38.142125 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.142221 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2321006}
I20260811 18:57:38.142706 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1060) took 2316 ms. Trace:
I20260811 18:57:38.142781 4167 rpcz_store.cc:276] 0811 18:57:35.826006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.826058 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:38.142591 (+2316533us) ts_tablet_manager.cc:1145] Deleting tablet 6f804703e20c4214b8b2a31efeaef59e
0811 18:57:38.142603 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.142678 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2316343}
I20260811 18:57:38.143088 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1061) took 2303 ms. Trace:
I20260811 18:57:38.143157 4167 rpcz_store.cc:276] 0811 18:57:35.839641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.839720 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:38.142994 (+2303274us) ts_tablet_manager.cc:1145] Deleting tablet 9b931eca7ba04c11af6571cb418f882e
0811 18:57:38.143004 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.143075 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2303001}
W20260811 18:57:38.143236 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ee5acbaa9f54d4ba523b603641e4a97 consensus state has no leader
I20260811 18:57:38.143421 4167 tablet_replica.cc:330] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.143640 4167 raft_consensus.cc:2229] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.143828 4167 raft_consensus.cc:2258] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.144398 4167 ts_tablet_manager.cc:1905] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.152639 4169 ts_tablet_manager.cc:1918] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.152808 4169 log.cc:1196] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8c07150e31994bdf9a00ac2e7a954691
I20260811 18:57:38.153169 4169 ts_tablet_manager.cc:1939] T 8c07150e31994bdf9a00ac2e7a954691 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.153549 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 955) took 3817 ms. Trace:
I20260811 18:57:38.153659 4169 rpcz_store.cc:276] 0811 18:57:34.336295 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336370 (+ 75us) service_pool.cc:227] Handling call
0811 18:57:38.027416 (+3691046us) ts_tablet_manager.cc:1145] Deleting tablet 8c07150e31994bdf9a00ac2e7a954691
0811 18:57:38.027426 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.148682 (+121256us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.152521 (+ 3839us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.153529 (+ 1008us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675678}
I20260811 18:57:38.154126 4169 tablet_replica.cc:330] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.157522 4163 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.157789 5407 raft_consensus.cc:2726] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.159246 4169 raft_consensus.cc:2229] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.159449 4169 raft_consensus.cc:2258] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.160427 4163 tablet_bootstrap.cc:654] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.162245 4162 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.162809 4169 ts_tablet_manager.cc:1905] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.163126 4162 ts_tablet_manager.cc:1397] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.218s user 0.016s sys 0.007s
I20260811 18:57:38.167901 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e8dcb132b5b5474b8ff081e54a55dd3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.167842 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.170575 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2dffa8e09c6948c092cfdd4d888e5f81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.175407 4162 raft_consensus.cc:348] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:38.175740 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 64085.1us
I20260811 18:57:38.169569 3860 tablet_service.cc:1467] Processing CreateTablet for tablet e6562b8db6d54b42a89d19df04cab07c (DEFAULT_TABLE table=default.Table_88737be020144b20a2c4df22b9bf7b52 [id=2ef647fe25634721afac1f7c2d4a359a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.176230 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6562b8db6d54b42a89d19df04cab07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.176481 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.171165 3871 tablet_service.cc:1467] Processing CreateTablet for tablet c590c98e39f84ea09b26bbe8640ca93d (DEFAULT_TABLE table=default.Table_ba12b590ed224215a8ea56022e39e6c0 [id=54bb76f0477a4a589ad12b460d4c7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.179677 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c590c98e39f84ea09b26bbe8640ca93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.172567 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 915ab20a6b404a1f89080ce78214001d (DEFAULT_TABLE table=default.Table_3b465c02cbcd4236a62ea1c357497b89 [id=5469d608a7c04d9b99adea950d56bdd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.182922 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915ab20a6b404a1f89080ce78214001d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.173125 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet ce94d37efd2245bf8cc71fdd90bd9735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.175369 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.175710 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0e443ab84424dac307b745b496986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.188751 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 69059a0667234a408cc1c09585df63bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.189231 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3799766ede4e4c8bb4e3ab35a723ebc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.189877 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8dcb132b5b5474b8ff081e54a55dd3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.190742 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ceca57cb245f4a3a6cb4b36f1306d with delete_type TABLET_DATA_DELETED (replaced by tablet f1800e9e2b064a6fb546b9d33cefa68b) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.190918 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2dffa8e09c6948c092cfdd4d888e5f81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.191640 4163 tablet_bootstrap.cc:492] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.191836 4163 ts_tablet_manager.cc:1397] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.001s
I20260811 18:57:38.195255 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b404e99e9414c67b3cd5aff12203a53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:21 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.197880 4163 raft_consensus.cc:348] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:38.202310 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8c07150e31994bdf9a00ac2e7a954691 (table default.Table_511d4c7dbd2b41ae95fc5f910147870d [id=4478bf63238b44a7bd6f891f3ac0a7c7]) successfully deleted
I20260811 18:57:38.207077 4163 raft_consensus.cc:374] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.207221 4163 raft_consensus.cc:729] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.207492 4163 consensus_queue.cc:260] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:38.202279 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet d11b889e9d044a049d0586382c263951 with delete_type TABLET_DATA_DELETED (replaced by tablet e4441c724b394e8092fb19c0d2427233) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.214776 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cccebfc54484a9b9abffcce4e459fb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:27 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.215029 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b04e37fc264fc08e0ed9517356a17d with delete_type TABLET_DATA_DELETED (replaced by tablet 94857ef729eb435c97d089b8ab108cea) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.205906 3715 tablet_service.cc:1467] Processing CreateTablet for tablet c590c98e39f84ea09b26bbe8640ca93d (DEFAULT_TABLE table=default.Table_ba12b590ed224215a8ea56022e39e6c0 [id=54bb76f0477a4a589ad12b460d4c7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.215575 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c590c98e39f84ea09b26bbe8640ca93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.206147 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.206279 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.216662 4162 raft_consensus.cc:374] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.216871 4162 raft_consensus.cc:729] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.217494 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80" } modify_external_catalogs: true
I20260811 18:57:38.217657 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.217733 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_fd5ee099cbc94fd285f1b71eea0cde80" } modify_external_catalogs: true
I20260811 18:57:38.206620 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 915ab20a6b404a1f89080ce78214001d (DEFAULT_TABLE table=default.Table_3b465c02cbcd4236a62ea1c357497b89 [id=5469d608a7c04d9b99adea950d56bdd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.218472 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915ab20a6b404a1f89080ce78214001d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.218683 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_46d76d83b3174f5da337585b00af5f68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.219293 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46d76d83b3174f5da337585b00af5f68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.221252 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3020dd34e96c426fbbca98e320405d0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.221757 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3020dd34e96c426fbbca98e320405d0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.222316 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91bea743ec0b491299f45f3ac23cb734" } modify_external_catalogs: true
I20260811 18:57:38.222497 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_91bea743ec0b491299f45f3ac23cb734" } modify_external_catalogs: true
I20260811 18:57:38.223379 4163 ts_tablet_manager.cc:1428] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.030s user 0.001s sys 0.001s
I20260811 18:57:38.224223 4162 consensus_queue.cc:260] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.224956 4162 ts_tablet_manager.cc:1428] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.050s user 0.001s sys 0.001s
I20260811 18:57:38.236351 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8" } modify_external_catalogs: true
I20260811 18:57:38.236601 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3799766ede4e4c8bb4e3ab35a723ebc8" } modify_external_catalogs: true
I20260811 18:57:38.240341 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_24401a03e8d54050a74da4ed3bbb60fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.241240 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24401a03e8d54050a74da4ed3bbb60fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.241572 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc1e05602be443528f758fba1ac0195f" } modify_external_catalogs: true
I20260811 18:57:38.241809 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dc1e05602be443528f758fba1ac0195f" } modify_external_catalogs: true
I20260811 18:57:38.246124 4162 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.247702 3552 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 43fcb5054edb47a3bea8bcd1fbef036e
W20260811 18:57:38.249675 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:38.249855 3824 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.249996 3824 leader_election.cc:334] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.250083 3824 leader_election.cc:302] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.250393 5593 raft_consensus.cc:2726] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.251508 4163 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.252681 4167 ts_tablet_manager.cc:1918] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.252858 4167 log.cc:1196] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/19243ab3d6744aaa87a133f1462cd2fa
I20260811 18:57:38.253206 4167 ts_tablet_manager.cc:1939] T 19243ab3d6744aaa87a133f1462cd2fa P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.253723 4163 tablet_bootstrap.cc:654] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.254091 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1062) took 2388 ms. Trace:
I20260811 18:57:38.261983 4163 tablet_bootstrap.cc:492] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.262388 4163 ts_tablet_manager.cc:1397] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260811 18:57:38.263266 4163 raft_consensus.cc:348] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.263515 4163 raft_consensus.cc:374] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.263572 4163 raft_consensus.cc:729] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.263839 4163 consensus_queue.cc:260] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.264636 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ef47211aa8864d13a55551da47804c19 (table default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c]) successfully deleted
I20260811 18:57:38.265864 4162 tablet_bootstrap.cc:654] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.266589 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 22e1a33348e34e35bab786f8966c408b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22e1a33348e34e35bab786f8966c408b already in progress: creating tablet
W20260811 18:57:38.266829 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6f804703e20c4214b8b2a31efeaef59e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f804703e20c4214b8b2a31efeaef59e already in progress: creating tablet
W20260811 18:57:38.267043 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 9b931eca7ba04c11af6571cb418f882e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b931eca7ba04c11af6571cb418f882e already in progress: creating tablet
I20260811 18:57:38.268746 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269059 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 69059a0667234a408cc1c09585df63bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269263 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b04e37fc264fc08e0ed9517356a17d with delete_type TABLET_DATA_DELETED (replaced by tablet 94857ef729eb435c97d089b8ab108cea) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269455 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet d11b889e9d044a049d0586382c263951 with delete_type TABLET_DATA_DELETED (replaced by tablet e4441c724b394e8092fb19c0d2427233) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.269833 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0e443ab84424dac307b745b496986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.270392 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 31bb11617f1548f88f534289706edb06 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.274178 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5437383c0bbb4384825f578e83c10acc" } new_table_name: "default.Table_5d568786ed494eae8f240cd653776099" modify_external_catalogs: true
I20260811 18:57:38.275642 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_51094e8e45fe427789a0f442148332f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.276449 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_325467f49f2648b5a83d5fc618fb302d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.277148 3678 consensus_peers.cc:487] T 9ad616256b9949ceaf8b300f660d00c5 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.277582 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_325467f49f2648b5a83d5fc618fb302d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.277762 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51094e8e45fe427789a0f442148332f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.280122 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_af28562bfbf94af98e28afef24fda608" } new_table_name: "default.Table_af422068a6ec4b0e8bb6f9851275fefc" modify_external_catalogs: true
I20260811 18:57:38.254220 4167 rpcz_store.cc:276] 0811 18:57:35.865809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.865876 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:38.143381 (+2277505us) ts_tablet_manager.cc:1145] Deleting tablet 19243ab3d6744aaa87a133f1462cd2fa
0811 18:57:38.143390 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.203452 (+ 60062us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.252565 (+ 49113us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.254061 (+ 1496us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2277282}
I20260811 18:57:38.283133 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_a055754403774bc0a94fda2f957c70d5" } new_table_name: "default.Table_18ec139c6f2e4ba6b43bc08eb73f85c6" modify_external_catalogs: true
I20260811 18:57:38.292543 3824 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:38.292811 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a385abaeb70347dcac06a5cedbdddeb4 consensus state has no leader
I20260811 18:57:38.293274 4167 tablet_replica.cc:330] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.293228 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34d2346219e14bd58aa02228ad109763" } modify_external_catalogs: true
I20260811 18:57:38.293462 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_34d2346219e14bd58aa02228ad109763" } modify_external_catalogs: true
I20260811 18:57:38.293525 4167 raft_consensus.cc:2229] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:38.293898 4167 raft_consensus.cc:2258] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.295122 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 6710084f21734dea9c7ce1f43d1bf587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.296507 5593 raft_consensus.cc:2781] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.296629 5593 raft_consensus.cc:482] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.296715 5593 raft_consensus.cc:3037] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.297103 4167 ts_tablet_manager.cc:1905] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.297307 3939 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:38.299571 5593 raft_consensus.cc:504] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.300262 5593 leader_election.cc:288] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.296502 4169 ts_tablet_manager.cc:1918] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.302549 4169 log.cc:1196] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b3f0cdcee681460084662e01152ad9dd
I20260811 18:57:38.302910 4169 ts_tablet_manager.cc:1939] T b3f0cdcee681460084662e01152ad9dd P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.303278 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 956) took 3966 ms. Trace:
I20260811 18:57:38.303378 4169 rpcz_store.cc:276] 0811 18:57:34.336408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336548 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:38.154058 (+3817510us) ts_tablet_manager.cc:1145] Deleting tablet b3f0cdcee681460084662e01152ad9dd
0811 18:57:38.154073 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.258486 (+104413us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.296353 (+ 37867us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.303257 (+ 6904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3802172}
I20260811 18:57:38.303730 4169 tablet_replica.cc:330] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.303956 4169 raft_consensus.cc:2229] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:38.304250 4169 pending_rounds.cc:62] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:38.304308 4169 pending_rounds.cc:68] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400182754709504 op_type: NO_OP noop_request { }
I20260811 18:57:38.304437 4169 raft_consensus.cc:2866] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:38.306036 4169 raft_consensus.cc:2258] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.307075 4162 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.307423 4162 ts_tablet_manager.cc:1397] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.004s
I20260811 18:57:38.307739 4163 ts_tablet_manager.cc:1428] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.045s user 0.002s sys 0.000s
I20260811 18:57:38.308605 4162 raft_consensus.cc:348] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.308987 4162 raft_consensus.cc:374] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.309110 4162 raft_consensus.cc:729] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.309458 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e79b6e054e914fd39ca0a6282eb0ce17" } new_table_name: "default.Table_9beab5ab1dab4912a4c7ff553fd620f8" modify_external_catalogs: true
I20260811 18:57:38.310650 5561 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.310784 5561 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.311297 4162 consensus_queue.cc:260] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.311482 5561 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.312083 4162 ts_tablet_manager.cc:1428] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 18:57:38.319370 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cce97bfdb1d41e685e091c8c5cc6946 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcfbf6f46e34163a230c0c82df7ecfa) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.321998 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb1cb45730b14b998a8f547983b83823 consensus state has no leader
I20260811 18:57:38.325484 3550 catalog_manager.cc:5495] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:38.325999 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]). Expected version 4 got 0
W20260811 18:57:38.326489 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c142d8ef40947e29f3c94e1e8e4c517 consensus state has no leader
I20260811 18:57:38.332681 4162 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.334846 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7057b8a7bae249caaed304b5c66e6871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:23 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.335638 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd39dcc0bed4623a06f00a56b740dba" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:38.335899 3745 raft_consensus.cc:3037] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.337106 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:38.338855 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b3f0cdcee681460084662e01152ad9dd (table default.Table_c33809efd2a943a3bfd700f8cf0750c6 [id=14a0795e7e104084b14f3fb1ca06ffb1]) successfully deleted
W20260811 18:57:38.340088 3824 leader_election.cc:334] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.340559 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:38.340838 3740 raft_consensus.cc:2454] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:38.343282 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 31bb11617f1548f88f534289706edb06
I20260811 18:57:38.343706 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 31bb11617f1548f88f534289706edb06 deleted and replaced by tablet 60f82aa67c614c6abcac348b67255904
I20260811 18:57:38.343703 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 76506ad179a9408c9e4cf8e4275ebddd
I20260811 18:57:38.343869 4169 ts_tablet_manager.cc:1905] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:38.343317 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23c85b2315254e4c8d9d69fec4cc9aa2 consensus state has no leader
I20260811 18:57:38.345959 4162 tablet_bootstrap.cc:654] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.347374 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 112d72e213dc415c959f8562bb83b31b (DEFAULT_TABLE table=default.Table_0c6490a83dec455bac3fe91b50da8a95 [id=d985a8592d304a808a31df85c9bba953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.347920 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112d72e213dc415c959f8562bb83b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.350590 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_149093c5dbb2402ebc6afe1a069fb195" } new_table_name: "default.Table_7cd9fda507254d029c62090b18422521" modify_external_catalogs: true
I20260811 18:57:38.350737 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9042bc231f0841a4a370be5e6251dd6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.351383 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9042bc231f0841a4a370be5e6251dd6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.352990 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bce1d6623de1467198e1dfaf763f08cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.353330 4167 ts_tablet_manager.cc:1918] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:38.353453 4167 log.cc:1196] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e3021c51525846e199f279f988248af0
W20260811 18:57:38.353523 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bce1d6623de1467198e1dfaf763f08cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.353873 4167 ts_tablet_manager.cc:1939] T e3021c51525846e199f279f988248af0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.354749 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1063) took 2467 ms. Trace:
I20260811 18:57:38.355444 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c7f500a5444bae9a015a5e903c0622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.355875 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 7fad52ead22f42dc9dd487557c9acce8 (DEFAULT_TABLE table=default.Table_bb1820be09714d0b976eda8e0151671e [id=f6550593f6194360b4e6d3d7ca1b2466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.356269 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fad52ead22f42dc9dd487557c9acce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.357465 4163 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.358933 4163 tablet_bootstrap.cc:654] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.360378 3852 tablet_service.cc:1467] Processing CreateTablet for tablet b48a82a7182b4bf89fb9fd3830829909 (DEFAULT_TABLE table=default.Table_cb4ec5ebc85e4562a65d10436234b4c2 [id=70fa03d99296435383f2d465179a431a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.360736 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48a82a7182b4bf89fb9fd3830829909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.370527 4167 rpcz_store.cc:276] 0811 18:57:35.887344 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.887413 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:38.282686 (+2395273us) ts_tablet_manager.cc:1145] Deleting tablet e3021c51525846e199f279f988248af0
0811 18:57:38.293207 (+ 10521us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.300894 (+ 7687us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.353213 (+ 52319us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.354722 (+ 1509us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2395034}
I20260811 18:57:38.371742 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 8dc07c28ccfd48ee9c33e49442c6047b (table default.Table_7f65859fb42941e3a8375f65182f9ec4 [id=cabc6cc20754488eb152c0d4f686dce0]). Expected version 2 got 0
I20260811 18:57:38.371945 3550 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet a811d4488d0e4541808a5e1aa44d663b (table default.Table_fb8dd203bf3941fbacfd7dad1418e566 [id=a71b71af19a44182aea7e742062558ca]). Expected version 1 got 0
I20260811 18:57:38.372364 4169 ts_tablet_manager.cc:1918] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:38.372505 4169 log.cc:1196] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3263da75f0d84528b7c8cc55dcaeaa46
I20260811 18:57:38.373301 4167 tablet_replica.cc:330] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.373530 4167 raft_consensus.cc:2229] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.373740 4167 raft_consensus.cc:2258] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.374495 4167 ts_tablet_manager.cc:1905] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.376283 4162 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.376559 4162 ts_tablet_manager.cc:1397] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.001s
W20260811 18:57:38.377035 3678 consensus_peers.cc:487] T 3c70d9db064e45af91fd9901d85e211f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.377713 4162 raft_consensus.cc:348] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.377967 4162 raft_consensus.cc:374] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.378044 4162 raft_consensus.cc:729] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.378271 4162 consensus_queue.cc:260] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.381407 4162 ts_tablet_manager.cc:1428] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:38.382083 4169 ts_tablet_manager.cc:1939] T 3263da75f0d84528b7c8cc55dcaeaa46 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:38.382951 5570 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 10282287177661667202) tablet 00000000000000000000000000000000 cur holder 0x617000081f80
I20260811 18:57:38.389910 4162 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.391688 4162 tablet_bootstrap.cc:654] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.393942 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 958) took 4057 ms. Trace:
I20260811 18:57:38.394326 3745 raft_consensus.cc:2454] T 0bd39dcc0bed4623a06f00a56b740dba P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:38.394150 4169 rpcz_store.cc:276] 0811 18:57:34.336659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.336708 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:38.303670 (+3966962us) ts_tablet_manager.cc:1145] Deleting tablet 3263da75f0d84528b7c8cc55dcaeaa46
0811 18:57:38.303682 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.349776 (+ 46094us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.372240 (+ 22464us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.393914 (+ 21674us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3951728}
I20260811 18:57:38.405823 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_97a83e019035401aa2d468bf20c5ec11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.406572 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97a83e019035401aa2d468bf20c5ec11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.406481 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b081a3f48c9048ecb88f1bd4571f934b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.407029 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b081a3f48c9048ecb88f1bd4571f934b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.408725 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_644eac4afde147dea1b6dc8cd69dc719"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.409207 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_644eac4afde147dea1b6dc8cd69dc719 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.399564 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.400170 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 8de4374609f04020b23ea6fe111a7cd7 (DEFAULT_TABLE table=default.Table_eb78d7140fc94f75b13bca56099bde32 [id=06a4c34fa672484692f4051cf30cf242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.417409 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de4374609f04020b23ea6fe111a7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.420979 3554 master_service.cc:946] ReplaceTablet: received request to replace tablet 4451cc4d7c5940cdb41c021ae486a5a9 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:38.401789 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bb11617f1548f88f534289706edb06 with delete_type TABLET_DATA_DELETED (replaced by tablet 60f82aa67c614c6abcac348b67255904) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.402595 3824 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.424618 4169 tablet_replica.cc:330] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.425923 5561 raft_consensus.cc:2781] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:38.426012 5561 raft_consensus.cc:482] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:38.426092 5561 raft_consensus.cc:3037] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.426306 4169 raft_consensus.cc:2229] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.428118 4169 raft_consensus.cc:2258] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.428826 4169 ts_tablet_manager.cc:1905] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:38.430831 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b203df4124364737839bf4c1ae072988: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.428861 5561 raft_consensus.cc:504] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.429677 3824 leader_election.cc:302] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.431579 5593 raft_consensus.cc:2781] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:38.431625 5561 leader_election.cc:288] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.431728 5593 raft_consensus.cc:686] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:38.431861 5557 consensus_queue.cc:1035] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.435388 3795 heartbeater.cc:347] Connected to a master server at 127.3.72.126:34821
I20260811 18:57:38.441047 4162 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.441488 4162 ts_tablet_manager.cc:1397] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.004s sys 0.000s
I20260811 18:57:38.443475 4162 raft_consensus.cc:348] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.444121 4162 raft_consensus.cc:374] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.444327 4162 raft_consensus.cc:729] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.444823 4162 consensus_queue.cc:260] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.431931 5593 consensus_queue.cc:237] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.453809 4162 ts_tablet_manager.cc:1428] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260811 18:57:38.459852 4167 ts_tablet_manager.cc:1918] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.460119 4167 log.cc:1196] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:38.461138 4162 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.463297 5622 raft_consensus.cc:482] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.463449 5622 raft_consensus.cc:504] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.464118 5622 leader_election.cc:288] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.466634 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f75dc515d7f3473b8e43f20255a70e3e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.467654 3676 leader_election.cc:341] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.467527 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_04519ba2021244b6922bea0e5b79ecb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.468242 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04519ba2021244b6922bea0e5b79ecb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.468281 3678 leader_election.cc:334] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.468389 3678 leader_election.cc:302] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.468873 5622 raft_consensus.cc:2726] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.469924 4167 ts_tablet_manager.cc:1939] T 62c365f79d4a4201ac323f16bddf3e62 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.472296 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 112d72e213dc415c959f8562bb83b31b (DEFAULT_TABLE table=default.Table_0c6490a83dec455bac3fe91b50da8a95 [id=d985a8592d304a808a31df85c9bba953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.472498 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 8de4374609f04020b23ea6fe111a7cd7 (DEFAULT_TABLE table=default.Table_eb78d7140fc94f75b13bca56099bde32 [id=06a4c34fa672484692f4051cf30cf242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.472807 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112d72e213dc415c959f8562bb83b31b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.473076 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d40b42e92ef24ade9a8d09a1ac910d82 (DEFAULT_TABLE table=default.Table_908f52c0075e4f6c8f60b6c84f13e004 [id=430c769710734a608fa3e3aa44488eb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.473420 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40b42e92ef24ade9a8d09a1ac910d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.474630 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 7fad52ead22f42dc9dd487557c9acce8 (DEFAULT_TABLE table=default.Table_bb1820be09714d0b976eda8e0151671e [id=f6550593f6194360b4e6d3d7ca1b2466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.475149 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fad52ead22f42dc9dd487557c9acce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.478395 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de4374609f04020b23ea6fe111a7cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.478443 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c7f500a5444bae9a015a5e903c0622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.478878 3705 tablet_service.cc:1467] Processing CreateTablet for tablet b48a82a7182b4bf89fb9fd3830829909 (DEFAULT_TABLE table=default.Table_cb4ec5ebc85e4562a65d10436234b4c2 [id=70fa03d99296435383f2d465179a431a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.479247 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48a82a7182b4bf89fb9fd3830829909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.482460 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bb11617f1548f88f534289706edb06 with delete_type TABLET_DATA_DELETED (replaced by tablet 60f82aa67c614c6abcac348b67255904) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.482717 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 76506ad179a9408c9e4cf8e4275ebddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:38.482522 4162 tablet_bootstrap.cc:654] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.490511 5622 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.490724 5622 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.491488 5622 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.494828 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.495952 3676 leader_election.cc:341] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.496351 3678 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.496526 3678 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.496814 5407 raft_consensus.cc:482] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.496944 5407 raft_consensus.cc:504] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.497555 5407 leader_election.cc:288] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.497691 5407 raft_consensus.cc:2726] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.499106 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eb43127dda814a718b338906fc2a3c47
I20260811 18:57:38.498993 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8304ac7709ca4392baec4cfa78ef5b41" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:38.499452 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_724b9320db2141f39b2511549a1aa8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.499927 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba12b590ed224215a8ea56022e39e6c0" } modify_external_catalogs: true
I20260811 18:57:38.500123 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ba12b590ed224215a8ea56022e39e6c0" } modify_external_catalogs: true
W20260811 18:57:38.500676 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f404fe05a8674605bb8038d676443046 has no consensus state
I20260811 18:57:38.501120 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4fd6bf06dd964e719ce1470aec9f60bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.501729 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fd6bf06dd964e719ce1470aec9f60bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.502576 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ee5b7d1e32514ad48177371582e85241
W20260811 18:57:38.504055 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_724b9320db2141f39b2511549a1aa8d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.505227 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_dbebdb613fa84efc9722613af36d52c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.505878 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbebdb613fa84efc9722613af36d52c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.509019 3676 leader_election.cc:341] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:38.509541 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_661f1ebd9f4c4cb9be591c5e1c203a86" } new_table_name: "default.Table_062f4ec20c7f4f41ace0731964dc4d96" modify_external_catalogs: true
I20260811 18:57:38.511410 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1064) took 2493 ms. Trace:
I20260811 18:57:38.512936 4167 rpcz_store.cc:276] 0811 18:57:36.018088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.018154 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:38.373241 (+2355087us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:38.373254 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.389349 (+ 16095us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.459714 (+ 70365us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.511359 (+ 51645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2354813}
I20260811 18:57:38.513612 4167 tablet_replica.cc:330] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.515020 4167 raft_consensus.cc:2229] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.515771 4169 ts_tablet_manager.cc:1918] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.516065 4167 raft_consensus.cc:2258] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.518306 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet 302cf69ea7c14fdaac72f6a9a9aff113 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:38.518786 3678 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.519058 3678 leader_election.cc:302] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.519641 4167 ts_tablet_manager.cc:1905] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.519869 4169 log.cc:1196] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/1e1374a80d15444cb0faff339275aa8d
I20260811 18:57:38.520365 4169 ts_tablet_manager.cc:1939] T 1e1374a80d15444cb0faff339275aa8d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.519842 5622 raft_consensus.cc:2726] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.523401 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 957) took 4186 ms. Trace:
I20260811 18:57:38.520520 5630 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.523713 5630 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.524482 5630 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.524681 4169 rpcz_store.cc:276] 0811 18:57:34.336524 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.343747 (+ 7223us) service_pool.cc:227] Handling call
0811 18:57:38.424558 (+4080811us) ts_tablet_manager.cc:1145] Deleting tablet 1e1374a80d15444cb0faff339275aa8d
0811 18:57:38.424573 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.453897 (+ 29324us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.515646 (+ 61749us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.523378 (+ 7732us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4065677}
I20260811 18:57:38.525413 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 961) took 3555 ms. Trace:
I20260811 18:57:38.525573 4169 rpcz_store.cc:276] 0811 18:57:34.970408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.970794 (+ 386us) service_pool.cc:227] Handling call
0811 18:57:38.525261 (+3554467us) ts_tablet_manager.cc:1145] Deleting tablet a25b1eddd4a64847b51ad3cdc0238ff0
0811 18:57:38.525275 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.525393 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3554326}
I20260811 18:57:38.526049 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 960) took 3555 ms. Trace:
W20260811 18:57:38.528191 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 96460.2us
I20260811 18:57:38.528615 4169 rpcz_store.cc:276] 0811 18:57:34.970271 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.970355 (+ 84us) service_pool.cc:227] Handling call
0811 18:57:38.525916 (+3555561us) ts_tablet_manager.cc:1145] Deleting tablet 40898ceb18cc4f2d9104818798049ae2
0811 18:57:38.525929 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.526031 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43,"tablet-delete.queue_time_us":3554979}
W20260811 18:57:38.529294 3824 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.528913 5557 consensus_queue.cc:1035] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.529649 4169 tablet_replica.cc:330] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:38.530625 4169 raft_consensus.cc:2229] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.530635 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_605185b9f67e456ba194522b24112e39" } new_table_name: "default.Table_e71f95e50a3b4ac188f0ca2b82fc95bb" modify_external_catalogs: true
I20260811 18:57:38.530871 4169 raft_consensus.cc:2258] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:38.529820 3824 consensus_peers.cc:487] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.532382 4162 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.532622 4162 ts_tablet_manager.cc:1397] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.002s
I20260811 18:57:38.532665 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 1e1374a80d15444cb0faff339275aa8d (table default.Table_647ae82e63464976abb7f26c9526a349 [id=e0ef202b868f47269e4f2a63cb5a390a]) successfully deleted
I20260811 18:57:38.532779 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a6af8048b8f14af899345d8213953792"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.532967 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a25b1eddd4a64847b51ad3cdc0238ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25b1eddd4a64847b51ad3cdc0238ff0 already in progress: creating tablet
I20260811 18:57:38.533278 3852 tablet_service.cc:1467] Processing CreateTablet for tablet d40b42e92ef24ade9a8d09a1ac910d82 (DEFAULT_TABLE table=default.Table_908f52c0075e4f6c8f60b6c84f13e004 [id=430c769710734a608fa3e3aa44488eb9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:38.533416 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6af8048b8f14af899345d8213953792 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.533751 4162 raft_consensus.cc:348] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.534055 4162 raft_consensus.cc:374] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.534173 4162 raft_consensus.cc:729] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.534356 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 6168d1f0fffc4fad91581d1795d721ce (DEFAULT_TABLE table=default.Table_5ef097030b2146bb9df7f7d2bffde6b6 [id=0291bde7daf64dc480710af744852acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.534462 4162 consensus_queue.cc:260] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.535069 3745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4814a3562ef14fb0bec79be8c3cff4d2" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:38.535333 3745 raft_consensus.cc:3037] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.536222 4162 ts_tablet_manager.cc:1428] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:38.540298 3745 raft_consensus.cc:2454] T 4814a3562ef14fb0bec79be8c3cff4d2 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:38.542137 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 49203e259d17449984dcd528df985537 (DEFAULT_TABLE table=default.Table_d2326f1112004b75bdb9d55298314884 [id=9833505c53af4634911465bd22485158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.542380 4162 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.543453 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 071a6513025341caaa0cac048d3caccd (DEFAULT_TABLE table=default.Table_b882f1fc510a41558369a5ced806d29a [id=d744d4b03cb64b5e8eb2b02f7387733b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.545169 3867 tablet_service.cc:1467] Processing CreateTablet for tablet f10006748b084a6bbd1a2f6a55cca9b9 (DEFAULT_TABLE table=default.Table_35bcb0c44a4840879a1f1d1386bf24c1 [id=955bf462395b45a3b6fea1fce8dad61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.547215 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4451cc4d7c5940cdb41c021ae486a5a9
I20260811 18:57:38.547715 3554 catalog_manager.cc:6565] ReplaceTablet: tablet 4451cc4d7c5940cdb41c021ae486a5a9 deleted and replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5
I20260811 18:57:38.547878 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40b42e92ef24ade9a8d09a1ac910d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.551918 4162 tablet_bootstrap.cc:654] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.553933 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071a6513025341caaa0cac048d3caccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.562042 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d" } modify_external_catalogs: true
I20260811 18:57:38.563921 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1bea4820fb5f4e65acb2a1a36332ac2d" } modify_external_catalogs: true
I20260811 18:57:38.567185 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b069c186694049998175e3d83241b318" } new_table_name: "default.Table_9fbe81922b5e4d228c3b17450feb18d5" modify_external_catalogs: true
I20260811 18:57:38.568940 5622 raft_consensus.cc:482] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.569108 5622 raft_consensus.cc:504] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.569746 5622 leader_election.cc:288] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:38.571097 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4dc2df4c6464d71bb13fe25e0046758" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.572019 3676 leader_election.cc:341] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.573967 3678 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.574091 3678 leader_election.cc:302] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.574546 5407 raft_consensus.cc:2726] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.579466 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49203e259d17449984dcd528df985537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.582646 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10006748b084a6bbd1a2f6a55cca9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.586362 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6168d1f0fffc4fad91581d1795d721ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.617215 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c590c98e39f84ea09b26bbe8640ca93d
I20260811 18:57:38.632541 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" } modify_external_catalogs: true
I20260811 18:57:38.632828 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7e51195784cd42c8a84446d9a5de87af" } modify_external_catalogs: true
I20260811 18:57:38.632997 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_acd269829eb743178d173c6cc3d7301e" } new_table_name: "default.Table_d990f007030d4d1e82827f549e934fa9" modify_external_catalogs: true
I20260811 18:57:38.634455 5622 raft_consensus.cc:482] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.634529 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cb54beb34a654fbdb9c7632901bb0b9e
W20260811 18:57:38.634655 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 547f6c33d6414bcfa3eeff20cb9a0302 consensus state has no leader
I20260811 18:57:38.634647 5622 raft_consensus.cc:504] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.635471 5622 leader_election.cc:288] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.637477 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8319453f68a54da9813747152ef08756 consensus state has no leader
I20260811 18:57:38.639803 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42405aec21e3405486b6c658438b5fb2" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.641501 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39bfbcfc74834cbd9f4363e8c545d5a8 consensus state has no leader
W20260811 18:57:38.641667 3676 leader_election.cc:341] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.646564 3678 leader_election.cc:334] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.646759 3678 leader_election.cc:302] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.647899 5622 raft_consensus.cc:2726] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:38.655084 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec27862124b74126a62f8588ec1237fa consensus state has no leader
I20260811 18:57:38.666321 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5b7d1e32514ad48177371582e85241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.666814 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 with delete_type TABLET_DATA_DELETED (replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.667248 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c590c98e39f84ea09b26bbe8640ca93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.667593 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:38.668151 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 40898ceb18cc4f2d9104818798049ae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 40898ceb18cc4f2d9104818798049ae2 already in progress: creating tablet
W20260811 18:57:38.668720 3824 consensus_peers.cc:487] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.670769 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd (DEFAULT_TABLE table=default.Table_31dac67fa2c146b0956b5a3b25672da3 [id=4d02195d37e64519a56460e55acf9ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.671229 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.673641 3852 tablet_service.cc:1467] Processing CreateTablet for tablet fa49814fb45f481a9c16e30ea947c79a (DEFAULT_TABLE table=default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb [id=22079dd74afc459081d1eb743f58c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.674105 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa49814fb45f481a9c16e30ea947c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.678985 4163 tablet_bootstrap.cc:492] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.679157 4163 ts_tablet_manager.cc:1397] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.322s user 0.015s sys 0.010s
I20260811 18:57:38.680097 4163 raft_consensus.cc:348] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.680867 4163 raft_consensus.cc:374] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.680967 4163 raft_consensus.cc:729] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.681282 4163 consensus_queue.cc:260] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.682075 4163 ts_tablet_manager.cc:1428] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:38.685063 4169 ts_tablet_manager.cc:1905] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.690974 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:38.693135 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a7946503bec9446d8deb02448eb231d5 (DEFAULT_TABLE table=default.Table_c925e9ac591346b1a5142f338c68bcc6 [id=c5d66f3cc70d498eb0db5c57bb06442d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.697223 3860 tablet_service.cc:1467] Processing CreateTablet for tablet de822817a9634958ba0d1a3128691dba (DEFAULT_TABLE table=default.Table_5dc02b59b48d45c8bee6458f21962eeb [id=7ee28ea901454542818bc026827867b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.697592 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet eb43127dda814a718b338906fc2a3c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.697854 5593 raft_consensus.cc:482] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.698113 5593 raft_consensus.cc:504] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:38.698781 5593 leader_election.cc:288] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.700403 3555 catalog_manager.cc:5495] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:38.701009 3555 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:38.702051 4167 ts_tablet_manager.cc:1918] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.721114 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8e26f75764214905965fa0f514cb0028"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.721329 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_440c0b981c8a42c28bf5f00f50ad7296" } new_table_name: "default.Table_9db7061f3798449280da2e921e2181bf" modify_external_catalogs: true
W20260811 18:57:38.721794 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e26f75764214905965fa0f514cb0028 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.727005 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ee627ac9abf47889b5fed972cecd97e" } modify_external_catalogs: true
I20260811 18:57:38.728312 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b" } modify_external_catalogs: true
I20260811 18:57:38.735134 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bf1e0bf055c44f45835974098ec141f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.738173 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 09366eccf38a4876840cc5ddddbbb103
I20260811 18:57:38.742483 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9d42d349a68447e39dbdda4dea1dd35b" } modify_external_catalogs: true
I20260811 18:57:38.751243 5617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:38.751678 5617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:38.751713 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a82a7352eb644b4a6baf09d8084b652. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:38.752161 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:38.739476 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3ee627ac9abf47889b5fed972cecd97e" } modify_external_catalogs: true
W20260811 18:57:38.762194 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf1e0bf055c44f45835974098ec141f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.762766 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_53bcd19697ad49b9aec577b1646c0d3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.763475 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53bcd19697ad49b9aec577b1646c0d3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.764575 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9919772109274064aee942f6a7f1220f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.765128 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9919772109274064aee942f6a7f1220f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.767928 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7946503bec9446d8deb02448eb231d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.771272 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de822817a9634958ba0d1a3128691dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.776389 5593 consensus_queue.cc:1035] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:38.788113 5278 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47804 (request call id 0) took 7995 ms. Trace:
I20260811 18:57:38.797987 4163 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:38.800503 5557 raft_consensus.cc:482] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.800740 5557 raft_consensus.cc:504] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.801632 5557 leader_election.cc:288] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:38.802495 3824 leader_election.cc:341] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:38.803143 3824 leader_election.cc:302] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:38.803402 5557 raft_consensus.cc:2781] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:38.803511 5557 raft_consensus.cc:686] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:38.803743 5557 consensus_queue.cc:237] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.805439 5617 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:38.741443 5622 raft_consensus.cc:482] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.812592 4167 log.cc:1196] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:38.812608 5622 raft_consensus.cc:504] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.813050 4167 ts_tablet_manager.cc:1939] T 6b4de110bd3348f0b0dae21c9f2b6a19 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:38.813462 5622 leader_election.cc:288] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:38.772256 3678 consensus_peers.cc:487] T 55c1a7d44717415ca76a2d0b7e94394e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.814049 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1065) took 2795 ms. Trace:
I20260811 18:57:38.814146 4167 rpcz_store.cc:276] 0811 18:57:36.018784 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.018838 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:38.513557 (+2494719us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:38.513571 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.527740 (+ 14169us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.701924 (+174184us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.814024 (+112100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2494528}
I20260811 18:57:38.814561 4167 tablet_replica.cc:330] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:38.814790 4167 raft_consensus.cc:2229] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:38.815603 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa8a4a1c0cb4834be501bdd7d468472" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:38.816725 3676 leader_election.cc:341] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:38.818182 3678 leader_election.cc:334] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.818320 3678 leader_election.cc:302] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:38.820303 5622 raft_consensus.cc:2726] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.828917 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2486985deae44613a5f8d342c8236784"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.841296 3824 leader_election.cc:334] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:38.841697 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.841843 3824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 183 similar messages]
W20260811 18:57:38.842450 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:38.842581 3824 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:38.860497 4163 tablet_bootstrap.cc:654] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.866780 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2486985deae44613a5f8d342c8236784 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:38.869310 3678 consensus_peers.cc:487] T fb51e419dd24459caf3ba3adfbbd4517 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:38.870216 3678 consensus_peers.cc:487] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:38.871809 5646 raft_consensus.cc:2726] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:38.873688 5642 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:38.876310 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:38.884279 5646 raft_consensus.cc:482] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.884495 5646 raft_consensus.cc:504] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.885391 5646 leader_election.cc:288] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.886869 4162 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:38.887050 4162 ts_tablet_manager.cc:1397] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.345s user 0.025s sys 0.025s
I20260811 18:57:38.887842 4162 raft_consensus.cc:348] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.888108 4162 raft_consensus.cc:374] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.888170 4162 raft_consensus.cc:729] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.888448 4162 consensus_queue.cc:260] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.889104 4162 ts_tablet_manager.cc:1428] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:38.889415 4162 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:38.891542 4162 tablet_bootstrap.cc:654] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:38.897831 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:38.898624 4167 raft_consensus.cc:2258] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:38.899343 4167 ts_tablet_manager.cc:1905] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:38.900476 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ad53b06897e4559b15d34a4d98f4005" } modify_external_catalogs: true
I20260811 18:57:38.900734 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5ad53b06897e4559b15d34a4d98f4005" } modify_external_catalogs: true
I20260811 18:57:38.908847 4163 tablet_bootstrap.cc:492] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:38.909041 4163 ts_tablet_manager.cc:1397] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.111s user 0.003s sys 0.000s
I20260811 18:57:38.910053 4163 raft_consensus.cc:348] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.910313 4163 raft_consensus.cc:374] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.910408 4163 raft_consensus.cc:729] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.910645 4163 consensus_queue.cc:260] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.901520 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:38.923786 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.926374 4169 ts_tablet_manager.cc:1918] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:38.926548 4169 log.cc:1196] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/f7d4c26da86b49b8a37f2289f4ef6614
I20260811 18:57:38.926905 4169 ts_tablet_manager.cc:1939] T f7d4c26da86b49b8a37f2289f4ef6614 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:38.927706 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 8072b2d13cbc45249f43411e9022d006 (DEFAULT_TABLE table=default.Table_46d76d83b3174f5da337585b00af5f68 [id=40394961c1b348b4b67b3322da0eb2c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.928375 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 812cc50dcac740c5ab7633abc04c1d9e (DEFAULT_TABLE table=default.Table_3020dd34e96c426fbbca98e320405d0f [id=7f1262b7d1754c54aa332bc31a4db6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.928424 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.928942 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 81f52a745f6243ae9b1ef2dd33b06d3a (DEFAULT_TABLE table=default.Table_fe6c2229e23048eb81c61314102eab38 [id=57de9419daff451a9ca6485e9230e621]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:38.929075 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eae6865a9edf4e0ebe72f448ac7ab770 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:38.929365 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 09366eccf38a4876840cc5ddddbbb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:38.929854 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 258d08de21434edc86f0cab49f839294 (DEFAULT_TABLE table=default.Table_2dffa8e09c6948c092cfdd4d888e5f81 [id=58c3e7941755444c978bd70d45db09f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.931057 3553 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (table default.Table_c71b8760db044bad851b4bcd819bf75f [id=7a869d9623bc4e07938b2d26688723a5]). Expected version 2 got 0
I20260811 18:57:38.936331 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 407b07d906d4447dbae5fd45d44e3ef8 (DEFAULT_TABLE table=default.Table_24401a03e8d54050a74da4ed3bbb60fd [id=936d78e258b341f98d02313652a325f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.937434 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f404fe05a8674605bb8038d676443046 (DEFAULT_TABLE table=default.Table_7cd9fda507254d029c62090b18422521 [id=50dde5de39e14c068d494813f4e68a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.938439 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 408c18a220d94809a5c43e1bfca310c8 (DEFAULT_TABLE table=default.Table_325467f49f2648b5a83d5fc618fb302d [id=4dcf548dd1e84122a82fa8971e993312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.938609 3865 tablet_service.cc:1467] Processing CreateTablet for tablet f87a8741e8ec481ea21284abe2d6c479 (DEFAULT_TABLE table=default.Table_788572afacab482590222fc4d084ef0f [id=77992422c86e47c79d4e1aea0a66b089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.939399 3856 tablet_service.cc:1467] Processing CreateTablet for tablet d3e28a19c145403485376b057bad4c9e (DEFAULT_TABLE table=default.Table_51094e8e45fe427789a0f442148332f0 [id=7aa13fc26748491db551f61611f7c62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.940733 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 60f82aa67c614c6abcac348b67255904 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.942132 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 9c048122b5434ec594fbf5b2665edfab (DEFAULT_TABLE table=default.Table_0765ca548a9649e88a2ad253c392107b [id=46788ef8b34947bbbffd5f651a41a102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.943665 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b752724648a499980ad71fd596a731f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:38.943863 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 94857ef729eb435c97d089b8ab108cea (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:38.948807 5557 raft_consensus.cc:482] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:38.949553 5557 raft_consensus.cc:504] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:38.952075 5557 leader_election.cc:288] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:38.953348 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b07d906d4447dbae5fd45d44e3ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.953963 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c18a220d94809a5c43e1bfca310c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.954150 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e28a19c145403485376b057bad4c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.954457 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f404fe05a8674605bb8038d676443046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.958539 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f82aa67c614c6abcac348b67255904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.962598 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f87a8741e8ec481ea21284abe2d6c479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.982892 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c048122b5434ec594fbf5b2665edfab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:38.986589 3678 consensus_peers.cc:487] T 21054bdd3d7c400c95b1f41e41bafc79 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:38.987039 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94857ef729eb435c97d089b8ab108cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.987723 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 962) took 4016 ms. Trace:
W20260811 18:57:39.010433 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b752724648a499980ad71fd596a731f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.011677 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 930b6e8839e24ec9a69854349bc3ecf8 (table default.Table_9db7061f3798449280da2e921e2181bf [id=d7d11af2eee54ab880a15c525b7ad0a9], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 930b6e8839e24ec9a69854349bc3ecf8 consensus state has no leader
I20260811 18:57:39.027179 4169 rpcz_store.cc:276] 0811 18:57:34.971091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.971687 (+ 596us) service_pool.cc:227] Handling call
0811 18:57:38.529595 (+3557908us) ts_tablet_manager.cc:1145] Deleting tablet f7d4c26da86b49b8a37f2289f4ef6614
0811 18:57:38.529607 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:38.716429 (+186822us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.926070 (+209641us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:38.987700 (+ 61630us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3557491}
I20260811 18:57:39.029419 4169 tablet_replica.cc:330] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.030401 4169 raft_consensus.cc:2229] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.031582 4169 raft_consensus.cc:2258] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:38.988991 3824 consensus_peers.cc:487] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:39.033349 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 44328.3us
I20260811 18:57:39.019375 4163 ts_tablet_manager.cc:1428] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.110s user 0.004s sys 0.000s
I20260811 18:57:39.024945 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812cc50dcac740c5ab7633abc04c1d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:38.788347 5278 rpcz_store.cc:276] 0811 18:57:30.792355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:30.792444 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:36.978279 (+6185835us) raft_consensus.cc:1457] Updating replica for 662 ops
0811 18:57:37.340756 (+362477us) raft_consensus.cc:1520] Early marking committed up to index 398
0811 18:57:37.344297 (+ 3541us) raft_consensus.cc:1525] Triggering prepare for 662 ops
0811 18:57:37.345239 (+ 942us) write_op.cc:270] Start()
0811 18:57:37.345297 (+ 58us) write_op.cc:276] Timestamp: P: 1786474635810894 usec, L: 0
0811 18:57:37.353495 (+ 8198us) write_op.cc:270] Start()
0811 18:57:37.353546 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635811701 usec, L: 0
0811 18:57:37.359002 (+ 5456us) write_op.cc:270] Start()
0811 18:57:37.359037 (+ 35us) write_op.cc:276] Timestamp: P: 1786474635812439 usec, L: 0
0811 18:57:37.360195 (+ 1158us) write_op.cc:270] Start()
0811 18:57:37.360225 (+ 30us) write_op.cc:276] Timestamp: P: 1786474635813111 usec, L: 0
0811 18:57:37.361375 (+ 1150us) write_op.cc:270] Start()
0811 18:57:37.361404 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635813734 usec, L: 0
0811 18:57:37.368292 (+ 6888us) write_op.cc:270] Start()
0811 18:57:37.368343 (+ 51us) write_op.cc:276] Timestamp: P: 1786474635860354 usec, L: 0
0811 18:57:37.370113 (+ 1770us) write_op.cc:270] Start()
0811 18:57:37.370210 (+ 97us) write_op.cc:276] Timestamp: P: 1786474635876830 usec, L: 0
0811 18:57:37.370893 (+ 683us) write_op.cc:270] Start()
0811 18:57:37.370922 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635916988 usec, L: 0
0811 18:57:37.371482 (+ 560us) write_op.cc:270] Start()
0811 18:57:37.371511 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635988940 usec, L: 0
0811 18:57:37.375527 (+ 4016us) write_op.cc:270] Start()
0811 18:57:37.375571 (+ 44us) write_op.cc:276] Timestamp: P: 1786474635990013 usec, L: 0
0811 18:57:37.376256 (+ 685us) write_op.cc:270] Start()
0811 18:57:37.376285 (+ 29us) write_op.cc:276] Timestamp: P: 1786474635991607 usec, L: 0
0811 18:57:37.376949 (+ 664us) write_op.cc:270] Start()
0811 18:57:37.376977 (+ 28us) write_op.cc:276] Timestamp: P: 1786474635992238 usec, L: 0
0811 18:57:37.378434 (+ 1457us) write_op.cc:270] Start()
0811 18:57:37.378460 (+ 26us) write_op.cc:276] Timestamp: P: 1786474635992863 usec, L: 0
0811 18:57:37.379111 (+ 651us) write_op.cc:270] Start()
0811 18:57:37.379197 (+ 86us) write_op.cc:276] Timestamp: P: 1786474636034210 usec, L: 0
0811 18:57:37.380503 (+ 1306us) write_op.cc:270] Start()
0811 18:57:37.380526 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636035501 usec, L: 0
0811 18:57:37.381522 (+ 996us) write_op.cc:270] Start()
0811 18:57:37.381551 (+ 29us) write_op.cc:276] Timestamp: P: 1786474636036316 usec, L: 0
0811 18:57:37.381757 (+ 206us) write_op.cc:270] Start()
0811 18:57:37.381836 (+ 79us) write_op.cc:276] Timestamp: P: 1786474636037305 usec, L: 0
0811 18:57:37.385531 (+ 3695us) write_op.cc:270] Start()
0811 18:57:37.385565 (+ 34us) write_op.cc:276] Timestamp: P: 1786474636037997 usec, L: 0
0811 18:57:37.385799 (+ 234us) write_op.cc:270] Start()
0811 18:57:37.385820 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636070832 usec, L: 0
0811 18:57:37.385932 (+ 112us) write_op.cc:270] Start()
0811 18:57:37.385948 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636071659 usec, L: 0
0811 18:57:37.386067 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.386084 (+ 17us) write_op.cc:276] Timestamp: P: 1786474636072248 usec, L: 0
0811 18:57:37.386179 (+ 95us) write_op.cc:270] Start()
0811 18:57:37.386193 (+ 14us) write_op.cc:276] Timestamp: P: 1786474636072755 usec, L: 0
0811 18:57:37.386289 (+ 96us) write_op.cc:270] Start()
0811 18:57:37.386304 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636075267 usec, L: 0
0811 18:57:37.387746 (+ 1442us) write_op.cc:270] Start()
0811 18:57:37.387779 (+ 33us) write_op.cc:276] Timestamp: P: 1786474636083654 usec, L: 0
0811 18:57:37.387975 (+ 196us) write_op.cc:270] Start()
0811 18:57:37.387999 (+ 24us) write_op.cc:276] Timestamp: P: 1786474636214582 usec, L: 0
0811 18:57:37.388165 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.388188 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636264097 usec, L: 0
0811 18:57:37.401971 (+ 13783us) write_op.cc:270] Start()
0811 18:57:37.402033 (+ 62us) write_op.cc:276] Timestamp: P: 1786474636272706 usec, L: 0
0811 18:57:37.402267 (+ 234us) write_op.cc:270] Start()
0811 18:57:37.402292 (+ 25us) write_op.cc:276] Timestamp: P: 1786474636310865 usec, L: 0
0811 18:57:37.402463 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.402486 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636320577 usec, L: 0
0811 18:57:37.402642 (+ 156us) write_op.cc:270] Start()
0811 18:57:37.402664 (+ 22us) write_op.cc:276] Timestamp: P: 1786474636321455 usec, L: 0
0811 18:57:37.408058 (+ 5394us) write_op.cc:270] Start()
0811 18:57:37.408104 (+ 46us) write_op.cc:276] Timestamp: P: 1786474636322172 usec, L: 0
0811 18:57:37.408334 (+ 230us) write_op.cc:270] Start()
0811 18:57:37.408354 (+ 20us) write_op.cc:276] Timestamp: P: 1786474636328657 usec, L: 0
0811 18:57:37.408466 (+ 112us) write_op.cc:270] Start()
0811 18:57:37.408482 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636330376 usec, L: 0
0811 18:57:37.408582 (+ 100us) write_op.cc:270] Start()
0811 18:57:37.408598 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636339931 usec, L: 0
0811 18:57:37.408717 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.408734 (+ 17us) write_op.cc:276] Timestamp: P: 1786474636342761 usec, L: 0
0811 18:57:37.408837 (+ 103us) write_op.cc:270] Start()
0811 18:57:37.408852 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636344470 usec, L: 0
0811 18:57:37.408957 (+ 105us) write_op.cc:270] Start()
0811 18:57:37.408986 (+ 29us) write_op.cc:276] Timestamp: P: 1786474636347176 usec, L: 0
0811 18:57:37.409102 (+ 116us) write_op.cc:270] Start()
0811 18:57:37.409117 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636564973 usec, L: 0
0811 18:57:37.409217 (+ 100us) write_op.cc:270] Start()
0811 18:57:37.409232 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636576210 usec, L: 0
0811 18:57:37.409331 (+ 99us) write_op.cc:270] Start()
0811 18:57:37.409346 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636579204 usec, L: 0
0811 18:57:37.409443 (+ 97us) write_op.cc:270] Start()
0811 18:57:37.409459 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636640162 usec, L: 0
0811 18:57:37.409578 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.409594 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636703566 usec, L: 0
0811 18:57:37.409713 (+ 119us) write_op.cc:270] Start()
0811 18:57:37.409728 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636711299 usec, L: 0
0811 18:57:37.409825 (+ 97us) write_op.cc:270] Start()
0811 18:57:37.409840 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636719825 usec, L: 0
0811 18:57:37.409942 (+ 102us) write_op.cc:270] Start()
0811 18:57:37.409957 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636721967 usec, L: 0
0811 18:57:37.410229 (+ 272us) write_op.cc:270] Start()
0811 18:57:37.410251 (+ 22us) write_op.cc:276] Timestamp: P: 1786474636793721 usec, L: 0
0811 18:57:37.410385 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.410404 (+ 19us) write_op.cc:276] Timestamp: P: 1786474636795286 usec, L: 0
0811 18:57:37.410511 (+ 107us) write_op.cc:270] Start()
0811 18:57:37.410527 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636827331 usec, L: 0
0811 18:57:37.410684 (+ 157us) write_op.cc:270] Start()
0811 18:57:37.410705 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636863900 usec, L: 0
0811 18:57:37.410931 (+ 226us) write_op.cc:270] Start()
0811 18:57:37.410951 (+ 20us) write_op.cc:276] Timestamp: P: 1786474636865795 usec, L: 0
0811 18:57:37.411068 (+ 117us) write_op.cc:270] Start()
0811 18:57:37.411084 (+ 16us) write_op.cc:276] Timestamp: P: 1786474636886677 usec, L: 0
0811 18:57:37.411183 (+ 99us) write_op.cc:270] Start()
0811 18:57:37.411198 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636898148 usec, L: 0
0811 18:57:37.411305 (+ 107us) write_op.cc:270] Start()
0811 18:57:37.411320 (+ 15us) write_op.cc:276] Timestamp: P: 1786474636899078 usec, L: 0
0811 18:57:37.413936 (+ 2616us) write_op.cc:270] Start()
0811 18:57:37.413971 (+ 35us) write_op.cc:276] Timestamp: P: 1786474636899680 usec, L: 0
0811 18:57:37.414165 (+ 194us) write_op.cc:270] Start()
0811 18:57:37.414186 (+ 21us) write_op.cc:276] Timestamp: P: 1786474636900267 usec, L: 0
0811 18:57:37.414367 (+ 181us) write_op.cc:270] Start()
0811 18:57:37.414392 (+ 25us) write_op.cc:276] Timestamp: P: 1786474636900847 usec, L: 0
0811 18:57:37.414556 (+ 164us) write_op.cc:270] Start()
0811 18:57:37.414579 (+ 23us) write_op.cc:276] Timestamp: P: 1786474636995805 usec, L: 0
0811 18:57:37.414737 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.414764 (+ 27us) write_op.cc:276] Timestamp: P: 1786474636996759 usec, L: 0
0811 18:57:37.416086 (+ 1322us) write_op.cc:270] Start()
0811 18:57:37.416188 (+ 102us) write_op.cc:276] Timestamp: P: 1786474636997412 usec, L: 0
0811 18:57:37.416416 (+ 228us) write_op.cc:270] Start()
0811 18:57:37.416469 (+ 53us) write_op.cc:276] Timestamp: P: 1786474637165325 usec, L: 0
0811 18:57:37.416651 (+ 182us) write_op.cc:270] Start()
0811 18:57:37.416692 (+ 41us) write_op.cc:276] Timestamp: P: 1786474637166611 usec, L: 0
0811 18:57:37.418816 (+ 2124us) write_op.cc:270] Start()
0811 18:57:37.418851 (+ 35us) write_op.cc:276] Timestamp: P: 1786474637183576 usec, L: 0
0811 18:57:37.419729 (+ 878us) write_op.cc:270] Start()
0811 18:57:37.419759 (+ 30us) write_op.cc:276] Timestamp: P: 1786474637197348 usec, L: 0
0811 18:57:37.419930 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.419952 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637198400 usec, L: 0
0811 18:57:37.420118 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.420138 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637223619 usec, L: 0
0811 18:57:37.420280 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.420302 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637226023 usec, L: 0
0811 18:57:37.420437 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.420458 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637227353 usec, L: 0
0811 18:57:37.420593 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.423062 (+ 2469us) write_op.cc:276] Timestamp: P: 1786474637256743 usec, L: 0
0811 18:57:37.423274 (+ 212us) write_op.cc:270] Start()
0811 18:57:37.423301 (+ 27us) write_op.cc:276] Timestamp: P: 1786474637261701 usec, L: 0
0811 18:57:37.423453 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.423798 (+ 345us) write_op.cc:276] Timestamp: P: 1786474637456592 usec, L: 0
0811 18:57:37.423968 (+ 170us) write_op.cc:270] Start()
0811 18:57:37.423993 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637499754 usec, L: 0
0811 18:57:37.424137 (+ 144us) write_op.cc:270] Start()
0811 18:57:37.424162 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637527823 usec, L: 0
0811 18:57:37.424322 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.424344 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637529119 usec, L: 0
0811 18:57:37.424504 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.424526 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637530126 usec, L: 0
0811 18:57:37.429495 (+ 4969us) write_op.cc:270] Start()
0811 18:57:37.429534 (+ 39us) write_op.cc:276] Timestamp: P: 1786474637530649 usec, L: 0
0811 18:57:37.429729 (+ 195us) write_op.cc:270] Start()
0811 18:57:37.429753 (+ 24us) write_op.cc:276] Timestamp: P: 1786474637531214 usec, L: 0
0811 18:57:37.429898 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.429923 (+ 25us) write_op.cc:276] Timestamp: P: 1786474637531724 usec, L: 0
0811 18:57:37.430062 (+ 139us) write_op.cc:270] Start()
0811 18:57:37.430098 (+ 36us) write_op.cc:276] Timestamp: P: 1786474637532251 usec, L: 0
0811 18:57:37.430245 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.430301 (+ 56us) write_op.cc:276] Timestamp: P: 1786474637532782 usec, L: 0
0811 18:57:37.430550 (+ 249us) write_op.cc:270] Start()
0811 18:57:37.430577 (+ 27us) write_op.cc:276] Timestamp: P: 1786474637533284 usec, L: 0
0811 18:57:37.430763 (+ 186us) write_op.cc:270] Start()
0811 18:57:37.430789 (+ 26us) write_op.cc:276] Timestamp: P: 1786474637651573 usec, L: 0
0811 18:57:37.430938 (+ 149us) write_op.cc:270] Start()
0811 18:57:37.430961 (+ 23us) write_op.cc:276] Timestamp: P: 1786474637673259 usec, L: 0
0811 18:57:37.433983 (+ 3022us) write_op.cc:270] Start()
0811 18:57:37.434015 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637691547 usec, L: 0
0811 18:57:37.434194 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.434216 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637774199 usec, L: 0
0811 18:57:37.435209 (+ 993us) write_op.cc:270] Start()
0811 18:57:37.435241 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637782845 usec, L: 0
0811 18:57:37.435412 (+ 171us) write_op.cc:270] Start()
0811 18:57:37.435432 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637806484 usec, L: 0
0811 18:57:37.435563 (+ 131us) write_op.cc:270] Start()
0811 18:57:37.435585 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637820252 usec, L: 0
0811 18:57:37.435737 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.435763 (+ 26us) write_op.cc:276] Timestamp: P: 1786474637820988 usec, L: 0
0811 18:57:37.435897 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.435919 (+ 22us) write_op.cc:276] Timestamp: P: 1786474637854994 usec, L: 0
0811 18:57:37.436126 (+ 207us) write_op.cc:270] Start()
0811 18:57:37.436147 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637874197 usec, L: 0
0811 18:57:37.436311 (+ 164us) write_op.cc:270] Start()
0811 18:57:37.436332 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637891043 usec, L: 0
0811 18:57:37.436462 (+ 130us) write_op.cc:270] Start()
0811 18:57:37.436482 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637939412 usec, L: 0
0811 18:57:37.436614 (+ 132us) write_op.cc:270] Start()
0811 18:57:37.436646 (+ 32us) write_op.cc:276] Timestamp: P: 1786474637942606 usec, L: 0
0811 18:57:37.436772 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.436791 (+ 19us) write_op.cc:276] Timestamp: P: 1786474637966850 usec, L: 0
0811 18:57:37.436937 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.436957 (+ 20us) write_op.cc:276] Timestamp: P: 1786474637980702 usec, L: 0
0811 18:57:37.437109 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.437130 (+ 21us) write_op.cc:276] Timestamp: P: 1786474637998597 usec, L: 0
0811 18:57:37.437275 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.437296 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638017625 usec, L: 0
0811 18:57:37.437404 (+ 108us) write_op.cc:270] Start()
0811 18:57:37.437424 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638131431 usec, L: 0
0811 18:57:37.437552 (+ 128us) write_op.cc:270] Start()
0811 18:57:37.437571 (+ 19us) write_op.cc:276] Timestamp: P: 1786474638169916 usec, L: 0
0811 18:57:37.447267 (+ 9696us) write_op.cc:270] Start()
0811 18:57:37.447331 (+ 64us) write_op.cc:276] Timestamp: P: 1786474638189043 usec, L: 0
0811 18:57:37.447604 (+ 273us) write_op.cc:270] Start()
0811 18:57:37.447646 (+ 42us) write_op.cc:276] Timestamp: P: 1786474638190523 usec, L: 0
0811 18:57:37.447829 (+ 183us) write_op.cc:270] Start()
0811 18:57:37.447853 (+ 24us) write_op.cc:276] Timestamp: P: 1786474638229450 usec, L: 0
0811 18:57:37.448005 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.448028 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638242469 usec, L: 0
0811 18:57:37.448167 (+ 139us) write_op.cc:270] Start()
0811 18:57:37.448189 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638273772 usec, L: 0
0811 18:57:37.448331 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.448354 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638293460 usec, L: 0
0811 18:57:37.448494 (+ 140us) write_op.cc:270] Start()
0811 18:57:37.448516 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638303494 usec, L: 0
0811 18:57:37.448670 (+ 154us) write_op.cc:270] Start()
0811 18:57:37.448691 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638328460 usec, L: 0
0811 18:57:37.448833 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.448854 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638355124 usec, L: 0
0811 18:57:37.448989 (+ 135us) write_op.cc:270] Start()
0811 18:57:37.449010 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638356761 usec, L: 0
0811 18:57:37.449151 (+ 141us) write_op.cc:270] Start()
0811 18:57:37.449173 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638384373 usec, L: 0
0811 18:57:37.449310 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.449331 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638385461 usec, L: 0
0811 18:57:37.449453 (+ 122us) write_op.cc:270] Start()
0811 18:57:37.449473 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638386482 usec, L: 0
0811 18:57:37.449636 (+ 163us) write_op.cc:270] Start()
0811 18:57:37.449658 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638387925 usec, L: 0
0811 18:57:37.449791 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.449812 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638425419 usec, L: 0
0811 18:57:37.449954 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.449975 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638505835 usec, L: 0
0811 18:57:37.450133 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.450157 (+ 24us) write_op.cc:276] Timestamp: P: 1786474638520071 usec, L: 0
0811 18:57:37.450302 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.450323 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638545392 usec, L: 0
0811 18:57:37.454595 (+ 4272us) write_op.cc:270] Start()
0811 18:57:37.454634 (+ 39us) write_op.cc:276] Timestamp: P: 1786474638546829 usec, L: 0
0811 18:57:37.454779 (+ 145us) write_op.cc:270] Start()
0811 18:57:37.454801 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638549740 usec, L: 0
0811 18:57:37.455596 (+ 795us) write_op.cc:270] Start()
0811 18:57:37.455644 (+ 48us) write_op.cc:276] Timestamp: P: 1786474638550716 usec, L: 0
0811 18:57:37.455826 (+ 182us) write_op.cc:270] Start()
0811 18:57:37.455847 (+ 21us) write_op.cc:276] Timestamp: P: 1786474638614412 usec, L: 0
0811 18:57:37.456020 (+ 173us) write_op.cc:270] Start()
0811 18:57:37.456043 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638665699 usec, L: 0
0811 18:57:37.456234 (+ 191us) write_op.cc:270] Start()
0811 18:57:37.456262 (+ 28us) write_op.cc:276] Timestamp: P: 1786474638667216 usec, L: 0
0811 18:57:37.456422 (+ 160us) write_op.cc:270] Start()
0811 18:57:37.456442 (+ 20us) write_op.cc:276] Timestamp: P: 1786474638757069 usec, L: 0
0811 18:57:37.456568 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.456587 (+ 19us) write_op.cc:276] Timestamp: P: 1786474638759093 usec, L: 0
0811 18:57:37.460279 (+ 3692us) write_op.cc:270] Start()
0811 18:57:37.460311 (+ 32us) write_op.cc:276] Timestamp: P: 1786474638760108 usec, L: 0
0811 18:57:37.460476 (+ 165us) write_op.cc:270] Start()
0811 18:57:37.460494 (+ 18us) write_op.cc:276] Timestamp: P: 1786474638762390 usec, L: 0
0811 18:57:37.460642 (+ 148us) write_op.cc:270] Start()
0811 18:57:37.460660 (+ 18us) write_op.cc:276] Timestamp: P: 1786474638763339 usec, L: 0
0811 18:57:37.460857 (+ 197us) write_op.cc:270] Start()
0811 18:57:37.460874 (+ 17us) write_op.cc:276] Timestamp: P: 1786474638764255 usec, L: 0
0811 18:57:37.460988 (+ 114us) write_op.cc:270] Start()
0811 18:57:37.461018 (+ 30us) write_op.cc:276] Timestamp: P: 1786474638779863 usec, L: 0
0811 18:57:37.523213 (+ 62195us) write_op.cc:270] Start()
0811 18:57:37.523619 (+ 406us) write_op.cc:276] Timestamp: P: 1786474638801484 usec, L: 0
0811 18:57:37.526558 (+ 2939us) write_op.cc:270] Start()
0811 18:57:37.536976 (+ 10418us) write_op.cc:276] Timestamp: P: 1786474638843754 usec, L: 0
0811 18:57:37.555395 (+ 18419us) write_op.cc:270] Start()
0811 18:57:37.555474 (+ 79us) write_op.cc:276] Timestamp: P: 1786474638887467 usec, L: 0
0811 18:57:37.555802 (+ 328us) write_op.cc:270] Start()
0811 18:57:37.555836 (+ 34us) write_op.cc:276] Timestamp: P: 1786474638891001 usec, L: 0
0811 18:57:37.556035 (+ 199us) write_op.cc:270] Start()
0811 18:57:37.556063 (+ 28us) write_op.cc:276] Timestamp: P: 1786474638899786 usec, L: 0
0811 18:57:37.556288 (+ 225us) write_op.cc:270] Start()
0811 18:57:37.556315 (+ 27us) write_op.cc:276] Timestamp: P: 1786474638950524 usec, L: 0
0811 18:57:37.556491 (+ 176us) write_op.cc:270] Start()
0811 18:57:37.556514 (+ 23us) write_op.cc:276] Timestamp: P: 1786474638957680 usec, L: 0
0811 18:57:37.556651 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.556673 (+ 22us) write_op.cc:276] Timestamp: P: 1786474638975842 usec, L: 0
0811 18:57:37.556793 (+ 120us) write_op.cc:270] Start()
0811 18:57:37.556818 (+ 25us) write_op.cc:276] Timestamp: P: 1786474638981818 usec, L: 0
0811 18:57:37.556941 (+ 123us) write_op.cc:270] Start()
0811 18:57:37.556960 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639000496 usec, L: 0
0811 18:57:37.557086 (+ 126us) write_op.cc:270] Start()
0811 18:57:37.557105 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639035540 usec, L: 0
0811 18:57:37.557251 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.557271 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639056811 usec, L: 0
0811 18:57:37.557404 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.557427 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639063348 usec, L: 0
0811 18:57:37.557541 (+ 114us) write_op.cc:270] Start()
0811 18:57:37.557579 (+ 38us) write_op.cc:276] Timestamp: P: 1786474639097605 usec, L: 0
0811 18:57:37.557727 (+ 148us) write_op.cc:270] Start()
0811 18:57:37.557748 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639103307 usec, L: 0
0811 18:57:37.557864 (+ 116us) write_op.cc:270] Start()
0811 18:57:37.557882 (+ 18us) write_op.cc:276] Timestamp: P: 1786474639127342 usec, L: 0
0811 18:57:37.558012 (+ 130us) write_op.cc:270] Start()
0811 18:57:37.558034 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639133744 usec, L: 0
0811 18:57:37.558167 (+ 133us) write_op.cc:270] Start()
0811 18:57:37.558207 (+ 40us) write_op.cc:276] Timestamp: P: 1786474639154690 usec, L: 0
0811 18:57:37.574443 (+ 16236us) write_op.cc:270] Start()
0811 18:57:37.574499 (+ 56us) write_op.cc:276] Timestamp: P: 1786474639163269 usec, L: 0
0811 18:57:37.574785 (+ 286us) write_op.cc:270] Start()
0811 18:57:37.574809 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639186118 usec, L: 0
0811 18:57:37.574994 (+ 185us) write_op.cc:270] Start()
0811 18:57:37.575025 (+ 31us) write_op.cc:276] Timestamp: P: 1786474639192838 usec, L: 0
0811 18:57:37.575162 (+ 137us) write_op.cc:270] Start()
0811 18:57:37.575183 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639194056 usec, L: 0
0811 18:57:37.575337 (+ 154us) write_op.cc:270] Start()
0811 18:57:37.575359 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639195224 usec, L: 0
0811 18:57:37.575497 (+ 138us) write_op.cc:270] Start()
0811 18:57:37.575518 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639196021 usec, L: 0
0811 18:57:37.579612 (+ 4094us) write_op.cc:270] Start()
0811 18:57:37.579657 (+ 45us) write_op.cc:276] Timestamp: P: 1786474639196830 usec, L: 0
0811 18:57:37.579892 (+ 235us) write_op.cc:270] Start()
0811 18:57:37.579918 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639237770 usec, L: 0
0811 18:57:37.580076 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.580103 (+ 27us) write_op.cc:276] Timestamp: P: 1786474639275400 usec, L: 0
0811 18:57:37.580291 (+ 188us) write_op.cc:270] Start()
0811 18:57:37.580321 (+ 30us) write_op.cc:276] Timestamp: P: 1786474639284916 usec, L: 0
0811 18:57:37.580505 (+ 184us) write_op.cc:270] Start()
0811 18:57:37.580531 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639301118 usec, L: 0
0811 18:57:37.580725 (+ 194us) write_op.cc:270] Start()
0811 18:57:37.580751 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639333396 usec, L: 0
0811 18:57:37.580926 (+ 175us) write_op.cc:270] Start()
0811 18:57:37.580952 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639365993 usec, L: 0
0811 18:57:37.581129 (+ 177us) write_op.cc:270] Start()
0811 18:57:37.581153 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639367848 usec, L: 0
0811 18:57:37.581308 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.581345 (+ 37us) write_op.cc:276] Timestamp: P: 1786474639384193 usec, L: 0
0811 18:57:37.581500 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.581523 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639385587 usec, L: 0
0811 18:57:37.581666 (+ 143us) write_op.cc:270] Start()
0811 18:57:37.581689 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639386926 usec, L: 0
0811 18:57:37.581845 (+ 156us) write_op.cc:270] Start()
0811 18:57:37.581867 (+ 22us) write_op.cc:276] Timestamp: P: 1786474639387893 usec, L: 0
0811 18:57:37.582051 (+ 184us) write_op.cc:270] Start()
0811 18:57:37.582180 (+ 129us) write_op.cc:276] Timestamp: P: 1786474639420510 usec, L: 0
0811 18:57:37.582438 (+ 258us) write_op.cc:270] Start()
0811 18:57:37.582467 (+ 29us) write_op.cc:276] Timestamp: P: 1786474639422108 usec, L: 0
0811 18:57:37.582633 (+ 166us) write_op.cc:270] Start()
0811 18:57:37.582656 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639423440 usec, L: 0
0811 18:57:37.582803 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.582824 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639457762 usec, L: 0
0811 18:57:37.582966 (+ 142us) write_op.cc:270] Start()
0811 18:57:37.582986 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639460913 usec, L: 0
0811 18:57:37.583114 (+ 128us) write_op.cc:270] Start()
0811 18:57:37.583134 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639554903 usec, L: 0
0811 18:57:37.583301 (+ 167us) write_op.cc:270] Start()
0811 18:57:37.583322 (+ 21us) write_op.cc:276] Timestamp: P: 1786474639590645 usec, L: 0
0811 18:57:37.583435 (+ 113us) write_op.cc:270] Start()
0811 18:57:37.583453 (+ 18us) write_op.cc:276] Timestamp: P: 1786474639592106 usec, L: 0
0811 18:57:37.583573 (+ 120us) write_op.cc:270] Start()
0811 18:57:37.583592 (+ 19us) write_op.cc:276] Timestamp: P: 1786474639593286 usec, L: 0
0811 18:57:37.583744 (+ 152us) write_op.cc:270] Start()
0811 18:57:37.583768 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639594797 usec, L: 0
0811 18:57:37.583947 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.583970 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639596454 usec, L: 0
0811 18:57:37.584135 (+ 165us) write_op.cc:270] Start()
0811 18:57:37.584163 (+ 28us) write_op.cc:276] Timestamp: P: 1786474639598451 usec, L: 0
0811 18:57:37.584337 (+ 174us) write_op.cc:270] Start()
0811 18:57:37.584362 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639665261 usec, L: 0
0811 18:57:37.584515 (+ 153us) write_op.cc:270] Start()
0811 18:57:37.584540 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639700714 usec, L: 0
0811 18:57:37.584713 (+ 173us) write_op.cc:270] Start()
0811 18:57:37.584738 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639702150 usec, L: 0
0811 18:57:37.584889 (+ 151us) write_op.cc:270] Start()
0811 18:57:37.584918 (+ 29us) write_op.cc:276] Timestamp: P: 1786474639703880 usec, L: 0
0811 18:57:37.585097 (+ 179us) write_op.cc:270] Start()
0811 18:57:37.585117 (+ 20us) write_op.cc:276] Timestamp: P: 1786474639705100 usec, L: 0
0811 18:57:37.585264 (+ 147us) write_op.cc:270] Start()
0811 18:57:37.585287 (+ 23us) write_op.cc:276] Timestamp: P: 1786474639760705 usec, L: 0
0811 18:57:37.585442 (+ 155us) write_op.cc:270] Start()
0811 18:57:37.585466 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639784262 usec, L: 0
0811 18:57:37.585624 (+ 158us) write_op.cc:270] Start()
0811 18:57:37.585648 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639786061 usec, L: 0
0811 18:57:37.585836 (+ 188us) write_op.cc:270] Start()
0811 18:57:37.585863 (+ 27us) write_op.cc:276] Timestamp: P: 1786474639812909 usec, L: 0
0811 18:57:37.586031 (+ 168us) write_op.cc:270] Start()
0811 18:57:37.586055 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639846001 usec, L: 0
0811 18:57:37.586246 (+ 191us) write_op.cc:270] Start()
0811 18:57:37.586271 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639848186 usec, L: 0
0811 18:57:37.618609 (+ 32338us) write_op.cc:270] Start()
0811 18:57:37.618660 (+ 51us) write_op.cc:276] Timestamp: P: 1786474639849712 usec, L: 0
0811 18:57:37.618910 (+ 250us) write_op.cc:270] Start()
0811 18:57:37.618940 (+ 30us) write_op.cc:276] Timestamp: P: 1786474639905300 usec, L: 0
0811 18:57:37.619109 (+ 169us) write_op.cc:270] Start()
0811 18:57:37.619135 (+ 26us) write_op.cc:276] Timestamp: P: 1786474639923827 usec, L: 0
0811 18:57:37.619307 (+ 172us) write_op.cc:270] Start()
0811 18:57:37.619332 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639925756 usec, L: 0
0811 18:57:37.619491 (+ 159us) write_op.cc:270] Start()
0811 18:57:37.619515 (+ 24us) write_op.cc:276] Timestamp: P: 1786474639927112 usec, L: 0
0811 18:57:37.619653 (+ 138us) write_op.cc:270] Start()
0811 18:57:37.619678 (+ 25us) write_op.cc:276] Timestamp: P: 1786474639928390 usec, L: 0
0811 18:57:37.619812 (+ 134us) write_op.cc:270] Start()
0811 18:57:37.619834 (+ 22us) write_op.cc:276] Timestamp: P: 1786474640021864 usec, L: 0
0811 18:57:37.619980 (+ 146us) write_op.cc:270] Start()
0811 18:57:37.620004 (+ 24us) write_op.cc:276] Timestamp: P: 17864
I20260811 18:57:39.048919 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d08de21434edc86f0cab49f839294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.054479 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8072b2d13cbc45249f43411e9022d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.055114 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f52a745f6243ae9b1ef2dd33b06d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.079494 5622 consensus_queue.cc:1035] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:39.102599 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1060 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:39.108290 5557 raft_consensus.cc:482] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.108477 5557 raft_consensus.cc:504] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:39.109071 5557 leader_election.cc:288] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.111642 4169 ts_tablet_manager.cc:1905] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.116914 3678 consensus_peers.cc:487] T 866ed614511649459b22a9fce55220b2 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.117565 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1ccd56a8b7cc454ea3f2f08b59374521
I20260811 18:57:39.118507 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 78868ff1f8024163ae1d782151100cd4
I20260811 18:57:39.122993 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c4b27a9eb0994efe9f4176f74c16a62e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.124233 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" } modify_external_catalogs: true
I20260811 18:57:39.124541 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_d7f8f7eb8a9d46448d28a439cfd49c8e" } modify_external_catalogs: true
I20260811 18:57:39.126206 5557 raft_consensus.cc:482] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.126528 5557 raft_consensus.cc:504] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.127195 5557 leader_election.cc:288] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:39.128374 4162 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.128595 4162 ts_tablet_manager.cc:1397] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.239s user 0.004s sys 0.000s
I20260811 18:57:39.129561 4162 raft_consensus.cc:348] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.129908 4162 raft_consensus.cc:374] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.130028 4162 raft_consensus.cc:729] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:39.130053 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4b27a9eb0994efe9f4176f74c16a62e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.130455 4162 consensus_queue.cc:260] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.131199 4162 ts_tablet_manager.cc:1428] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.136062 4162 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.138197 4162 tablet_bootstrap.cc:654] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.138846 4163 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.140586 4163 tablet_bootstrap.cc:654] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.142481 3824 leader_election.cc:341] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 58c515c153df4317b3fb427983ee15e0
I20260811 18:57:39.143096 5646 consensus_queue.cc:1035] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:39.143781 3824 leader_election.cc:341] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0cccebfc54484a9b9abffcce4e459fb5
W20260811 18:57:39.151459 3678 consensus_peers.cc:487] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.159533 4162 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.159957 4162 ts_tablet_manager.cc:1397] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.002s
I20260811 18:57:39.161032 4162 raft_consensus.cc:348] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.161381 4162 raft_consensus.cc:374] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.161502 4162 raft_consensus.cc:729] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.161850 4162 consensus_queue.cc:260] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.162703 4162 ts_tablet_manager.cc:1428] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:39.163074 4162 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.163576 4167 ts_tablet_manager.cc:1918] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.163724 4167 log.cc:1196] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:39.164062 4167 ts_tablet_manager.cc:1939] T 27cda53b7ca54346a7a22bf26f5db990 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.164932 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1066) took 3139 ms. Trace:
I20260811 18:57:39.165046 4167 rpcz_store.cc:276] 0811 18:57:36.025985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.026058 (+ 73us) service_pool.cc:227] Handling call
0811 18:57:38.814510 (+2788452us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:38.814522 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.012264 (+197742us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.163424 (+151160us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.164909 (+ 1485us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2788206}
I20260811 18:57:39.165527 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1076) took 2978 ms. Trace:
W20260811 18:57:39.169234 3824 leader_election.cc:341] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7057b8a7bae249caaed304b5c66e6871
W20260811 18:57:39.169797 3824 leader_election.cc:341] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:39.173426 5264 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" instance_seqno: 1786474620633424) as {username='slave'} at 127.3.72.66:42153; Asking this server to re-register.
I20260811 18:57:39.174685 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet f7d4c26da86b49b8a37f2289f4ef6614 (table default.Table_523277d801974e708b0f7e9c401bb36a [id=5600dcaaf83c4f978c3d47f79e01568c]) successfully deleted
I20260811 18:57:39.175380 4163 tablet_bootstrap.cc:492] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.175519 4163 ts_tablet_manager.cc:1397] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.004s
I20260811 18:57:39.176290 4163 raft_consensus.cc:348] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.176525 4163 raft_consensus.cc:374] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.176584 4163 raft_consensus.cc:729] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.176785 4163 consensus_queue.cc:260] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:39.177826 3824 leader_election.cc:334] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.177935 3824 leader_election.cc:302] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:39.178758 3824 leader_election.cc:334] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.178867 3824 leader_election.cc:302] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.179060 5593 raft_consensus.cc:2726] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.179241 5593 raft_consensus.cc:2726] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.181885 3939 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.188386 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1060 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:39.189359 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:39.189913 5617 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:39.190327 5407 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.190521 5407 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.191200 5407 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.193131 4163 ts_tablet_manager.cc:1428] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.018s user 0.000s sys 0.002s
I20260811 18:57:39.165642 4167 rpcz_store.cc:276] 0811 18:57:36.187397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.187448 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:39.165394 (+2977946us) ts_tablet_manager.cc:1145] Deleting tablet 1700ee40ac0d40198c5b8a161a90b6b6
0811 18:57:39.165405 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.165510 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2977730}
I20260811 18:57:39.195429 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1080) took 3007 ms. Trace:
I20260811 18:57:39.195592 4167 rpcz_store.cc:276] 0811 18:57:36.187788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.188574 (+ 786us) service_pool.cc:227] Handling call
0811 18:57:39.195299 (+3006725us) ts_tablet_manager.cc:1145] Deleting tablet 374c94a12e374d0e82180421c80d92d0
0811 18:57:39.195311 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.195408 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3006538}
I20260811 18:57:39.196058 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1079) took 3008 ms. Trace:
I20260811 18:57:39.196226 4167 rpcz_store.cc:276] 0811 18:57:36.187681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.188633 (+ 952us) service_pool.cc:227] Handling call
0811 18:57:39.195955 (+3007322us) ts_tablet_manager.cc:1145] Deleting tablet 38a4783168554053aa953dba5e893b6b
0811 18:57:39.195965 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.196041 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22,"tablet-delete.queue_time_us":3007182}
I20260811 18:57:39.196667 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1088) took 2850 ms. Trace:
I20260811 18:57:39.196820 4163 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.196805 4167 rpcz_store.cc:276] 0811 18:57:36.346290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.346400 (+ 110us) service_pool.cc:227] Handling call
0811 18:57:39.196551 (+2850151us) ts_tablet_manager.cc:1145] Deleting tablet 60091bd891a94cc4aac6ef47673a4ee5
0811 18:57:39.196562 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.196649 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2849864}
I20260811 18:57:39.197111 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1089) took 2849 ms. Trace:
I20260811 18:57:39.197196 4167 rpcz_store.cc:276] 0811 18:57:36.348102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.348172 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:39.197018 (+2848846us) ts_tablet_manager.cc:1145] Deleting tablet fe080f2071964da6b54f1ee5af98d918
0811 18:57:39.197027 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.197094 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2848604}
I20260811 18:57:39.197449 4167 tablet_replica.cc:330] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.197664 4167 raft_consensus.cc:2229] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.197829 4167 raft_consensus.cc:2258] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.198414 4167 ts_tablet_manager.cc:1905] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.199652 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.199797 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.200838 3678 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.203603 5266 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" instance_seqno: 1786474620110993) as {username='slave'} at 127.3.72.65:38671; Asking this server to re-register.
I20260811 18:57:39.207625 3795 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.208945 4163 tablet_bootstrap.cc:654] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.211526 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0b4fff40ba9d49828facafd431939fd4 (DEFAULT_TABLE table=default.Table_e8dcb132b5b5474b8ff081e54a55dd3f [id=58e6357b58b4481c9fad17e3aa498147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.211964 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4fff40ba9d49828facafd431939fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.212535 5266 ts_manager.cc:194] Registered new tserver with Master: 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.210564 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.211107 3867 tablet_service.cc:1467] Processing CreateTablet for tablet e4441c724b394e8092fb19c0d2427233 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.217983 5407 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.218513 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4441c724b394e8092fb19c0d2427233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.218371 5407 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.219100 5407 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.219853 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 1e51bb756dff43caace2c0626aed1443 (DEFAULT_TABLE table=default.Table_1afb343a7aa84568ad8ad65cab128123 [id=1fa01fbb7a4946da82085dab1e0a71ae]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:39.219872 3676 leader_election.cc:341] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.220014 3676 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.220459 5407 raft_consensus.cc:2726] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.220443 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.221073 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.221120 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet 78868ff1f8024163ae1d782151100cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.221762 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c8d542ae2489ba466ffadd518db12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:39.221985 3676 leader_election.cc:341] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.222465 3824 leader_election.cc:334] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.222632 3824 leader_election.cc:302] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.222916 5630 raft_consensus.cc:2726] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.223474 4169 ts_tablet_manager.cc:1918] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.226990 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e51bb756dff43caace2c0626aed1443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.234655 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 19243ab3d6744aaa87a133f1462cd2fa (table default.Table_133a44e94783442e8c27d4730310f996 [id=07fd2e17f2d64b5f8ff6304bff94f046]) successfully deleted
I20260811 18:57:39.234936 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e3021c51525846e199f279f988248af0 (table default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]) successfully deleted
I20260811 18:57:39.235183 4169 log.cc:1196] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/45fde25bcb40493aafb083e161aa6932
I20260811 18:57:39.235440 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]) successfully deleted
I20260811 18:57:39.235797 4169 ts_tablet_manager.cc:1939] T 45fde25bcb40493aafb083e161aa6932 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.235877 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]) successfully deleted
I20260811 18:57:39.236066 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 27cda53b7ca54346a7a22bf26f5db990 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
W20260811 18:57:39.236322 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1700ee40ac0d40198c5b8a161a90b6b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1700ee40ac0d40198c5b8a161a90b6b6 already in progress: creating tablet
W20260811 18:57:39.236541 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 374c94a12e374d0e82180421c80d92d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374c94a12e374d0e82180421c80d92d0 already in progress: creating tablet
I20260811 18:57:39.236558 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 963) took 4264 ms. Trace:
W20260811 18:57:39.236760 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 38a4783168554053aa953dba5e893b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38a4783168554053aa953dba5e893b6b already in progress: creating tablet
W20260811 18:57:39.236961 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 60091bd891a94cc4aac6ef47673a4ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60091bd891a94cc4aac6ef47673a4ee5 already in progress: creating tablet
I20260811 18:57:39.236799 4169 rpcz_store.cc:276] 0811 18:57:34.972256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.976524 (+ 4268us) service_pool.cc:227] Handling call
0811 18:57:39.029210 (+4052686us) ts_tablet_manager.cc:1145] Deleting tablet 45fde25bcb40493aafb083e161aa6932
0811 18:57:39.029226 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.167064 (+137838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.223364 (+ 56300us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.236536 (+ 13172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4052422}
I20260811 18:57:39.237423 4169 tablet_replica.cc:330] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.237715 4169 raft_consensus.cc:2229] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.237977 4169 raft_consensus.cc:2258] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.239013 4169 ts_tablet_manager.cc:1905] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.243861 3678 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.244064 3678 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.244354 5407 raft_consensus.cc:2726] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.248680 4162 tablet_bootstrap.cc:654] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.248663 3824 leader_election.cc:334] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.248833 3824 leader_election.cc:302] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.249106 5630 raft_consensus.cc:2726] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:39.249518 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.253810 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:39.258038 3824 leader_election.cc:334] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.257717 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 33a79243bd874861b16f43339adf777c (DEFAULT_TABLE table=default.Table_80140471fa194b22bf8347836f95fdca [id=6de24db40cfe4fb49177dedc3d64443b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.261942 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a79243bd874861b16f43339adf777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.266049 4167 ts_tablet_manager.cc:1918] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.267678 4163 tablet_bootstrap.cc:492] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:39.262578 3824 leader_election.cc:341] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 7251204325dd424192a6f58e18e71c34
I20260811 18:57:39.270314 4163 ts_tablet_manager.cc:1397] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.074s user 0.007s sys 0.003s
I20260811 18:57:39.270476 3824 leader_election.cc:302] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.267521 4162 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.274148 4162 ts_tablet_manager.cc:1397] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.111s user 0.004s sys 0.005s
I20260811 18:57:39.274148 5630 raft_consensus.cc:2726] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.274241 4163 raft_consensus.cc:348] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.275071 4162 raft_consensus.cc:348] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.275370 4162 raft_consensus.cc:374] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.275475 4162 raft_consensus.cc:729] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:39.275580 3824 leader_election.cc:341] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:39.275864 4162 consensus_queue.cc:260] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:39.276031 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fe080f2071964da6b54f1ee5af98d918 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe080f2071964da6b54f1ee5af98d918 already in progress: creating tablet
I20260811 18:57:39.276592 4162 ts_tablet_manager.cc:1428] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.274501 4163 raft_consensus.cc:374] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.279299 4162 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.279484 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 6168d1f0fffc4fad91581d1795d721ce (DEFAULT_TABLE table=default.Table_5ef097030b2146bb9df7f7d2bffde6b6 [id=0291bde7daf64dc480710af744852acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.279906 4163 raft_consensus.cc:729] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.279994 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6168d1f0fffc4fad91581d1795d721ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.280221 4163 consensus_queue.cc:260] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.280807 4163 ts_tablet_manager.cc:1428] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:39.281193 4162 tablet_bootstrap.cc:654] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.278366 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.282240 4167 log.cc:1196] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:39.282953 4167 ts_tablet_manager.cc:1939] T 452ee450778e4d33adfa7e2745b5d970 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.284665 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1090) took 2449 ms. Trace:
I20260811 18:57:39.286119 4163 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.286422 4167 rpcz_store.cc:276] 0811 18:57:36.835209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.835663 (+ 454us) service_pool.cc:227] Handling call
0811 18:57:39.197410 (+2361747us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:39.197418 (+ 8us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.246946 (+ 49528us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.265921 (+ 18975us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.284639 (+ 18718us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2359154}
I20260811 18:57:39.287200 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1093) took 2433 ms. Trace:
I20260811 18:57:39.289566 4163 tablet_bootstrap.cc:654] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.290100 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 45fde25bcb40493aafb083e161aa6932 (table default.Table_be4900dcd0a34c43a3c0cdcebbcfdc1c [id=4bc5d2d6894e40ebab164e5ffd6ac600]) successfully deleted
I20260811 18:57:39.290355 4167 rpcz_store.cc:276] 0811 18:57:36.854096 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.854798 (+ 702us) service_pool.cc:227] Handling call
0811 18:57:39.287054 (+2432256us) ts_tablet_manager.cc:1145] Deleting tablet 03c50dcffac54c7bac210eeaed340ca6
0811 18:57:39.287067 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.287181 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2432050}
I20260811 18:57:39.291034 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1100) took 2337 ms. Trace:
I20260811 18:57:39.294391 4167 rpcz_store.cc:276] 0811 18:57:36.954043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.954112 (+ 69us) service_pool.cc:227] Handling call
0811 18:57:39.290908 (+2336796us) ts_tablet_manager.cc:1145] Deleting tablet 73c519528f75423195b6bdf6683fd91b
0811 18:57:39.290921 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.291016 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2336560}
I20260811 18:57:39.295115 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1101) took 2334 ms. Trace:
I20260811 18:57:39.295279 4167 rpcz_store.cc:276] 0811 18:57:36.960813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.960902 (+ 89us) service_pool.cc:227] Handling call
0811 18:57:39.294975 (+2334073us) ts_tablet_manager.cc:1145] Deleting tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
0811 18:57:39.294988 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.295096 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2333853}
I20260811 18:57:39.295997 4167 tablet_replica.cc:330] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.296476 4167 raft_consensus.cc:2229] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:39.297240 4167 raft_consensus.cc:2258] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.298910 4167 ts_tablet_manager.cc:1905] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.298822 3824 leader_election.cc:334] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.298249 4162 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.303004 3824 leader_election.cc:302] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.303392 5557 raft_consensus.cc:2726] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.303527 4162 ts_tablet_manager.cc:1397] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
W20260811 18:57:39.303752 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.306247 4162 raft_consensus.cc:348] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.306645 4162 raft_consensus.cc:374] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.306774 4162 raft_consensus.cc:729] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.307089 4162 consensus_queue.cc:260] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.308158 4162 ts_tablet_manager.cc:1428] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:39.308511 4162 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:39.309760 3824 consensus_peers.cc:487] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:39.312929 4169 ts_tablet_manager.cc:1918] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.313158 4169 log.cc:1196] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0ac9bdc606be40558dfa33e1e7a8ebc7
I20260811 18:57:39.313560 4169 ts_tablet_manager.cc:1939] T 0ac9bdc606be40558dfa33e1e7a8ebc7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.313927 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 964) took 4337 ms. Trace:
I20260811 18:57:39.314149 4162 tablet_bootstrap.cc:654] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.314030 4169 rpcz_store.cc:276] 0811 18:57:34.976904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.977365 (+ 461us) service_pool.cc:227] Handling call
0811 18:57:39.237371 (+4260006us) ts_tablet_manager.cc:1145] Deleting tablet 0ac9bdc606be40558dfa33e1e7a8ebc7
0811 18:57:39.237383 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.272633 (+ 35250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.311819 (+ 39186us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.313905 (+ 2086us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4259808}
I20260811 18:57:39.314517 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 969) took 4327 ms. Trace:
I20260811 18:57:39.314617 4169 rpcz_store.cc:276] 0811 18:57:34.987152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.988817 (+ 1665us) service_pool.cc:227] Handling call
0811 18:57:39.314366 (+4325549us) ts_tablet_manager.cc:1145] Deleting tablet 0bfb54175c834517b7af291c942333cd
0811 18:57:39.314381 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.314497 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4325326}
I20260811 18:57:39.314965 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 971) took 4324 ms. Trace:
I20260811 18:57:39.315052 4169 rpcz_store.cc:276] 0811 18:57:34.990014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.994866 (+ 4852us) service_pool.cc:227] Handling call
0811 18:57:39.314854 (+4319988us) ts_tablet_manager.cc:1145] Deleting tablet 1c1896700e724204a37439f134a4d988
0811 18:57:39.314865 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.314946 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319778}
I20260811 18:57:39.315390 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 972) took 4320 ms. Trace:
I20260811 18:57:39.315476 4169 rpcz_store.cc:276] 0811 18:57:34.995293 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.995495 (+ 202us) service_pool.cc:227] Handling call
0811 18:57:39.315282 (+4319787us) ts_tablet_manager.cc:1145] Deleting tablet 4dd203b7eb8643f493b4642c0396e258
0811 18:57:39.315294 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.315374 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319574}
I20260811 18:57:39.315785 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 973) took 4319 ms. Trace:
I20260811 18:57:39.315862 4169 rpcz_store.cc:276] 0811 18:57:34.995983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.996027 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:39.315696 (+4319669us) ts_tablet_manager.cc:1145] Deleting tablet afa57ed239ac4afebe517cc48d879645
0811 18:57:39.315705 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.315771 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319454}
I20260811 18:57:39.316191 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 974) took 4319 ms. Trace:
I20260811 18:57:39.316294 4169 rpcz_store.cc:276] 0811 18:57:34.996781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:34.996832 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:39.316063 (+4319231us) ts_tablet_manager.cc:1145] Deleting tablet b27f9dbb52f34687a33c2ebf20e8f780
0811 18:57:39.316092 (+ 29us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.316175 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4319064}
I20260811 18:57:39.316615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 989) took 4248 ms. Trace:
I20260811 18:57:39.316709 4169 rpcz_store.cc:276] 0811 18:57:35.068216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.068271 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.316519 (+4248248us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:39.316530 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.316599 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4247993}
I20260811 18:57:39.317068 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 990) took 4248 ms. Trace:
I20260811 18:57:39.317162 4169 rpcz_store.cc:276] 0811 18:57:35.068664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.068719 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.316945 (+4248226us) ts_tablet_manager.cc:1145] Deleting tablet f86bfa65a8774b51bc7e7fe59442bf2f
0811 18:57:39.316956 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.317047 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4248050}
I20260811 18:57:39.317448 4169 tablet_replica.cc:330] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.317986 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0ac9bdc606be40558dfa33e1e7a8ebc7 (table default.Table_8f581cf3ac8147ac844b27d7c201d6c4 [id=4a64bb1d12324474a39317c1210869fc]) successfully deleted
W20260811 18:57:39.318272 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0bfb54175c834517b7af291c942333cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0bfb54175c834517b7af291c942333cd already in progress: creating tablet
W20260811 18:57:39.318473 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1c1896700e724204a37439f134a4d988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c1896700e724204a37439f134a4d988 already in progress: creating tablet
W20260811 18:57:39.318676 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4dd203b7eb8643f493b4642c0396e258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4dd203b7eb8643f493b4642c0396e258 already in progress: creating tablet
W20260811 18:57:39.318818 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet afa57ed239ac4afebe517cc48d879645 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa57ed239ac4afebe517cc48d879645 already in progress: creating tablet
W20260811 18:57:39.318957 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b27f9dbb52f34687a33c2ebf20e8f780 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b27f9dbb52f34687a33c2ebf20e8f780 already in progress: creating tablet
W20260811 18:57:39.319079 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d37e81d376e14907b13746cca831a96b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d37e81d376e14907b13746cca831a96b already in progress: creating tablet
W20260811 18:57:39.319223 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f86bfa65a8774b51bc7e7fe59442bf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f86bfa65a8774b51bc7e7fe59442bf2f already in progress: creating tablet
I20260811 18:57:39.322161 5593 consensus_queue.cc:1035] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:39.325944 4167 ts_tablet_manager.cc:1918] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:39.328663 4169 raft_consensus.cc:2229] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.328896 4169 raft_consensus.cc:2258] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.333117 4169 ts_tablet_manager.cc:1905] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.328980 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 071a6513025341caaa0cac048d3caccd (DEFAULT_TABLE table=default.Table_b882f1fc510a41558369a5ced806d29a [id=d744d4b03cb64b5e8eb2b02f7387733b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.335388 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071a6513025341caaa0cac048d3caccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.339212 3723 tablet_service.cc:1467] Processing CreateTablet for tablet fa49814fb45f481a9c16e30ea947c79a (DEFAULT_TABLE table=default.Table_3e5a8d33544b4b7fb0da2f60a46c6ecb [id=22079dd74afc459081d1eb743f58c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.339651 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa49814fb45f481a9c16e30ea947c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.341440 4163 tablet_bootstrap.cc:492] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.341683 4163 ts_tablet_manager.cc:1397] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.002s
I20260811 18:57:39.341777 4167 log.cc:1196] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/3ad5f39f59b8447cb3f15b9646e4b852
W20260811 18:57:39.342128 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 3247 similar messages]
I20260811 18:57:39.326694 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 49203e259d17449984dcd528df985537 (DEFAULT_TABLE table=default.Table_d2326f1112004b75bdb9d55298314884 [id=9833505c53af4634911465bd22485158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.329835 3710 tablet_service.cc:1467] Processing CreateTablet for tablet f10006748b084a6bbd1a2f6a55cca9b9 (DEFAULT_TABLE table=default.Table_35bcb0c44a4840879a1f1d1386bf24c1 [id=955bf462395b45a3b6fea1fce8dad61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.330188 3715 tablet_service.cc:1467] Processing CreateTablet for tablet a7946503bec9446d8deb02448eb231d5 (DEFAULT_TABLE table=default.Table_c925e9ac591346b1a5142f338c68bcc6 [id=c5d66f3cc70d498eb0db5c57bb06442d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.330600 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd (DEFAULT_TABLE table=default.Table_31dac67fa2c146b0956b5a3b25672da3 [id=4d02195d37e64519a56460e55acf9ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.331035 3722 tablet_service.cc:1467] Processing CreateTablet for tablet de822817a9634958ba0d1a3128691dba (DEFAULT_TABLE table=default.Table_5dc02b59b48d45c8bee6458f21962eeb [id=7ee28ea901454542818bc026827867b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.342577 4163 raft_consensus.cc:348] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.342784 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10006748b084a6bbd1a2f6a55cca9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.342804 4163 raft_consensus.cc:374] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.342857 4163 raft_consensus.cc:729] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.343102 4163 consensus_queue.cc:260] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.343672 4163 ts_tablet_manager.cc:1428] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:39.343948 4163 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.345706 4163 tablet_bootstrap.cc:654] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.347054 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bfd8bf6d9e04fb59d1e3b36256274fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.347261 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7946503bec9446d8deb02448eb231d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.348858 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de822817a9634958ba0d1a3128691dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.351104 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1365655a76d7454c8415102e75960152" } new_table_name: "default.Table_58a314da9a724012bc84997945c98047" modify_external_catalogs: true
I20260811 18:57:39.352777 5659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a82a7352eb644b4a6baf09d8084b652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } } }
I20260811 18:57:39.353015 5659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:39.355006 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:39.335007 4162 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.355544 4162 ts_tablet_manager.cc:1397] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.000s
I20260811 18:57:39.356257 4162 raft_consensus.cc:348] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.356444 4162 raft_consensus.cc:374] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.356487 4162 raft_consensus.cc:729] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.356732 4162 consensus_queue.cc:260] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.357780 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5affb8acc7e5466ba5ed988728ba50cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.358718 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_0f271e4958af4bfbb65af29a053ac2d4" } new_table_name: "default.Table_44bbdf86ad5b4ba4a43a3ff0819dbeca" modify_external_catalogs: true
I20260811 18:57:39.360396 4162 ts_tablet_manager.cc:1428] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:39.360685 4162 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.362399 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49203e259d17449984dcd528df985537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.362915 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8
I20260811 18:57:39.363829 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_dbebdb613fa84efc9722613af36d52c8" } new_table_name: "default.Table_51582df654e7406dbc574ba8de215baf" modify_external_catalogs: true
I20260811 18:57:39.368029 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4f6cd18a5e464333bbdc36290a29eaea" } new_table_name: "default.Table_69d238eee5ac443da8e2fa5a011c42c8" modify_external_catalogs: true
W20260811 18:57:39.370103 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5affb8acc7e5466ba5ed988728ba50cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.370589 4162 tablet_bootstrap.cc:654] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.370748 4167 ts_tablet_manager.cc:1939] T 3ad5f39f59b8447cb3f15b9646e4b852 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.372406 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.372558 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1102) took 2411 ms. Trace:
I20260811 18:57:39.375783 5641 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.380841 4167 rpcz_store.cc:276] 0811 18:57:36.961001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.982466 (+ 21465us) service_pool.cc:227] Handling call
0811 18:57:39.295948 (+2313482us) ts_tablet_manager.cc:1145] Deleting tablet 3ad5f39f59b8447cb3f15b9646e4b852
0811 18:57:39.295960 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.322798 (+ 26838us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.325841 (+ 3043us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.372535 (+ 46694us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2313030}
I20260811 18:57:39.381049 5641 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.381481 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1103) took 2317 ms. Trace:
I20260811 18:57:39.381575 4167 rpcz_store.cc:276] 0811 18:57:37.064490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.068157 (+ 3667us) service_pool.cc:227] Handling call
0811 18:57:39.381317 (+2313160us) ts_tablet_manager.cc:1145] Deleting tablet 7cb3e997b4454598bdce102e92296887
0811 18:57:39.381332 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.381458 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2312937}
I20260811 18:57:39.382061 5641 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:39.382126 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1104) took 2281 ms. Trace:
I20260811 18:57:39.382205 4167 rpcz_store.cc:276] 0811 18:57:37.100957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.101349 (+ 392us) service_pool.cc:227] Handling call
0811 18:57:39.382016 (+2280667us) ts_tablet_manager.cc:1145] Deleting tablet 8f9b279c6c7344f1931b81e10248b1b6
0811 18:57:39.382029 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.382110 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2269657}
I20260811 18:57:39.383670 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.384356 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1122) took 2195 ms. Trace:
I20260811 18:57:39.384465 4167 rpcz_store.cc:276] 0811 18:57:37.188944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.188997 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:39.384247 (+2195250us) ts_tablet_manager.cc:1145] Deleting tablet ea579c33f8b14e9fa7fa389ce7f45931
0811 18:57:39.384259 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.384339 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2195065}
W20260811 18:57:39.384590 3676 leader_election.cc:341] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.388573 3678 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.389096 3678 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.389369 5641 raft_consensus.cc:2726] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.389849 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1121) took 2201 ms. Trace:
I20260811 18:57:39.390380 4167 rpcz_store.cc:276] 0811 18:57:37.188849 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.189059 (+ 210us) service_pool.cc:227] Handling call
0811 18:57:39.389728 (+2200669us) ts_tablet_manager.cc:1145] Deleting tablet 7f4e1fc5686944dfa3eb57702c810eb8
0811 18:57:39.389739 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.389833 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2200476}
I20260811 18:57:39.392184 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1132) took 2201 ms. Trace:
I20260811 18:57:39.393147 4169 ts_tablet_manager.cc:1918] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.393317 4169 log.cc:1196] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6432abd4fc5c4f9ca95971ca4795e1cf
I20260811 18:57:39.393324 4167 rpcz_store.cc:276] 0811 18:57:37.190996 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.191045 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:39.392063 (+2201018us) ts_tablet_manager.cc:1145] Deleting tablet d21973bc8e514a089c2beb24a686060f
0811 18:57:39.392076 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.392166 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2200855}
I20260811 18:57:39.393705 4169 ts_tablet_manager.cc:1939] T 6432abd4fc5c4f9ca95971ca4795e1cf P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.394024 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1133) took 2202 ms. Trace:
I20260811 18:57:39.394194 4167 rpcz_store.cc:276] 0811 18:57:37.191432 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.191478 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:39.393911 (+2202433us) ts_tablet_manager.cc:1145] Deleting tablet 2f2f5ce2b1d14a9a988cff644e196088
0811 18:57:39.393922 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.394007 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2202148}
I20260811 18:57:39.394682 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet dd80633f926942e88f54b32d9a03ea3a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.395012 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1134) took 2191 ms. Trace:
I20260811 18:57:39.395363 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9042bc231f0841a4a370be5e6251dd6e" } modify_external_catalogs: true
I20260811 18:57:39.395581 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9042bc231f0841a4a370be5e6251dd6e" } modify_external_catalogs: true
I20260811 18:57:39.395849 4167 rpcz_store.cc:276] 0811 18:57:37.204042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.204450 (+ 408us) service_pool.cc:227] Handling call
0811 18:57:39.394913 (+2190463us) ts_tablet_manager.cc:1145] Deleting tablet f2b66196716a4ba3893b6b8bfec2b8c0
0811 18:57:39.394923 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.394997 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2190268}
I20260811 18:57:39.398324 4163 tablet_bootstrap.cc:492] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.398576 4163 ts_tablet_manager.cc:1397] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20260811 18:57:39.399188 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 993) took 4317 ms. Trace:
I20260811 18:57:39.399319 4169 rpcz_store.cc:276] 0811 18:57:35.081574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.081644 (+ 70us) service_pool.cc:227] Handling call
0811 18:57:39.317401 (+4235757us) ts_tablet_manager.cc:1145] Deleting tablet 6432abd4fc5c4f9ca95971ca4795e1cf
0811 18:57:39.317411 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.383686 (+ 66275us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.393017 (+ 9331us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.399162 (+ 6145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4235494}
I20260811 18:57:39.399471 4163 raft_consensus.cc:348] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.399884 4163 raft_consensus.cc:374] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.400002 4163 raft_consensus.cc:729] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.400235 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6432abd4fc5c4f9ca95971ca4795e1cf (table default.Table_69803efc857941408d32f7da625acfb6 [id=c66eee573c5d43b39b33ff907b3136b0]) successfully deleted
I20260811 18:57:39.400323 4163 consensus_queue.cc:260] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.400909 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1135) took 2018 ms. Trace:
I20260811 18:57:39.400945 4169 tablet_replica.cc:330] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.401054 4163 ts_tablet_manager.cc:1428] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.401407 4163 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.401429 4167 rpcz_store.cc:276] 0811 18:57:37.382084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.386399 (+ 4315us) service_pool.cc:227] Handling call
0811 18:57:39.400765 (+2014366us) ts_tablet_manager.cc:1145] Deleting tablet a3973dfdaac746a6af7a402277d2711b
0811 18:57:39.400779 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.400887 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2012295}
I20260811 18:57:39.402305 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1138) took 2015 ms. Trace:
I20260811 18:57:39.402537 4169 raft_consensus.cc:2229] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.402709 4169 raft_consensus.cc:2258] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.402866 4167 rpcz_store.cc:276] 0811 18:57:37.386929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.387035 (+ 106us) service_pool.cc:227] Handling call
0811 18:57:39.402170 (+2015135us) ts_tablet_manager.cc:1145] Deleting tablet 15dd14e2b4bd4a08b5b08f532a55291b
0811 18:57:39.402185 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.402285 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2013626}
I20260811 18:57:39.403237 4163 tablet_bootstrap.cc:654] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.403265 4169 ts_tablet_manager.cc:1905] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.407151 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1153) took 1872 ms. Trace:
I20260811 18:57:39.407578 4167 rpcz_store.cc:276] 0811 18:57:37.534574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.534640 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:39.407027 (+1872387us) ts_tablet_manager.cc:1145] Deleting tablet 75ceba7bfc8240268b16c789f07813d1
0811 18:57:39.407039 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.407133 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1872120}
I20260811 18:57:39.408361 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1152) took 1873 ms. Trace:
I20260811 18:57:39.408722 4167 rpcz_store.cc:276] 0811 18:57:37.534458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.534616 (+ 158us) service_pool.cc:227] Handling call
0811 18:57:39.408255 (+1873639us) ts_tablet_manager.cc:1145] Deleting tablet 0495c5b770b34d54afb1379a6e36204c
0811 18:57:39.408267 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.408346 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1873109}
I20260811 18:57:39.409394 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1154) took 1874 ms. Trace:
I20260811 18:57:39.411907 4169 ts_tablet_manager.cc:1918] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.412057 4169 log.cc:1196] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/58c515c153df4317b3fb427983ee15e0
I20260811 18:57:39.412398 4169 ts_tablet_manager.cc:1939] T 58c515c153df4317b3fb427983ee15e0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.412434 4167 rpcz_store.cc:276] 0811 18:57:37.535057 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.535100 (+ 43us) service_pool.cc:227] Handling call
0811 18:57:39.409302 (+1874202us) ts_tablet_manager.cc:1145] Deleting tablet 5a75a480f38047d880b0c1fa51403786
0811 18:57:39.409312 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.409381 (+ 69us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1874043}
I20260811 18:57:39.413249 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 992) took 4331 ms. Trace:
I20260811 18:57:39.413360 4169 rpcz_store.cc:276] 0811 18:57:35.081452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.082144 (+ 692us) service_pool.cc:227] Handling call
0811 18:57:39.400894 (+4318750us) ts_tablet_manager.cc:1145] Deleting tablet 58c515c153df4317b3fb427983ee15e0
0811 18:57:39.400908 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.408103 (+ 7195us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.411800 (+ 3697us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.413207 (+ 1407us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1352,"tablet-delete.queue_time_us":4318564}
I20260811 18:57:39.413833 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 58c515c153df4317b3fb427983ee15e0 (table default.Table_46eb50be61c143f2930a20efd2cd56b8 [id=fde52e1d72cf4dbe801bc68588471e5f]) successfully deleted
I20260811 18:57:39.413950 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 999) took 4068 ms. Trace:
I20260811 18:57:39.414032 4169 rpcz_store.cc:276] 0811 18:57:35.345421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346431 (+ 1010us) service_pool.cc:227] Handling call
0811 18:57:39.413817 (+4067386us) ts_tablet_manager.cc:1145] Deleting tablet 36861ae786c549d687e687fe853d1801
0811 18:57:39.413832 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.413932 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4067213}
W20260811 18:57:39.414431 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 36861ae786c549d687e687fe853d1801 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36861ae786c549d687e687fe853d1801 already in progress: creating tablet
I20260811 18:57:39.414529 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1000) took 4069 ms. Trace:
I20260811 18:57:39.414605 4169 rpcz_store.cc:276] 0811 18:57:35.345533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346642 (+ 1109us) service_pool.cc:227] Handling call
0811 18:57:39.414426 (+4067784us) ts_tablet_manager.cc:1145] Deleting tablet 46505ba544104458888dbc2a37115dcc
0811 18:57:39.414437 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.414514 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4067600}
W20260811 18:57:39.414975 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 46505ba544104458888dbc2a37115dcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46505ba544104458888dbc2a37115dcc already in progress: creating tablet
I20260811 18:57:39.415006 4169 tablet_replica.cc:330] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.415218 4169 raft_consensus.cc:2229] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.415395 4169 raft_consensus.cc:2258] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.416013 4169 ts_tablet_manager.cc:1905] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.417165 5557 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.417315 5557 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.417932 5557 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.420426 3824 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.422120 4162 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.422331 4162 ts_tablet_manager.cc:1397] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.062s user 0.001s sys 0.009s
I20260811 18:57:39.424041 4162 raft_consensus.cc:348] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.424355 4162 raft_consensus.cc:374] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.424466 4162 raft_consensus.cc:729] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.424772 4162 consensus_queue.cc:260] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.425570 4162 ts_tablet_manager.cc:1428] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:39.426635 4169 ts_tablet_manager.cc:1918] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.426815 4169 log.cc:1196] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7251204325dd424192a6f58e18e71c34
I20260811 18:57:39.427145 4169 ts_tablet_manager.cc:1939] T 7251204325dd424192a6f58e18e71c34 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.429563 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 996) took 4085 ms. Trace:
I20260811 18:57:39.429766 4169 rpcz_store.cc:276] 0811 18:57:35.344396 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344459 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:39.414963 (+4070504us) ts_tablet_manager.cc:1145] Deleting tablet 7251204325dd424192a6f58e18e71c34
0811 18:57:39.414973 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.422982 (+ 8009us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.426530 (+ 3548us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.429542 (+ 3012us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4066895}
I20260811 18:57:39.430203 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7251204325dd424192a6f58e18e71c34 (table default.Table_e52a767b81cd4dd186dacdeb1d4120cb [id=9ddc018ce1524b52bcd5494bc335b42b]) successfully deleted
I20260811 18:57:39.430393 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 997) took 4085 ms. Trace:
I20260811 18:57:39.430495 4169 rpcz_store.cc:276] 0811 18:57:35.344804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344956 (+ 152us) service_pool.cc:227] Handling call
0811 18:57:39.430202 (+4085246us) ts_tablet_manager.cc:1145] Deleting tablet fcb1c9787d3f4fd181f86deca28ad160
0811 18:57:39.430214 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.430331 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4082058}
W20260811 18:57:39.430943 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fcb1c9787d3f4fd181f86deca28ad160 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb1c9787d3f4fd181f86deca28ad160 already in progress: creating tablet
I20260811 18:57:39.431015 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 998) took 4086 ms. Trace:
I20260811 18:57:39.431095 4169 rpcz_store.cc:276] 0811 18:57:35.344925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.344983 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:39.430894 (+4085911us) ts_tablet_manager.cc:1145] Deleting tablet 302cf69ea7c14fdaac72f6a9a9aff113
0811 18:57:39.430906 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.430998 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4082703}
W20260811 18:57:39.432065 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 302cf69ea7c14fdaac72f6a9a9aff113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302cf69ea7c14fdaac72f6a9a9aff113 already in progress: creating tablet
I20260811 18:57:39.433027 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1001) took 4087 ms. Trace:
I20260811 18:57:39.433235 4169 rpcz_store.cc:276] 0811 18:57:35.345639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.345694 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:39.432086 (+4086392us) ts_tablet_manager.cc:1145] Deleting tablet 18c599d945044e0088bc66c8472d2495
0811 18:57:39.432925 (+ 839us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.433011 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4083852}
W20260811 18:57:39.433657 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 18c599d945044e0088bc66c8472d2495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 18c599d945044e0088bc66c8472d2495 already in progress: creating tablet
I20260811 18:57:39.433794 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 994) took 4089 ms. Trace:
I20260811 18:57:39.433892 4169 rpcz_store.cc:276] 0811 18:57:35.344089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346039 (+ 1950us) service_pool.cc:227] Handling call
0811 18:57:39.433675 (+4087636us) ts_tablet_manager.cc:1145] Deleting tablet f1342b8cd65b4c31a47b27a79c678d2b
0811 18:57:39.433688 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.433777 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085156}
W20260811 18:57:39.434206 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f1342b8cd65b4c31a47b27a79c678d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1342b8cd65b4c31a47b27a79c678d2b already in progress: creating tablet
I20260811 18:57:39.434128 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.434429 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 995) took 4090 ms. Trace:
I20260811 18:57:39.434425 3746 raft_consensus.cc:2454] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:39.434523 4169 rpcz_store.cc:276] 0811 18:57:35.344233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.346188 (+ 1955us) service_pool.cc:227] Handling call
0811 18:57:39.434258 (+4088070us) ts_tablet_manager.cc:1145] Deleting tablet e5f47ccc193d48efa15d60a06b148e00
0811 18:57:39.434269 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.434412 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4085695}
I20260811 18:57:39.434931 3824 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.435352 5557 raft_consensus.cc:2781] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.435442 5557 raft_consensus.cc:482] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.435515 5557 raft_consensus.cc:3037] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.436877 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1155) took 1894 ms. Trace:
I20260811 18:57:39.437186 4167 rpcz_store.cc:276] 0811 18:57:37.542314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.542401 (+ 87us) service_pool.cc:227] Handling call
0811 18:57:39.413197 (+1870796us) ts_tablet_manager.cc:1145] Deleting tablet d32212c817c94fad8379462b027e7d3d
0811 18:57:39.434011 (+ 20814us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.436850 (+ 2839us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1870575}
I20260811 18:57:39.437687 4167 tablet_replica.cc:330] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.437916 4162 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.437985 4167 raft_consensus.cc:2229] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.438472 4167 raft_consensus.cc:2258] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.438382 5557 raft_consensus.cc:504] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439504 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb56d23ccabc4b02961249312257d9c4" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.439819 4162 tablet_bootstrap.cc:654] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.439950 5557 leader_election.cc:288] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.440506 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e5f47ccc193d48efa15d60a06b148e00 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5f47ccc193d48efa15d60a06b148e00 already in progress: creating tablet
I20260811 18:57:39.441092 4163 tablet_bootstrap.cc:492] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:39.441154 3824 leader_election.cc:334] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.441339 4163 ts_tablet_manager.cc:1397] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260811 18:57:39.441395 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1005) took 3999 ms. Trace:
I20260811 18:57:39.441490 4169 rpcz_store.cc:276] 0811 18:57:35.442210 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.442372 (+ 162us) service_pool.cc:227] Handling call
0811 18:57:39.441021 (+3998649us) ts_tablet_manager.cc:1145] Deleting tablet 8eaafc7677344cac8d9616996c5f416c
0811 18:57:39.441036 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.441379 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3998204}
W20260811 18:57:39.441886 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8eaafc7677344cac8d9616996c5f416c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8eaafc7677344cac8d9616996c5f416c already in progress: creating tablet
I20260811 18:57:39.441993 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1008) took 3970 ms. Trace:
I20260811 18:57:39.442072 4169 rpcz_store.cc:276] 0811 18:57:35.471565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.471645 (+ 80us) service_pool.cc:227] Handling call
0811 18:57:39.441887 (+3970242us) ts_tablet_manager.cc:1145] Deleting tablet e2fb15b72c1d49d2ac4961bc04655a29
0811 18:57:39.441902 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.441978 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969887}
W20260811 18:57:39.442476 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e2fb15b72c1d49d2ac4961bc04655a29 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2fb15b72c1d49d2ac4961bc04655a29 already in progress: creating tablet
I20260811 18:57:39.442615 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1009) took 3970 ms. Trace:
I20260811 18:57:39.442703 4169 rpcz_store.cc:276] 0811 18:57:35.472263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.472331 (+ 68us) service_pool.cc:227] Handling call
0811 18:57:39.442509 (+3970178us) ts_tablet_manager.cc:1145] Deleting tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae
0811 18:57:39.442520 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.442600 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969897}
I20260811 18:57:39.443063 4169 tablet_replica.cc:330] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.443289 4169 raft_consensus.cc:2229] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:39.443328 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e12cb4bd0b6e42f9a33cb46ed1d2c1ae already in progress: creating tablet
I20260811 18:57:39.443500 4169 raft_consensus.cc:2258] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.443924 4163 raft_consensus.cc:348] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439785 3746 raft_consensus.cc:3037] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.445123 4163 raft_consensus.cc:374] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.445192 4163 raft_consensus.cc:729] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.445382 4163 consensus_queue.cc:260] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.445986 4163 ts_tablet_manager.cc:1428] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 18:57:39.446216 4163 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.447768 4163 tablet_bootstrap.cc:654] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.448395 3746 raft_consensus.cc:2454] T bb56d23ccabc4b02961249312257d9c4 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:39.449087 3824 leader_election.cc:302] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.444056 4169 ts_tablet_manager.cc:1905] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.454528 5557 raft_consensus.cc:2781] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.454691 5557 raft_consensus.cc:686] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.454972 5557 consensus_queue.cc:237] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.468675 4163 tablet_bootstrap.cc:492] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.469329 4163 ts_tablet_manager.cc:1397] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260811 18:57:39.471329 4163 raft_consensus.cc:348] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.439337 4167 ts_tablet_manager.cc:1905] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:39.475864 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 133729us
I20260811 18:57:39.479883 4162 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.480068 4162 ts_tablet_manager.cc:1397] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.003s
I20260811 18:57:39.480839 4162 raft_consensus.cc:348] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.481088 4162 raft_consensus.cc:374] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.481169 4162 raft_consensus.cc:729] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.481393 4162 consensus_queue.cc:260] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.482060 4162 ts_tablet_manager.cc:1428] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:39.485606 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fa95131ce50943e3b0c6b5bdd5a30939
I20260811 18:57:39.488471 4162 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.490209 4162 tablet_bootstrap.cc:654] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.490598 5641 raft_consensus.cc:482] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.490731 5641 raft_consensus.cc:504] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.491329 5641 leader_election.cc:288] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.471886 4169 ts_tablet_manager.cc:1918] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260811 18:57:39.492781 4169 log.cc:1196] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b1ab2eee21264b1aa5ba00528cae3843
I20260811 18:57:39.490664 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.492394 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfea0cd619024204b9bff1e2dd62b7b0" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.493283 4169 ts_tablet_manager.cc:1939] T b1ab2eee21264b1aa5ba00528cae3843 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:39.493520 3676 leader_election.cc:341] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.495509 3678 leader_election.cc:334] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.495635 3678 leader_election.cc:302] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.496169 5641 raft_consensus.cc:2726] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.492782 4163 raft_consensus.cc:374] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.498208 4163 raft_consensus.cc:729] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.498627 4163 consensus_queue.cc:260] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.499518 4163 ts_tablet_manager.cc:1428] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.029s user 0.002s sys 0.000s
I20260811 18:57:39.502542 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f83d8784174f40e29b96eba2de396065"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.503190 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f83d8784174f40e29b96eba2de396065 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.503970 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 275ec54207504e9f8267925e2224b2ab consensus state has no leader
I20260811 18:57:39.504212 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet c944d5cfd9b3405b8151abfc9f2fd998 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.504663 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 7645a242bc4e4845ad49bfbe20b252f2 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.505167 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_23655b2975534b81b076fd031ea5022a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.505704 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23655b2975534b81b076fd031ea5022a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.510550 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_7f65859fb42941e3a8375f65182f9ec4" } new_table_name: "default.Table_710bf71d5383421ca80e2caac75ec12f" modify_external_catalogs: true
I20260811 18:57:39.512020 5557 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.512189 5557 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.512913 5557 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.513842 5557 raft_consensus.cc:482] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.513969 5557 raft_consensus.cc:504] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.514511 5557 leader_election.cc:288] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.517330 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.517642 3746 raft_consensus.cc:2454] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:39.519101 3824 leader_election.cc:341] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: a847581412494e34b2c76e605ff11772
I20260811 18:57:39.519366 3824 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:39.520619 5557 raft_consensus.cc:2781] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.520756 5557 raft_consensus.cc:482] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.520850 5557 raft_consensus.cc:3037] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:39.523432 3824 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:39.523764 3824 leader_election.cc:334] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.523891 3824 leader_election.cc:302] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:39.524264 5630 raft_consensus.cc:2726] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.526326 5557 raft_consensus.cc:504] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.528319 5557 leader_election.cc:288] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.530441 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf0594d55844ae49c99247ecd8b810f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.530668 3746 raft_consensus.cc:3037] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.534142 3746 raft_consensus.cc:2454] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:39.534994 3824 leader_election.cc:334] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.535584 3824 leader_election.cc:302] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.537052 5557 raft_consensus.cc:2781] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.537395 5557 raft_consensus.cc:686] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.537948 5557 consensus_queue.cc:237] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:39.541558 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06519254820241f1bd4bbceb3faedca3 consensus state has no leader
I20260811 18:57:39.542052 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e" } modify_external_catalogs: true
I20260811 18:57:39.542354 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_2ad9e3991d0f4274a4770a8f264f403e" } modify_external_catalogs: true
W20260811 18:57:39.545305 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9fbac57c9c44a028f5b258593aff86d has no consensus state
I20260811 18:57:39.545675 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_227406eaa3694731a78479da6280faac" } modify_external_catalogs: true
I20260811 18:57:39.545852 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_227406eaa3694731a78479da6280faac" } modify_external_catalogs: true
W20260811 18:57:39.554685 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ccceeb89da34853a768df69e1298457 consensus state has no leader
I20260811 18:57:39.555058 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e142d545d8b249f3835bce3bbef9f94e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.555719 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e142d545d8b249f3835bce3bbef9f94e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.555065 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f614f2bd34c949cd94f9493a086dd8c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.557461 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f614f2bd34c949cd94f9493a086dd8c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.551277 5689 tablet.cc:1722] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:39.539875 5687 tablet.cc:1722] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260811 18:57:39.567662 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1010) took 4094 ms. Trace:
I20260811 18:57:39.567792 4169 rpcz_store.cc:276] 0811 18:57:35.472858 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.472911 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:39.443024 (+3970113us) ts_tablet_manager.cc:1145] Deleting tablet b1ab2eee21264b1aa5ba00528cae3843
0811 18:57:39.443034 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.457463 (+ 14429us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.471775 (+ 14312us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.567634 (+ 95859us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3969890}
I20260811 18:57:39.568162 4169 tablet_replica.cc:330] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.568395 4169 raft_consensus.cc:2229] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.568625 4169 raft_consensus.cc:2258] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.569262 4169 ts_tablet_manager.cc:1905] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.569904 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet eb43127dda814a718b338906fc2a3c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.570166 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.570381 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5b7d1e32514ad48177371582e85241 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.571061 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 4451cc4d7c5940cdb41c021ae486a5a9 with delete_type TABLET_DATA_DELETED (replaced by tablet ab9fe26e8e1e4c3886af83ac3e6956f5) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.577091 4162 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.577423 4162 ts_tablet_manager.cc:1397] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.022s
I20260811 18:57:39.577870 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d8f35609e86b47aa9a8ab711834da08f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.578305 4162 raft_consensus.cc:348] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.578620 4162 raft_consensus.cc:374] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.578713 4162 raft_consensus.cc:729] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.579071 4162 consensus_queue.cc:260] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.579849 4162 ts_tablet_manager.cc:1428] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 18:57:39.580175 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8f35609e86b47aa9a8ab711834da08f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.580246 4162 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.580967 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.581470 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b1ab2eee21264b1aa5ba00528cae3843 (table default.Table_0c5780ef51fc41098318ac636d2c7c7b [id=64fc5489749347609383603760522f27]) successfully deleted
I20260811 18:57:39.583388 4169 ts_tablet_manager.cc:1918] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.583539 4169 log.cc:1196] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/45aff3e8cb0c4f7191e4522c6ef7d156
I20260811 18:57:39.583766 4162 tablet_bootstrap.cc:654] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.583928 4169 ts_tablet_manager.cc:1939] T 45aff3e8cb0c4f7191e4522c6ef7d156 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:39.584149 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc07c28ccfd48ee9c33e49442c6047b consensus state has no leader
I20260811 18:57:39.584466 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet e544b4b1592c4d51a2ed0d6461edc75e from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.584803 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1027) took 3946 ms. Trace:
I20260811 18:57:39.584908 4169 rpcz_store.cc:276] 0811 18:57:35.637902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.638816 (+ 914us) service_pool.cc:227] Handling call
0811 18:57:39.568110 (+3929294us) ts_tablet_manager.cc:1145] Deleting tablet 45aff3e8cb0c4f7191e4522c6ef7d156
0811 18:57:39.568124 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.576646 (+ 8522us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.583273 (+ 6627us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.584776 (+ 1503us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3929096}
I20260811 18:57:39.585822 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 45aff3e8cb0c4f7191e4522c6ef7d156 (table default.Table_13fbac538d16421e921aa37f54683662 [id=25166c50114d4c4abef596e7cda9de7f]) successfully deleted
I20260811 18:57:39.587891 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_e894321d446d4d6da9e866788ff2b1e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.588516 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e894321d446d4d6da9e866788ff2b1e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.590564 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_7a85117c40464174beed058911301729"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.591144 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a85117c40464174beed058911301729 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.595883 4163 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.599494 5641 raft_consensus.cc:482] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.599673 5641 raft_consensus.cc:504] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.600375 5641 leader_election.cc:288] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:39.605795 3678 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.598460 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0fde489a381549f394a78e98c8cade0f (DEFAULT_TABLE table=default.Table_8e26f75764214905965fa0f514cb0028 [id=3aa9faf1a865494e9e6b33917e0cd501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.598490 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 26f55fe6a9e840a4a5e7ba6bf727c779 (DEFAULT_TABLE table=default.Table_724b9320db2141f39b2511549a1aa8d4 [id=de7aea74dccc46f68bc416ad0f023c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.599496 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 625daea5324b4494996a51f0f9e9bcb7 (DEFAULT_TABLE table=default.Table_a6af8048b8f14af899345d8213953792 [id=677b3b20925544f28c2fb9ae18878c69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.600203 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1028) took 3962 ms. Trace:
I20260811 18:57:39.601573 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20872d80c9545e5822bfc82d496ce97" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.600586 3862 tablet_service.cc:1467] Processing CreateTablet for tablet eea325ca357146fcaa408d87803da349 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.607420 4169 rpcz_store.cc:276] 0811 18:57:35.638086 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.638140 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:39.600053 (+3961913us) ts_tablet_manager.cc:1145] Deleting tablet 46a15e7a0c0f4283ac5e575d3c15c9da
0811 18:57:39.600067 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.600183 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3930804}
W20260811 18:57:39.608361 3676 leader_election.cc:341] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.597818 4163 tablet_bootstrap.cc:654] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:39.608429 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 46a15e7a0c0f4283ac5e575d3c15c9da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46a15e7a0c0f4283ac5e575d3c15c9da already in progress: creating tablet
I20260811 18:57:39.608498 3676 leader_election.cc:302] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.609086 5630 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.609241 5630 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.610091 5630 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.613847 4162 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.613997 4162 ts_tablet_manager.cc:1397] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.001s
I20260811 18:57:39.614459 5407 raft_consensus.cc:2726] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.614734 4162 raft_consensus.cc:348] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.614943 4162 raft_consensus.cc:374] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.614985 4162 raft_consensus.cc:729] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.615283 4162 consensus_queue.cc:260] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.615935 4162 ts_tablet_manager.cc:1428] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.616335 4162 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.623620 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:39.623988 3746 raft_consensus.cc:2454] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:39.626658 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770" } modify_external_catalogs: true
I20260811 18:57:39.626909 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_eae6865a9edf4e0ebe72f448ac7ab770" } modify_external_catalogs: true
I20260811 18:57:39.628968 4162 tablet_bootstrap.cc:654] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.629668 3824 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:39.629967 3824 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.629967 5557 raft_consensus.cc:2781] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:39.630149 5557 raft_consensus.cc:482] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:39.630247 5557 raft_consensus.cc:3037] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.633314 5557 raft_consensus.cc:504] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.634049 5557 leader_election.cc:288] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:39.630259 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:39.638895 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb82243777a9494b96959ced0c24de5a" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:39.639099 4169 tablet_replica.cc:330] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.639204 3746 raft_consensus.cc:3037] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:39.639333 4169 raft_consensus.cc:2229] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.639554 4169 raft_consensus.cc:2258] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.640244 4169 ts_tablet_manager.cc:1905] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.643684 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet 8b7fc4a898cd4e2c9e27d6f5f964412a from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:39.644873 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:39.645471 3548 catalog_manager.cc:6565] ReplaceTablet: tablet e544b4b1592c4d51a2ed0d6461edc75e deleted and replaced by tablet a07f682b68114c2b9cb0df27a94406ec
I20260811 18:57:39.646101 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:39.646407 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:39.646620 3549 catalog_manager.cc:5495] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.650768 4167 ts_tablet_manager.cc:1918] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.650926 4167 log.cc:1196] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e18242f762d8407db0e2f8716890feb4
I20260811 18:57:39.651362 4167 ts_tablet_manager.cc:1939] T e18242f762d8407db0e2f8716890feb4 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.652309 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1156) took 2051 ms. Trace:
I20260811 18:57:39.652427 4167 rpcz_store.cc:276] 0811 18:57:37.601131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.602071 (+ 940us) service_pool.cc:227] Handling call
0811 18:57:39.437609 (+1835538us) ts_tablet_manager.cc:1145] Deleting tablet e18242f762d8407db0e2f8716890feb4
0811 18:57:39.437622 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.579532 (+141910us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.650641 (+ 71109us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.652278 (+ 1637us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1835002}
I20260811 18:57:39.652935 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1168) took 1821 ms. Trace:
I20260811 18:57:39.653024 4167 rpcz_store.cc:276] 0811 18:57:37.831174 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.831269 (+ 95us) service_pool.cc:227] Handling call
0811 18:57:39.652778 (+1821509us) ts_tablet_manager.cc:1145] Deleting tablet 3195082643264e0d90f70e1007ab9b24
0811 18:57:39.652792 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.652916 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1821255}
I20260811 18:57:39.654042 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f9cef27540ff46aebc9b6f4760ae0db4
I20260811 18:57:39.653950 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_4b90b4ead71940e58e93af4a9804a7d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.654624 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b90b4ead71940e58e93af4a9804a7d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.655047 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bc38e6760754449fbd6209d765cce70c
I20260811 18:57:39.655668 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ab03996c5c624e5eb17267b124e27f45
I20260811 18:57:39.655922 3555 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 602fd9a92962483ebc84ca2bcdb2bebc
I20260811 18:57:39.658238 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fde489a381549f394a78e98c8cade0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.658974 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f55fe6a9e840a4a5e7ba6bf727c779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659518 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 625daea5324b4494996a51f0f9e9bcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659967 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea325ca357146fcaa408d87803da349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.659955 3746 raft_consensus.cc:2454] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:39.668270 4162 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.668627 4162 ts_tablet_manager.cc:1397] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.005s
I20260811 18:57:39.669509 4162 raft_consensus.cc:348] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.669878 4162 raft_consensus.cc:374] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.669992 4162 raft_consensus.cc:729] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.670511 4162 consensus_queue.cc:260] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.671311 4162 ts_tablet_manager.cc:1428] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:39.671655 4162 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.673393 4162 tablet_bootstrap.cc:654] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.675597 5630 raft_consensus.cc:482] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.675773 5630 raft_consensus.cc:504] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.676381 5630 leader_election.cc:288] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.676987 3824 leader_election.cc:302] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:39.677335 5630 raft_consensus.cc:2781] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:39.677465 5630 raft_consensus.cc:686] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:39.677752 5630 consensus_queue.cc:237] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.674301 5641 raft_consensus.cc:482] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.679077 5641 raft_consensus.cc:504] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.679734 5641 leader_election.cc:288] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.680632 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e24f8d4522466d81605ea37fa7064a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:39.681286 3676 leader_election.cc:341] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:39.683305 3678 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.683418 3678 leader_election.cc:302] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.683853 5641 raft_consensus.cc:2726] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.666280 4169 ts_tablet_manager.cc:1918] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:39.687491 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 929586965ce6468c82950102cfed41da (DEFAULT_TABLE table=default.Table_bce1d6623de1467198e1dfaf763f08cf [id=a85df4c40f944b598dcd718a4e9a3a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.687911 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 6688ed13d1334ed98503b21a937facd2 (DEFAULT_TABLE table=default.Table_644eac4afde147dea1b6dc8cd69dc719 [id=583ba00a08184bc38ff31f87596e3788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.688292 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929586965ce6468c82950102cfed41da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.688611 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688ed13d1334ed98503b21a937facd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.687691 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 441feddae43741678ef0f64888b5c6f3 (DEFAULT_TABLE table=default.Table_b081a3f48c9048ecb88f1bd4571f934b [id=54b7f379f89e4c009585c94a5f7e1a55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.689193 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 441feddae43741678ef0f64888b5c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.689131 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" } modify_external_catalogs: true
I20260811 18:57:39.689355 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f9092758427e45b8a6d0472de8178ea7" } modify_external_catalogs: true
I20260811 18:57:39.692121 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet c590c98e39f84ea09b26bbe8640ca93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.687454 4169 log.cc:1196] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ef47211aa8864d13a55551da47804c19
I20260811 18:57:39.686990 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 (DEFAULT_TABLE table=default.Table_9042bc231f0841a4a370be5e6251dd6e [id=044362a3a169467396a6be8959d1d07b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.687153 3853 tablet_service.cc:1514] Processing DeleteTablet for tablet ab03996c5c624e5eb17267b124e27f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.694413 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet cb54beb34a654fbdb9c7632901bb0b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.687045 3862 tablet_service.cc:1514] Processing DeleteTablet for tablet bc38e6760754449fbd6209d765cce70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.687212 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e with delete_type TABLET_DATA_DELETED (replaced by tablet a07f682b68114c2b9cb0df27a94406ec) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.687336 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.697485 4169 ts_tablet_manager.cc:1939] T ef47211aa8864d13a55551da47804c19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:39.697995 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1030) took 3967 ms. Trace:
I20260811 18:57:39.698141 4169 rpcz_store.cc:276] 0811 18:57:35.730672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.735506 (+ 4834us) service_pool.cc:227] Handling call
0811 18:57:39.608414 (+3872908us) ts_tablet_manager.cc:1145] Deleting tablet ef47211aa8864d13a55551da47804c19
0811 18:57:39.639030 (+ 30616us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.662936 (+ 23906us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.666176 (+ 3240us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.697975 (+ 31799us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3872435}
I20260811 18:57:39.694052 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cef27540ff46aebc9b6f4760ae0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.702169 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:39.702445 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
I20260811 18:57:39.702607 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:39.702749 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet e1af33fd98c6486f975368ec90493db7 (table default.Table_22bd9bbe66b64e209048c8164f116ba7 [id=7736472828a3499c8ad8026e164b9759]). Expected version 3 got 0
I20260811 18:57:39.702905 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_075181f96b7b42059289018e831e2671 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 2 got 0
I20260811 18:57:39.703045 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (table default.Table_9194817120074797a6f60bf8e24c0636 [id=1323bd99594a4382ba121cc11e79de31]). Expected version 1 got 0
I20260811 18:57:39.703183 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:39.707228 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1031) took 3969 ms. Trace:
I20260811 18:57:39.707374 4169 rpcz_store.cc:276] 0811 18:57:35.738991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.739315 (+ 324us) service_pool.cc:227] Handling call
0811 18:57:39.707060 (+3967745us) ts_tablet_manager.cc:1145] Deleting tablet 22e1a33348e34e35bab786f8966c408b
0811 18:57:39.707076 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.707195 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3967424}
I20260811 18:57:39.708340 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1032) took 3908 ms. Trace:
I20260811 18:57:39.708432 4169 rpcz_store.cc:276] 0811 18:57:35.799599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.799673 (+ 74us) service_pool.cc:227] Handling call
0811 18:57:39.708206 (+3908533us) ts_tablet_manager.cc:1145] Deleting tablet 6f804703e20c4214b8b2a31efeaef59e
0811 18:57:39.708220 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.708320 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3907738}
I20260811 18:57:39.709143 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1033) took 3909 ms. Trace:
I20260811 18:57:39.709236 4169 rpcz_store.cc:276] 0811 18:57:35.799738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.799815 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:39.709026 (+3909211us) ts_tablet_manager.cc:1145] Deleting tablet 9b931eca7ba04c11af6571cb418f882e
0811 18:57:39.709038 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.709125 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3909008}
I20260811 18:57:39.709507 4169 tablet_replica.cc:330] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:39.709971 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet f389d2b7ff354ac4bab5d68b9ff201c6
I20260811 18:57:39.713902 4163 tablet_bootstrap.cc:492] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.714143 4163 ts_tablet_manager.cc:1397] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.118s user 0.013s sys 0.004s
I20260811 18:57:39.715138 4163 raft_consensus.cc:348] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.715534 4163 raft_consensus.cc:374] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.715647 4163 raft_consensus.cc:729] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.715973 4163 consensus_queue.cc:260] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.716773 4163 ts_tablet_manager.cc:1428] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.720862 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c499c7d38b65414692ba6f4d7307a7bf" } new_table_name: "default.Table_4d54139265ad4964846ab0b2a1331b90" modify_external_catalogs: true
I20260811 18:57:39.721808 4163 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.723747 4163 tablet_bootstrap.cc:654] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.727411 4162 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.727674 4162 ts_tablet_manager.cc:1397] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.005s
I20260811 18:57:39.728547 4162 raft_consensus.cc:348] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.728852 4162 raft_consensus.cc:374] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.728955 4162 raft_consensus.cc:729] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.729271 4162 consensus_queue.cc:260] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.730012 4162 ts_tablet_manager.cc:1428] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:39.730377 4162 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:39.730415 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a49eeecb6ae34fe28d40847ab1738944"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.731066 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a49eeecb6ae34fe28d40847ab1738944 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.732353 4162 tablet_bootstrap.cc:654] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.734509 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1170) took 1758 ms. Trace:
I20260811 18:57:39.736496 3549 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 330801031cc74eb2a5bc2caf53a18ead (table default.Table_04ac8209b46549e3bb3f12971ab01b73 [id=54b88a6b23fe45e8b0f235baf8f7234f]). Expected version 1 got 0
I20260811 18:57:39.737267 4167 rpcz_store.cc:276] 0811 18:57:37.976345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.976640 (+ 295us) service_pool.cc:227] Handling call
0811 18:57:39.734295 (+1757655us) ts_tablet_manager.cc:1145] Deleting tablet 429eb5824f3f4b2b8da6e17ba7c69b92
0811 18:57:39.734312 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.734478 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1757421}
I20260811 18:57:39.738471 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1171) took 1762 ms. Trace:
I20260811 18:57:39.744102 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_03b64ac0b80f46288fcf2b34397efcaa" } new_table_name: "default.Table_4a427b04935c4d62b0595df0085e3a98" modify_external_catalogs: true
I20260811 18:57:39.744436 4167 rpcz_store.cc:276] 0811 18:57:37.976486 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.976907 (+ 421us) service_pool.cc:227] Handling call
0811 18:57:39.738279 (+1761372us) ts_tablet_manager.cc:1145] Deleting tablet f89eebbc40234794a081734f834294ab
0811 18:57:39.738296 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.738445 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1761212}
I20260811 18:57:39.744709 4163 tablet_bootstrap.cc:492] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.744859 4163 ts_tablet_manager.cc:1397] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260811 18:57:39.745118 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1172) took 1738 ms. Trace:
I20260811 18:57:39.745235 4167 rpcz_store.cc:276] 0811 18:57:38.007054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.007116 (+ 62us) service_pool.cc:227] Handling call
0811 18:57:39.744937 (+1737821us) ts_tablet_manager.cc:1145] Deleting tablet a5bbd3274980470b9fe71457b49bdd62
0811 18:57:39.744951 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.745075 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1737579}
I20260811 18:57:39.745720 4163 raft_consensus.cc:348] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.746089 4163 raft_consensus.cc:374] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.746205 4163 raft_consensus.cc:729] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.746502 4163 consensus_queue.cc:260] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.747195 4163 ts_tablet_manager.cc:1428] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:39.747530 4163 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:39.749835 4163 tablet_bootstrap.cc:654] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.753669 3554 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_faccbd3f194043708dbbd12a50f037cf" } new_table_name: "default.Table_4400f63ffc1944a1bd8a37ed1a0ed505" modify_external_catalogs: true
I20260811 18:57:39.745994 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1169) took 1769 ms. Trace:
I20260811 18:57:39.754530 4167 rpcz_store.cc:276] 0811 18:57:37.976159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.010438 (+ 34279us) service_pool.cc:227] Handling call
0811 18:57:39.745849 (+1735411us) ts_tablet_manager.cc:1145] Deleting tablet b7b38d3b9570467a9128bd7fe73acf2f
0811 18:57:39.745864 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.745953 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1734821}
I20260811 18:57:39.755209 4167 tablet_replica.cc:330] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.755512 4167 raft_consensus.cc:2229] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.755750 4167 raft_consensus.cc:2258] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.756422 4167 ts_tablet_manager.cc:1905] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.759436 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_2e44f4b4162648d6911b8a00b8fd7ba6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.760087 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e44f4b4162648d6911b8a00b8fd7ba6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.752969 5593 raft_consensus.cc:482] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.761679 5593 raft_consensus.cc:504] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.762133 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_30ac5178633e44aeafd707e48d7c4cba" } new_table_name: "default.Table_f94d755a69ab49dcbbc91974db72cb77" modify_external_catalogs: true
I20260811 18:57:39.766465 5593 leader_election.cc:288] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.766554 3852 tablet_service.cc:1467] Processing CreateTablet for tablet ab9fe26e8e1e4c3886af83ac3e6956f5 (DEFAULT_TABLE table=default.Table_7e6c466c96e8465c9677dcaddee8819a [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.767084 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9fe26e8e1e4c3886af83ac3e6956f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.780583 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_1612fbcb897947ee843430ed61c2e50e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.781488 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1612fbcb897947ee843430ed61c2e50e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.782327 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:39.782708 3552 catalog_manager.cc:5495] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.783143 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 1 got 0
I20260811 18:57:39.786767 5641 raft_consensus.cc:482] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.786945 5641 raft_consensus.cc:504] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.787601 5641 leader_election.cc:288] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.787598 3552 catalog_manager.cc:5495] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.788034 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8b0f40c58a1c4f028c865253aba93296" } new_table_name: "default.Table_573bd28ff5814150abf3d238663273d7" modify_external_catalogs: true
I20260811 18:57:39.788808 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ec4bba841dd446aa3db751358ce7f32" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.788678 3552 catalog_manager.cc:5495] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:39.789075 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:39.789347 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
W20260811 18:57:39.789495 3676 leader_election.cc:341] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.789537 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:39.789703 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 3c96a264e017471ea114ca0c6a9500ef (table default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d]). Expected version 1 got 0
W20260811 18:57:39.790975 3678 leader_election.cc:334] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.791090 3678 leader_election.cc:302] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.791577 4163 tablet_bootstrap.cc:492] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:39.791733 4163 ts_tablet_manager.cc:1397] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20260811 18:57:39.791985 5641 raft_consensus.cc:2726] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.793007 4163 raft_consensus.cc:348] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.793342 4163 raft_consensus.cc:374] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.793468 4163 raft_consensus.cc:729] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.793833 4163 consensus_queue.cc:260] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.794184 4169 raft_consensus.cc:2229] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:39.794832 5641 raft_consensus.cc:482] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.794957 5641 raft_consensus.cc:504] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.795503 5641 leader_election.cc:288] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.796854 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 5d1b94cb25b84ae1a4c22801b1a98094 (DEFAULT_TABLE table=default.Table_04519ba2021244b6922bea0e5b79ecb6 [id=02744826873c4079a8d983c8e0d71c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.794656 4163 ts_tablet_manager.cc:1428] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:39.797297 4169 pending_rounds.cc:62] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Trying to abort 1 pending ops.
I20260811 18:57:39.796103 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e14f8717bcd94d6d88295fd8be130ca5" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:39.797358 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d1b94cb25b84ae1a4c22801b1a98094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.797832 4163 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:39.797828 3676 leader_election.cc:341] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.800180 3859 tablet_service.cc:1467] Processing CreateTablet for tablet b9fbac57c9c44a028f5b258593aff86d (DEFAULT_TABLE table=default.Table_51582df654e7406dbc574ba8de215baf [id=0eb328384d824024bcf7782bc4cf5b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.800694 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9fbac57c9c44a028f5b258593aff86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.800999 4163 tablet_bootstrap.cc:654] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:39.802484 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:39.802906 3871 tablet_service.cc:1467] Processing CreateTablet for tablet b3ef682132a04b338b03a4387999d6b9 (DEFAULT_TABLE table=default.Table_4fd6bf06dd964e719ce1470aec9f60bc [id=54c1a4605328449dbb0a4928e192e271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.803292 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ef682132a04b338b03a4387999d6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.803777 4167 ts_tablet_manager.cc:1918] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.803901 4167 log.cc:1196] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/42405aec21e3405486b6c658438b5fb2
I20260811 18:57:39.804200 4167 ts_tablet_manager.cc:1939] T 42405aec21e3405486b6c658438b5fb2 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.804328 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_343b8634c5a946feacafd49b9aba3c62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.805027 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1174) took 1716 ms. Trace:
I20260811 18:57:39.805120 4167 rpcz_store.cc:276] 0811 18:57:38.088470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.088610 (+ 140us) service_pool.cc:227] Handling call
0811 18:57:39.755112 (+1666502us) ts_tablet_manager.cc:1145] Deleting tablet 42405aec21e3405486b6c658438b5fb2
0811 18:57:39.755126 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.800687 (+ 45561us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.803646 (+ 2959us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.804998 (+ 1352us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1666255}
W20260811 18:57:39.808506 3678 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.808669 3678 leader_election.cc:302] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.808962 5641 raft_consensus.cc:2726] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.810125 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ef47211aa8864d13a55551da47804c19 (table default.Table_74e0e65fa61543fc939b36aee569344d [id=59e59153732d407d94b287a5588e974c]) successfully deleted
W20260811 18:57:39.810689 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 22e1a33348e34e35bab786f8966c408b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22e1a33348e34e35bab786f8966c408b already in progress: creating tablet
W20260811 18:57:39.810920 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6f804703e20c4214b8b2a31efeaef59e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f804703e20c4214b8b2a31efeaef59e already in progress: creating tablet
W20260811 18:57:39.811115 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 9b931eca7ba04c11af6571cb418f882e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b931eca7ba04c11af6571cb418f882e already in progress: creating tablet
W20260811 18:57:39.813042 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_343b8634c5a946feacafd49b9aba3c62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.815361 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06c8776bb47c4cb891ef13a2a1e36781 consensus state has no leader
I20260811 18:57:39.815922 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93d80fcaa1a646119a16dcb3e546b426"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.816550 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93d80fcaa1a646119a16dcb3e546b426 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.816715 3554 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75daed89122c406caa4bd9bbaa67d9b8 consensus state has no leader
W20260811 18:57:39.817889 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b609177908d04a829b1fe744ef0f53e1 consensus state has no leader
W20260811 18:57:39.826543 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b540ae350254346a675b54118c5cfcb consensus state has no leader
W20260811 18:57:39.826543 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:39.832258 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_0b6fac836f9146ab9a5700100e2bda0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.832957 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b6fac836f9146ab9a5700100e2bda0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.797354 4169 pending_rounds.cc:68] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317400190985445376 op_type: NO_OP noop_request { }
I20260811 18:57:39.843808 4169 raft_consensus.cc:2866] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260811 18:57:39.844903 4169 raft_consensus.cc:2258] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.846997 4169 ts_tablet_manager.cc:1905] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.851079 5641 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.851255 5641 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.851864 5641 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:39.854700 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 407b07d906d4447dbae5fd45d44e3ef8 (DEFAULT_TABLE table=default.Table_24401a03e8d54050a74da4ed3bbb60fd [id=936d78e258b341f98d02313652a325f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.855178 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 407b07d906d4447dbae5fd45d44e3ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.858621 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 812cc50dcac740c5ab7633abc04c1d9e (DEFAULT_TABLE table=default.Table_3020dd34e96c426fbbca98e320405d0f [id=7f1262b7d1754c54aa332bc31a4db6c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.859042 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812cc50dcac740c5ab7633abc04c1d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.863246 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 81f52a745f6243ae9b1ef2dd33b06d3a (DEFAULT_TABLE table=default.Table_fe6c2229e23048eb81c61314102eab38 [id=57de9419daff451a9ca6485e9230e621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.863695 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f52a745f6243ae9b1ef2dd33b06d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.868870 4167 tablet_replica.cc:330] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.869212 4167 raft_consensus.cc:2229] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.869386 4167 raft_consensus.cc:2258] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.869947 4167 ts_tablet_manager.cc:1905] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.858201 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 8072b2d13cbc45249f43411e9022d006 (DEFAULT_TABLE table=default.Table_46d76d83b3174f5da337585b00af5f68 [id=40394961c1b348b4b67b3322da0eb2c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.871235 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8072b2d13cbc45249f43411e9022d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.875533 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51582df654e7406dbc574ba8de215baf" } modify_external_catalogs: true
I20260811 18:57:39.875599 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f7751679e1394c27bdfb569a73a14165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:39.875828 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_51582df654e7406dbc574ba8de215baf" } modify_external_catalogs: true
W20260811 18:57:39.876264 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7751679e1394c27bdfb569a73a14165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.877401 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" } modify_external_catalogs: true
I20260811 18:57:39.877583 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1623cbb9139845d2a67dab2017c8e7ff" } modify_external_catalogs: true
I20260811 18:57:39.879017 3551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6533d74b53c54546bdd082863b38257c" } modify_external_catalogs: true
I20260811 18:57:39.879172 3551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6533d74b53c54546bdd082863b38257c" } modify_external_catalogs: true
I20260811 18:57:39.880829 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_194b336657844c34ac93e94c90db9444"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.881424 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_194b336657844c34ac93e94c90db9444 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:39.885403 3678 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.899451 3551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:39.901794 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7a67e56b3f641139862409e95439c6a" } modify_external_catalogs: true
I20260811 18:57:39.907416 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_b7a67e56b3f641139862409e95439c6a" } modify_external_catalogs: true
I20260811 18:57:39.898618 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b9fbac57c9c44a028f5b258593aff86d
I20260811 18:57:39.913606 4167 ts_tablet_manager.cc:1918] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:39.915272 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_fb4433f8104245af96d1746998f92ba1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.916165 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb4433f8104245af96d1746998f92ba1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.916716 5698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:39.916978 5698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:39.917851 5698 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:39.919184 4167 log.cc:1196] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/eb8f5b87062349c4b19ec99fa0175f24
I20260811 18:57:39.919462 5266 ts_manager.cc:194] Registered new tserver with Master: bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.919577 4167 ts_tablet_manager.cc:1939] T eb8f5b87062349c4b19ec99fa0175f24 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:39.919571 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1531 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:39.920536 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1177) took 1714 ms. Trace:
I20260811 18:57:39.920644 4167 rpcz_store.cc:276] 0811 18:57:38.205694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.206098 (+ 404us) service_pool.cc:227] Handling call
0811 18:57:39.868802 (+1662704us) ts_tablet_manager.cc:1145] Deleting tablet eb8f5b87062349c4b19ec99fa0175f24
0811 18:57:39.868818 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.873068 (+ 4250us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.913481 (+ 40413us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:39.920511 (+ 7030us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1652400}
I20260811 18:57:39.921202 4167 tablet_replica.cc:330] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:39.921411 4167 raft_consensus.cc:2229] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:39.923799 4167 raft_consensus.cc:2258] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:39.924372 4167 ts_tablet_manager.cc:1905] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:39.927892 3548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a" } modify_external_catalogs: true
I20260811 18:57:39.928139 3548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8dbfb0ccfdef4c018c7eca1d0452375a" } modify_external_catalogs: true
I20260811 18:57:39.928849 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b5f718e0dd6347099abbd1de1d43ca73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:39.929493 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5f718e0dd6347099abbd1de1d43ca73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:39.852658 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:39.859241 3824 leader_election.cc:334] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.922151 5593 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:39.939088 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 79868.4us
I20260811 18:57:39.939165 5593 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.939956 5593 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:39.940325 5593 raft_consensus.cc:482] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.940457 5593 raft_consensus.cc:504] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.941026 5593 leader_election.cc:288] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
W20260811 18:57:39.944519 3676 leader_election.cc:341] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:39.944700 3676 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:39.945070 5641 raft_consensus.cc:2726] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:39.946148 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1531 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:39.947057 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:39.953877 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 258d08de21434edc86f0cab49f839294 (DEFAULT_TABLE table=default.Table_2dffa8e09c6948c092cfdd4d888e5f81 [id=58c3e7941755444c978bd70d45db09f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.954427 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d08de21434edc86f0cab49f839294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.967521 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d65cb8d901c04d8ba2b7a40efc7fa3af
I20260811 18:57:39.970872 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 09366eccf38a4876840cc5ddddbbb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:39.972577 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 94857ef729eb435c97d089b8ab108cea (DEFAULT_TABLE table=default.Table_0155ea394d7d4e6a9fb44e6f54cfce5e [id=65437486622c48dab4d1a7bf265849ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.973147 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94857ef729eb435c97d089b8ab108cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.977262 4162 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:39.977437 4162 ts_tablet_manager.cc:1397] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.247s user 0.019s sys 0.003s
I20260811 18:57:39.978169 4162 raft_consensus.cc:348] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.978475 4162 raft_consensus.cc:374] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:39.978549 4162 raft_consensus.cc:729] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:39.978873 4162 consensus_queue.cc:260] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:39.987686 4162 ts_tablet_manager.cc:1428] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260811 18:57:39.988047 4162 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:39.990145 3540 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 25266bc7788f49f8af8eb9a948e61c8d (table default.Table_f94d755a69ab49dcbbc91974db72cb77 [id=31b3d877b5da424eb8f4b3fd7086cda7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:39.991844 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 9c048122b5434ec594fbf5b2665edfab (DEFAULT_TABLE table=default.Table_0765ca548a9649e88a2ad253c392107b [id=46788ef8b34947bbbffd5f651a41a102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:39.994441 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c048122b5434ec594fbf5b2665edfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.996125 5641 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:39.996279 5641 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:39.996932 5641 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:39.999397 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:39.997678 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.004184 3887 raft_consensus.cc:2454] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:40.003396 4162 tablet_bootstrap.cc:654] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.004716 3676 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.007113 5641 raft_consensus.cc:2781] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.007191 5641 raft_consensus.cc:482] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.007253 5641 raft_consensus.cc:3037] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.010016 5641 raft_consensus.cc:504] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.012092 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.012348 3887 raft_consensus.cc:3037] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.014146 5641 leader_election.cc:288] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.015152 3547 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:40.015405 3678 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.017304 3887 raft_consensus.cc:2454] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.017828 3676 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.018114 5641 raft_consensus.cc:2781] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.018411 5641 raft_consensus.cc:686] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.018721 5641 consensus_queue.cc:237] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.018882 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_db613e17da86441bb1ab37b46e730db9" } new_table_name: "default.Table_c8cfed8b43a34205a0b491d9374ba760" modify_external_catalogs: true
I20260811 18:57:40.020020 4163 tablet_bootstrap.cc:492] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.020159 4163 ts_tablet_manager.cc:1397] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.222s user 0.025s sys 0.000s
I20260811 18:57:40.020437 3546 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_deb475a5050c4f3dbaece0eb1e4ac618" } new_table_name: "default.Table_8be7a98fc3664e94b5af6924771fdd01" modify_external_catalogs: true
I20260811 18:57:40.020865 4163 raft_consensus.cc:348] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.021070 4163 raft_consensus.cc:374] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.021123 4163 raft_consensus.cc:729] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.021368 4163 consensus_queue.cc:260] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.021893 4163 ts_tablet_manager.cc:1428] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:40.040330 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_3309d84857df4970a9891fd64c181bf5" } new_table_name: "default.Table_298b25b6ed544a9fb0118e1af056b525" modify_external_catalogs: true
I20260811 18:57:40.041119 4167 ts_tablet_manager.cc:1918] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.041289 4167 log.cc:1196] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:40.041178 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f404fe05a8674605bb8038d676443046 (DEFAULT_TABLE table=default.Table_7cd9fda507254d029c62090b18422521 [id=50dde5de39e14c068d494813f4e68a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.041622 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f404fe05a8674605bb8038d676443046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.040344 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_93b2871d4c454120a6b609ae91e096e6" } new_table_name: "default.Table_cf30a335d3ce4b438ae4b12e64576085" modify_external_catalogs: true
I20260811 18:57:40.045076 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_69f270a4181547b2a2cc604a814c82c7" } new_table_name: "default.Table_a2a931a8e22943d7906481658e211322" modify_external_catalogs: true
I20260811 18:57:40.049540 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c0ad178862a14f968c9a334a1570eb82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.050242 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0ad178862a14f968c9a334a1570eb82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.050625 5407 raft_consensus.cc:482] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.050768 5407 raft_consensus.cc:504] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.051409 5407 leader_election.cc:288] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.054109 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3a9fdc3d594c1a84a5d8efa8490dfa" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.053540 4162 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.054590 4162 ts_tablet_manager.cc:1397] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.000s
W20260811 18:57:40.054749 3676 leader_election.cc:341] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:40.055186 3678 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.055315 3678 leader_election.cc:302] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.055809 5641 raft_consensus.cc:2726] T fb3a9fdc3d594c1a84a5d8efa8490dfa P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.041631 4167 ts_tablet_manager.cc:1939] T f75dc515d7f3473b8e43f20255a70e3e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.059087 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 408c18a220d94809a5c43e1bfca310c8 (DEFAULT_TABLE table=default.Table_325467f49f2648b5a83d5fc618fb302d [id=4dcf548dd1e84122a82fa8971e993312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.059489 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408c18a220d94809a5c43e1bfca310c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.059932 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1178) took 1854 ms. Trace:
I20260811 18:57:40.060022 4167 rpcz_store.cc:276] 0811 18:57:38.205784 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.206247 (+ 463us) service_pool.cc:227] Handling call
0811 18:57:39.921151 (+1714904us) ts_tablet_manager.cc:1145] Deleting tablet f75dc515d7f3473b8e43f20255a70e3e
0811 18:57:39.921164 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.971565 (+ 50401us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.040992 (+ 69427us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.059909 (+ 18917us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1704664}
I20260811 18:57:40.060343 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 60f82aa67c614c6abcac348b67255904 (DEFAULT_TABLE table=default.Table_21556170a88148319a2e03775ac1e02f [id=17a2f9ac3f074964b8e64a9567261924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.060696 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f82aa67c614c6abcac348b67255904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.061349 4162 raft_consensus.cc:348] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.061558 4162 raft_consensus.cc:374] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.061611 4162 raft_consensus.cc:729] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.061825 4162 consensus_queue.cc:260] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.062698 3705 tablet_service.cc:1467] Processing CreateTablet for tablet f87a8741e8ec481ea21284abe2d6c479 (DEFAULT_TABLE table=default.Table_788572afacab482590222fc4d084ef0f [id=77992422c86e47c79d4e1aea0a66b089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.063081 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f87a8741e8ec481ea21284abe2d6c479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.066644 3711 tablet_service.cc:1467] Processing CreateTablet for tablet d3e28a19c145403485376b057bad4c9e (DEFAULT_TABLE table=default.Table_51094e8e45fe427789a0f442148332f0 [id=7aa13fc26748491db551f61611f7c62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.066788 4167 tablet_replica.cc:330] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.066998 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e28a19c145403485376b057bad4c9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:40.068375 3546 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7ce2b43b2134defb14d45112e4f5e5e consensus state has no leader
W20260811 18:57:40.070982 3553 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de6245131a3a4513b88611fd4c2e031c consensus state has no leader
I20260811 18:57:40.071346 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_40846b4d992644f9ad4942d6e529abb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.071959 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40846b4d992644f9ad4942d6e529abb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.075611 4162 ts_tablet_manager.cc:1428] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260811 18:57:40.075917 4162 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.076457 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet 78868ff1f8024163ae1d782151100cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.066998 4167 raft_consensus.cc:2229] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.077049 4167 raft_consensus.cc:2258] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.077598 4167 ts_tablet_manager.cc:1905] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.078061 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccd56a8b7cc454ea3f2f08b59374521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.078545 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 33a79243bd874861b16f43339adf777c (DEFAULT_TABLE table=default.Table_80140471fa194b22bf8347836f95fdca [id=6de24db40cfe4fb49177dedc3d64443b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.079234 3711 tablet_service.cc:1467] Processing CreateTablet for tablet e4441c724b394e8092fb19c0d2427233 (DEFAULT_TABLE table=default.Table_4b41b319741e458abbf697fc780af7da [id=5c743d88bbca49e6b4dbc23c91040c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.081084 4163 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:40.082360 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 142404us
W20260811 18:57:40.083434 3824 leader_election.cc:341] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: cf6c8d542ae2489ba466ffadd518db12
W20260811 18:57:40.083798 3824 leader_election.cc:341] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0c6e0d1eeb63493bbea4cd78dd085a3c
W20260811 18:57:40.084594 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c80955bd879c4c1eb16b07ad38096259 consensus state has no leader
I20260811 18:57:40.085007 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8cc94cebd1834be58b21d1512bb24571"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.085628 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cc94cebd1834be58b21d1512bb24571 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.086040 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1922c7317c24aa984945aaaa6ce8003 consensus state has no leader
I20260811 18:57:40.086416 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_193dfc5ac3914a1f90e082521d0f84d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.086602 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4441c724b394e8092fb19c0d2427233. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:40.086917 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_193dfc5ac3914a1f90e082521d0f84d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.087446 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 112de1a11805406b9183100c6c66f720 consensus state has no leader
I20260811 18:57:40.087765 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5a407d0c02054f5ca4370e8e1a1c4c71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.088552 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a407d0c02054f5ca4370e8e1a1c4c71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.090469 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a79243bd874861b16f43339adf777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.091377 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a98bd35bd444195fd32fe6c67a04f" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:40.094704 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 1e51bb756dff43caace2c0626aed1443 (DEFAULT_TABLE table=default.Table_1afb343a7aa84568ad8ad65cab128123 [id=1fa01fbb7a4946da82085dab1e0a71ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.095726 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e51bb756dff43caace2c0626aed1443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.096283 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1fc828f73642c88171d6ac827de90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.096736 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b9fbac57c9c44a028f5b258593aff86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097119 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f97f37fa7c483cacaeed54f93e7a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:32 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097491 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet a847581412494e34b2c76e605ff11772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.097890 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6e0d1eeb63493bbea4cd78dd085a3c with delete_type TABLET_DATA_DELETED (replaced by tablet b203df4124364737839bf4c1ae072988) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.098551 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4de110bd3348f0b0dae21c9f2b6a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.099089 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c365f79d4a4201ac323f16bddf3e62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4093564a88264ed59c945523c9ddc9b3) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.099045 3554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_da206548a6a34eb78f9b4bddfc58d921"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.099651 3554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da206548a6a34eb78f9b4bddfc58d921 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.103118 4162 tablet_bootstrap.cc:654] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.104702 4169 ts_tablet_manager.cc:1918] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:40.104900 4169 log.cc:1196] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b57ed7814a1e440cba362613cbab32f8
I20260811 18:57:40.105472 4169 ts_tablet_manager.cc:1939] T b57ed7814a1e440cba362613cbab32f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:40.105896 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1034) took 4305 ms. Trace:
I20260811 18:57:40.108448 5641 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.108603 5641 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.109584 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8a306b155dd94cfaa6649e14665e9418"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.109897 5698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:57:40.110249 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a306b155dd94cfaa6649e14665e9418 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.110776 5641 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.112550 3678 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.121006 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" } modify_external_catalogs: true
I20260811 18:57:40.121230 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_ca15fa5e7ecb47239c86047e1c1ec789" } modify_external_catalogs: true
I20260811 18:57:40.123054 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_93e4cfd44cab49469aa14294ac0ba348"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.123608 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93e4cfd44cab49469aa14294ac0ba348 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.125418 3554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" } modify_external_catalogs: true
I20260811 18:57:40.125892 3554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6743ac70503f4bbc8811494daa73361f" } modify_external_catalogs: true
I20260811 18:57:40.106843 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 27cda53b7ca54346a7a22bf26f5db990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.107146 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.111033 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.125449 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 452ee450778e4d33adfa7e2745b5d970 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:40.130650 3676 leader_election.cc:341] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.130784 3676 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.131253 5641 raft_consensus.cc:2726] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.126508 3824 leader_election.cc:334] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.131595 3824 leader_election.cc:302] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.129730 4163 tablet_bootstrap.cc:654] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.132722 5701 raft_consensus.cc:2726] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.133397 4167 ts_tablet_manager.cc:1918] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:40.133912 3824 leader_election.cc:334] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.134052 3824 leader_election.cc:302] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.134282 5701 raft_consensus.cc:2726] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.137845 4169 rpcz_store.cc:276] 0811 18:57:35.800651 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.801165 (+ 514us) service_pool.cc:227] Handling call
0811 18:57:39.709459 (+3908294us) ts_tablet_manager.cc:1145] Deleting tablet b57ed7814a1e440cba362613cbab32f8
0811 18:57:39.709469 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:39.984358 (+274889us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.104584 (+120226us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.105875 (+ 1291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3908061}
I20260811 18:57:40.138234 4169 tablet_replica.cc:330] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.138479 4169 raft_consensus.cc:2229] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.138633 4169 raft_consensus.cc:2258] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.139181 4169 ts_tablet_manager.cc:1905] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.139436 4162 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
W20260811 18:57:40.142455 3824 leader_election.cc:341] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:40.142633 4162 ts_tablet_manager.cc:1397] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.067s user 0.003s sys 0.000s
I20260811 18:57:40.143419 4162 raft_consensus.cc:348] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.143710 4162 raft_consensus.cc:374] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.143818 4162 raft_consensus.cc:729] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.143790 5701 consensus_queue.cc:1035] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.144316 4162 consensus_queue.cc:260] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.155476 4162 ts_tablet_manager.cc:1428] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:40.158365 5701 raft_consensus.cc:482] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.158557 5701 raft_consensus.cc:504] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.159142 5701 leader_election.cc:288] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:40.162611 3824 leader_election.cc:334] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.162938 5641 raft_consensus.cc:482] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.163151 5641 raft_consensus.cc:504] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.165786 3678 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.166359 5641 leader_election.cc:288] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.166682 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd393d9c4d24aab8a170e1329400656" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.162767 3824 leader_election.cc:302] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:40.168311 3676 leader_election.cc:341] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.168442 3676 leader_election.cc:302] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.168741 5701 raft_consensus.cc:2726] T d73a98bd35bd444195fd32fe6c67a04f P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.168990 5641 raft_consensus.cc:2726] T edd393d9c4d24aab8a170e1329400656 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.169056 3824 consensus_peers.cc:487] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.170187 4162 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.171593 4162 tablet_bootstrap.cc:654] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.143770 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0b4fff40ba9d49828facafd431939fd4 (DEFAULT_TABLE table=default.Table_e8dcb132b5b5474b8ff081e54a55dd3f [id=58e6357b58b4481c9fad17e3aa498147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.172528 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4fff40ba9d49828facafd431939fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.173103 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160" } modify_external_catalogs: true
I20260811 18:57:40.173391 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_056c759cfe3543f7a43de5b2fa3a4160" } modify_external_catalogs: true
W20260811 18:57:40.173928 3824 leader_election.cc:334] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.174059 3824 leader_election.cc:302] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.174593 5701 raft_consensus.cc:2726] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.175141 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b57ed7814a1e440cba362613cbab32f8 (table default.Table_fdc11a6352eb4aac9ff657cf245aceee [id=d195244ef4c34971940768caaad50721]) successfully deleted
I20260811 18:57:40.176375 4167 log.cc:1196] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/d37e81d376e14907b13746cca831a96b
I20260811 18:57:40.176546 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet fdab1c3c431c4dceb08b00eabcfa1ceb
I20260811 18:57:40.176738 4167 ts_tablet_manager.cc:1939] T d37e81d376e14907b13746cca831a96b P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.177520 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1179) took 1960 ms. Trace:
I20260811 18:57:40.177613 4167 rpcz_store.cc:276] 0811 18:57:38.217330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.217540 (+ 210us) service_pool.cc:227] Handling call
0811 18:57:40.066741 (+1849201us) ts_tablet_manager.cc:1145] Deleting tablet d37e81d376e14907b13746cca831a96b
0811 18:57:40.066753 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.090384 (+ 23631us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.133291 (+ 42907us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.177497 (+ 44206us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1848922}
I20260811 18:57:40.178021 4167 tablet_replica.cc:330] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.179567 4162 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.179700 4162 ts_tablet_manager.cc:1397] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260811 18:57:40.180330 4162 raft_consensus.cc:348] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.180558 4162 raft_consensus.cc:374] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.180622 4162 raft_consensus.cc:729] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.180842 4162 consensus_queue.cc:260] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.181275 4162 ts_tablet_manager.cc:1428] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:57:40.181527 4162 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.183145 4162 tablet_bootstrap.cc:654] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.197353 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_d57e6ac8edbc46a69270b9277778e7b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.198027 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d57e6ac8edbc46a69270b9277778e7b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.201332 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_020300b2b8b949278c35159bd43fe19d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.201882 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_020300b2b8b949278c35159bd43fe19d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.202328 5641 raft_consensus.cc:482] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.202261 5701 raft_consensus.cc:482] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.202507 5641 raft_consensus.cc:504] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.202555 5701 raft_consensus.cc:504] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.203193 5701 leader_election.cc:288] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.203408 5641 leader_election.cc:288] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.203951 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330801031cc74eb2a5bc2caf53a18ead" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.204537 3676 leader_election.cc:341] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:40.205178 3678 leader_election.cc:334] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.205323 3678 leader_election.cc:302] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.205722 5641 raft_consensus.cc:2726] T 330801031cc74eb2a5bc2caf53a18ead P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.206225 4163 tablet_bootstrap.cc:492] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.206393 4163 ts_tablet_manager.cc:1397] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.125s user 0.006s sys 0.001s
I20260811 18:57:40.206614 4167 raft_consensus.cc:2229] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.206799 4167 raft_consensus.cc:2258] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.207193 4163 raft_consensus.cc:348] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.207474 4163 raft_consensus.cc:374] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.207546 4163 raft_consensus.cc:729] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.207779 4163 consensus_queue.cc:260] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.210656 3745 raft_consensus.cc:1261] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Refusing update from remote peer 57e8f62fa63a4fee97f268f5b7d3ae24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:57:40.211723 3824 consensus_peers.cc:487] T 65234acc3d5a462683edbe8f223eeefc P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:40.215876 4167 ts_tablet_manager.cc:1905] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.234740 4163 ts_tablet_manager.cc:1428] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
W20260811 18:57:40.236487 3824 leader_election.cc:341] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:40.247491 3550 master_service.cc:946] ReplaceTablet: received request to replace tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:40.248734 4163 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.249541 5701 consensus_queue.cc:1035] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:40.250447 3824 leader_election.cc:341] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:40.263342 4163 tablet_bootstrap.cc:654] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.266461 4167 ts_tablet_manager.cc:1918] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.266686 4167 log.cc:1196] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/bfea0cd619024204b9bff1e2dd62b7b0
I20260811 18:57:40.267030 4167 ts_tablet_manager.cc:1939] T bfea0cd619024204b9bff1e2dd62b7b0 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.267882 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1182) took 1999 ms. Trace:
I20260811 18:57:40.267992 4167 rpcz_store.cc:276] 0811 18:57:38.268190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.268658 (+ 468us) service_pool.cc:227] Handling call
0811 18:57:40.177974 (+1909316us) ts_tablet_manager.cc:1145] Deleting tablet bfea0cd619024204b9bff1e2dd62b7b0
0811 18:57:40.177984 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.247216 (+ 69232us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.266314 (+ 19098us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.267859 (+ 1545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1909014}
I20260811 18:57:40.273845 4162 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.274003 4162 ts_tablet_manager.cc:1397] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.002s
I20260811 18:57:40.274814 4162 raft_consensus.cc:348] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.275038 4162 raft_consensus.cc:374] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.275096 4162 raft_consensus.cc:729] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.275307 4162 consensus_queue.cc:260] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.276403 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1183) took 2008 ms. Trace:
I20260811 18:57:40.276938 4167 rpcz_store.cc:276] 0811 18:57:38.268326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269008 (+ 682us) service_pool.cc:227] Handling call
0811 18:57:40.276240 (+2007232us) ts_tablet_manager.cc:1145] Deleting tablet 69059a0667234a408cc1c09585df63bc
0811 18:57:40.276254 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.276378 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2007040}
I20260811 18:57:40.277998 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1184) took 2009 ms. Trace:
I20260811 18:57:40.278492 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.278462 4167 rpcz_store.cc:276] 0811 18:57:38.268446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269225 (+ 779us) service_pool.cc:227] Handling call
0811 18:57:40.277860 (+2008635us) ts_tablet_manager.cc:1145] Deleting tablet 82b04e37fc264fc08e0ed9517356a17d
0811 18:57:40.277873 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.277980 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2008478}
I20260811 18:57:40.280203 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_8be7a98fc3664e94b5af6924771fdd01" } new_table_name: "default.Table_0005d076764c413e92498cb07ad6920a" modify_external_catalogs: true
I20260811 18:57:40.279584 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1185) took 2011 ms. Trace:
I20260811 18:57:40.280473 4167 rpcz_store.cc:276] 0811 18:57:38.268592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269416 (+ 824us) service_pool.cc:227] Handling call
0811 18:57:40.279450 (+2010034us) ts_tablet_manager.cc:1145] Deleting tablet d11b889e9d044a049d0586382c263951
0811 18:57:40.279463 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.279564 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2009885}
I20260811 18:57:40.281639 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1181) took 2013 ms. Trace:
I20260811 18:57:40.283087 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_53bcd19697ad49b9aec577b1646c0d3e" } new_table_name: "default.Table_69847be1563a4aad8d093d935864f531" modify_external_catalogs: true
I20260811 18:57:40.285334 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.286736 4167 rpcz_store.cc:276] 0811 18:57:38.268064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.269783 (+ 1719us) service_pool.cc:227] Handling call
0811 18:57:40.281515 (+2011732us) ts_tablet_manager.cc:1145] Deleting tablet e9f0e443ab84424dac307b745b496986
0811 18:57:40.281526 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.281623 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2011543}
I20260811 18:57:40.287838 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1180) took 2020 ms. Trace:
I20260811 18:57:40.288245 4167 rpcz_store.cc:276] 0811 18:57:38.267899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.295034 (+ 27135us) service_pool.cc:227] Handling call
0811 18:57:40.287672 (+1992638us) ts_tablet_manager.cc:1145] Deleting tablet 6710084f21734dea9c7ce1f43d1bf587
0811 18:57:40.287691 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.287818 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1992161}
I20260811 18:57:40.288964 4167 tablet_replica.cc:330] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.289393 4167 raft_consensus.cc:2229] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:40.293269 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7ce2b43b2134defb14d45112e4f5e5e consensus state has no leader
I20260811 18:57:40.294030 4167 raft_consensus.cc:2258] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.295076 4163 tablet_bootstrap.cc:492] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:40.295320 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0bac7ac254f4471397580dc510648dfc has no consensus state
I20260811 18:57:40.295384 4163 ts_tablet_manager.cc:1397] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.003s
I20260811 18:57:40.296180 4163 raft_consensus.cc:348] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.296411 4163 raft_consensus.cc:374] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.296463 4163 raft_consensus.cc:729] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.296635 4163 consensus_queue.cc:260] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.297252 4163 ts_tablet_manager.cc:1428] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:40.297559 4163 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.299288 4163 tablet_bootstrap.cc:654] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.300597 4167 ts_tablet_manager.cc:1905] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.307492 4162 ts_tablet_manager.cc:1428] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260811 18:57:40.309325 4167 ts_tablet_manager.cc:1918] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:40.316587 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_5a407d0c02054f5ca4370e8e1a1c4c71" } new_table_name: "default.Table_4821250a7fa84b68a74499b0d7a1228d" modify_external_catalogs: true
W20260811 18:57:40.317546 3824 leader_election.cc:334] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.317694 3824 leader_election.cc:302] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.317920 5630 raft_consensus.cc:2726] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.318301 3824 leader_election.cc:334] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.318526 3824 leader_election.cc:302] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.318836 5630 raft_consensus.cc:2726] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.319167 3824 consensus_peers.cc:487] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.319407 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_01425dcfb75d41f6ad6349c65957ebac" } new_table_name: "default.Table_7f4309a3032b46a59a721bb32943bff8" modify_external_catalogs: true
I20260811 18:57:40.320807 5630 raft_consensus.cc:482] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.320930 5630 raft_consensus.cc:504] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.321538 5630 leader_election.cc:288] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.328706 4169 ts_tablet_manager.cc:1918] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.328878 4169 log.cc:1196] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0aee6beb2e70421083841520733087f8
I20260811 18:57:40.329257 4169 ts_tablet_manager.cc:1939] T 0aee6beb2e70421083841520733087f8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:40.329763 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66e70ff397bd493992d2f08a30221b69 has no consensus state
I20260811 18:57:40.332235 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_075181f96b7b42059289018e831e2671" } new_table_name: "default.Table_9ec52027b1c944a69e731fed8ad2bd30" modify_external_catalogs: true
I20260811 18:57:40.332383 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1035) took 4504 ms. Trace:
I20260811 18:57:40.332473 4169 rpcz_store.cc:276] 0811 18:57:35.827679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.827838 (+ 159us) service_pool.cc:227] Handling call
0811 18:57:40.138180 (+4310342us) ts_tablet_manager.cc:1145] Deleting tablet 0aee6beb2e70421083841520733087f8
0811 18:57:40.138191 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.213178 (+ 74987us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.277840 (+ 64662us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.332361 (+ 54521us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4310134}
I20260811 18:57:40.332890 4169 tablet_replica.cc:330] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.333122 4169 raft_consensus.cc:2229] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.333304 4169 raft_consensus.cc:2258] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.333932 4169 ts_tablet_manager.cc:1905] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.340049 4167 log.cc:1196] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:40.318482 3715 tablet_service.cc:1514] Processing DeleteTablet for tablet fa95131ce50943e3b0c6b5bdd5a30939 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:40.330789 5407 raft_consensus.cc:482] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.340639 5407 raft_consensus.cc:504] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.341315 5407 leader_election.cc:288] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.310797 4162 tablet_bootstrap.cc:492] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.342275 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a64df7c295c4d9baded1b63a9980581" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.342464 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1b92a533c58b43c69fd65a1ec335cb3a consensus state has no leader
W20260811 18:57:40.342959 3676 leader_election.cc:341] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.344071 4162 tablet_bootstrap.cc:654] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.344304 3678 leader_election.cc:334] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.344434 3678 leader_election.cc:302] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.344674 5407 raft_consensus.cc:2726] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.346910 4162 tablet_bootstrap.cc:492] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.347060 4162 ts_tablet_manager.cc:1397] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260811 18:57:40.347756 4162 raft_consensus.cc:348] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.347990 4162 raft_consensus.cc:374] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.348053 4162 raft_consensus.cc:729] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.348047 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0aee6beb2e70421083841520733087f8 (table default.Table_365efecf4f8e4f4993ba5579614a3049 [id=22478156dcb4450d9c8b30abaed1bc9a]) successfully deleted
I20260811 18:57:40.348635 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_c17b389153604aeeb9c985eaf661f44a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.349282 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c17b389153604aeeb9c985eaf661f44a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:57:40.349764 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet 49203e259d17449984dcd528df985537: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.353294 4167 ts_tablet_manager.cc:1939] T b4c7f500a5444bae9a015a5e903c0622 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.353024 5642 catalog_manager.cc:1259] Loaded cluster ID: fefb698c4e0e40b0a6c24392acdfe38e
I20260811 18:57:40.359447 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1188) took 1888 ms. Trace:
I20260811 18:57:40.360322 3549 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.360790 4162 consensus_queue.cc:260] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.361215 4167 rpcz_store.cc:276] 0811 18:57:38.471318 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.478361 (+ 7043us) service_pool.cc:227] Handling call
0811 18:57:40.288916 (+1810555us) ts_tablet_manager.cc:1145] Deleting tablet b4c7f500a5444bae9a015a5e903c0622
0811 18:57:40.288927 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.306248 (+ 17321us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.309217 (+ 2969us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.359417 (+ 50200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1810327}
I20260811 18:57:40.362432 4162 ts_tablet_manager.cc:1428] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260811 18:57:40.362740 4162 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.363312 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_5e94a3f1b3a1418d8d5ff4ae641da751"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.363931 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e94a3f1b3a1418d8d5ff4ae641da751 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.364163 4162 tablet_bootstrap.cc:654] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.365332 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1190) took 1893 ms. Trace:
I20260811 18:57:40.365797 4167 rpcz_store.cc:276] 0811 18:57:38.471548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.482397 (+ 10849us) service_pool.cc:227] Handling call
0811 18:57:40.365190 (+1882793us) ts_tablet_manager.cc:1145] Deleting tablet 31bb11617f1548f88f534289706edb06
0811 18:57:40.365201 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.365311 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1882576}
I20260811 18:57:40.366487 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1191) took 1894 ms. Trace:
I20260811 18:57:40.366981 4167 rpcz_store.cc:276] 0811 18:57:38.471668 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.482665 (+ 10997us) service_pool.cc:227] Handling call
0811 18:57:40.366330 (+1883665us) ts_tablet_manager.cc:1145] Deleting tablet 76506ad179a9408c9e4cf8e4275ebddd
0811 18:57:40.366377 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.366469 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1883495}
I20260811 18:57:40.367514 4167 tablet_replica.cc:330] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.367584 3546 catalog_manager.cc:5495] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:40.368086 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:40.369325 4167 raft_consensus.cc:2229] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.369506 4167 raft_consensus.cc:2258] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.370088 4167 ts_tablet_manager.cc:1905] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.370855 5642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: loading cluster ID for follower catalog manager: success
W20260811 18:57:40.376416 3824 leader_election.cc:341] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:40.383720 3871 tablet_service.cc:1467] Processing CreateTablet for tablet c82456373d974f35846ade45e2398bbd (DEFAULT_TABLE table=default.Table_4b90b4ead71940e58e93af4a9804a7d8 [id=ac620652bd4b404da86f7dda6ed307d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.384250 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82456373d974f35846ade45e2398bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.387961 4163 tablet_bootstrap.cc:492] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.388146 4163 ts_tablet_manager.cc:1397] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.006s
I20260811 18:57:40.388046 5728 tablet.cc:1722] T 3c96a264e017471ea114ca0c6a9500ef P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:40.388450 5407 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.388581 5407 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.389225 5407 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.389956 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.390203 3887 raft_consensus.cc:2454] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
W20260811 18:57:40.390635 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d73a43558dbb48889ab205108f4ce6be consensus state has no leader
I20260811 18:57:40.390714 3676 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:40.390654 4163 raft_consensus.cc:348] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.390980 4163 raft_consensus.cc:374] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.391045 4163 raft_consensus.cc:729] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.391081 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_388bf7f2862e42ebb5e3a779b1ade554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.391258 4163 consensus_queue.cc:260] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.391389 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.391826 4163 ts_tablet_manager.cc:1428] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 18:57:40.392109 4163 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.393184 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.393363 3552 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:40.393865 4163 tablet_bootstrap.cc:654] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.394506 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_388bf7f2862e42ebb5e3a779b1ade554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.394969 5407 raft_consensus.cc:2781] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.395066 5407 raft_consensus.cc:482] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.395140 5407 raft_consensus.cc:3037] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.396366 3552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_f20cec778f3248419eb734a5100ae8d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.396910 3552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f20cec778f3248419eb734a5100ae8d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.398056 5407 raft_consensus.cc:504] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.407722 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 0e556811ad1643f1baa3bbf4edebd157
I20260811 18:57:40.409080 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "473040a90e6941c38beb697316d2a27b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.409322 3887 raft_consensus.cc:3037] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.412243 3887 raft_consensus.cc:2454] T 473040a90e6941c38beb697316d2a27b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.413586 5407 leader_election.cc:288] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.414808 3678 leader_election.cc:334] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.415508 3676 leader_election.cc:302] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.416277 4167 ts_tablet_manager.cc:1918] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.416236 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b691c53749bc40a0a829753ba4c7e6aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.417223 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b691c53749bc40a0a829753ba4c7e6aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.420261 3871 tablet_service.cc:1467] Processing CreateTablet for tablet defce8dba232424ab74ebae42d6a078a (DEFAULT_TABLE table=default.Table_7a85117c40464174beed058911301729 [id=728fa9eb1e2947c8994d451b9a80eaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.420809 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defce8dba232424ab74ebae42d6a078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.416615 5407 raft_consensus.cc:2781] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.424954 4167 log.cc:1196] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8304ac7709ca4392baec4cfa78ef5b41
I20260811 18:57:40.425194 5407 raft_consensus.cc:686] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.425889 4167 ts_tablet_manager.cc:1939] T 8304ac7709ca4392baec4cfa78ef5b41 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.426122 5407 consensus_queue.cc:237] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.427708 4167 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38286 (request call id 1194) took 1149 ms. Trace:
I20260811 18:57:40.428449 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 4a4afeb6069e4d3d9457fffcf5ec48f0
I20260811 18:57:40.428948 3547 catalog_manager.cc:6565] ReplaceTablet: tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 deleted and replaced by tablet fe48bbab3d7e4f7591883296a9996db7
I20260811 18:57:40.433809 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_8edb4b4387a444abb3b70122ae70018e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.434778 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8edb4b4387a444abb3b70122ae70018e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.434768 4167 rpcz_store.cc:276] 0811 18:57:39.278179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.278257 (+ 78us) service_pool.cc:227] Handling call
0811 18:57:40.367470 (+1089213us) ts_tablet_manager.cc:1145] Deleting tablet 8304ac7709ca4392baec4cfa78ef5b41
0811 18:57:40.367481 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.410571 (+ 43090us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.416173 (+ 5602us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.427683 (+ 11510us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1086122}
I20260811 18:57:40.437476 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.437971 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 19243ab3d6744aaa87a133f1462cd2fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.438643 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 9d09471428a5403da8b3bbf0f1597ed3 (DEFAULT_TABLE table=default.Table_e894321d446d4d6da9e866788ff2b1e7 [id=fd4705a377a94d3eb676f4dd0f37e096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.439096 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d09471428a5403da8b3bbf0f1597ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.440461 4167 tablet_replica.cc:330] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.440722 4167 raft_consensus.cc:2229] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.440891 4167 raft_consensus.cc:2258] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.441450 4167 ts_tablet_manager.cc:1905] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.442699 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 5d93073a69c143ff9460248848a108d4
I20260811 18:57:40.448390 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6dab2aa50f394dabbbae787dca84b57a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.449004 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dab2aa50f394dabbbae787dca84b57a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.455476 5723 tablet.cc:1722] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260811 18:57:40.457772 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet ffa8a4a1c0cb4834be501bdd7d468472 from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:40.458696 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 83141.9us
I20260811 18:57:40.460860 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2" } modify_external_catalogs: true
I20260811 18:57:40.461071 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c6284ad4471b4b7095b11fe7a8f304e2" } modify_external_catalogs: true
I20260811 18:57:40.462713 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 with delete_type TABLET_DATA_DELETED (replaced by tablet fe48bbab3d7e4f7591883296a9996db7) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.467928 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d93073a69c143ff9460248848a108d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.468961 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 8794dcb46ccc442e9271fc33453e4800 (DEFAULT_TABLE table=default.Table_f614f2bd34c949cd94f9493a086dd8c7 [id=9b6853170f7b4ebb9c266f96c7d99e10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.469528 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8794dcb46ccc442e9271fc33453e4800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.474979 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 56542f3afe304feb888f101961d855c7 (DEFAULT_TABLE table=default.Table_9919772109274064aee942f6a7f1220f [id=75a15771a07045039c110309e3dc0beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.475267 4163 tablet_bootstrap.cc:492] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.475402 4163 ts_tablet_manager.cc:1397] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.083s user 0.004s sys 0.010s
I20260811 18:57:40.475452 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56542f3afe304feb888f101961d855c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.476094 4163 raft_consensus.cc:348] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.476310 4163 raft_consensus.cc:374] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.476361 4163 raft_consensus.cc:729] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.477160 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 71a7eda2248c4f708bdc772ea3598dec
I20260811 18:57:40.479887 4167 ts_tablet_manager.cc:1918] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.480786 4167 log.cc:1196] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b4dc2df4c6464d71bb13fe25e0046758
I20260811 18:57:40.481312 4167 ts_tablet_manager.cc:1939] T b4dc2df4c6464d71bb13fe25e0046758 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.482920 5630 raft_consensus.cc:482] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.482786 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:40.483387 5630 raft_consensus.cc:504] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.483893 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.484136 5630 leader_election.cc:288] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.487383 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 71a7eda2248c4f708bdc772ea3598dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.486692 4163 consensus_queue.cc:260] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.487720 3824 leader_election.cc:334] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.487918 3824 leader_election.cc:302] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:40.490728 4163 ts_tablet_manager.cc:1428] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260811 18:57:40.491080 4163 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.492985 4163 tablet_bootstrap.cc:654] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.494007 4167 tablet_replica.cc:330] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:40.494532 4167 raft_consensus.cc:2229] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.495179 4167 raft_consensus.cc:2258] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.499025 4167 ts_tablet_manager.cc:1905] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.500532 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a49a2ac4cc894d7fb17104d0ea4fa6e7
I20260811 18:57:40.500977 3550 catalog_manager.cc:6565] ReplaceTablet: tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 deleted and replaced by tablet 79539e937f81437ab3b12b8465f12d97
I20260811 18:57:40.502851 5630 raft_consensus.cc:2726] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.509923 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_a4a5dc90bb3449ee94760cc397fd49f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.511152 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4a5dc90bb3449ee94760cc397fd49f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.522330 5641 raft_consensus.cc:482] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.522586 5641 raft_consensus.cc:504] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.523362 5641 leader_election.cc:288] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.526888 3366 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.3.72.66:40837, user_credentials={real_user=slave}} blocked reactor thread for 51297.5us
W20260811 18:57:40.527031 3678 leader_election.cc:334] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.527709 5641 raft_consensus.cc:482] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.527892 5641 raft_consensus.cc:504] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.528599 5641 leader_election.cc:288] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.528988 4167 ts_tablet_manager.cc:1918] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:40.529675 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1fc828f73642c88171d6ac827de90e" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.530544 3676 leader_election.cc:341] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.530946 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a994b3f4d7d4c92946ad8d595e6ad2b" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.530540 4167 log.cc:1196] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ffa8a4a1c0cb4834be501bdd7d468472
W20260811 18:57:40.531483 3676 leader_election.cc:341] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.531635 3676 leader_election.cc:302] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:40.532217 3678 leader_election.cc:334] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.532516 3678 leader_election.cc:302] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.532943 4167 ts_tablet_manager.cc:1939] T ffa8a4a1c0cb4834be501bdd7d468472 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:40.533259 5641 raft_consensus.cc:2726] T 3a994b3f4d7d4c92946ad8d595e6ad2b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.533584 5641 raft_consensus.cc:2726] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:40.544001 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 56300.6us
I20260811 18:57:40.544764 3546 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_078a10661db44f05a3516fe2dd35d154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.547345 3546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_078a10661db44f05a3516fe2dd35d154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.558652 3554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb
I20260811 18:57:40.561429 4169 ts_tablet_manager.cc:1918] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:40.561607 4169 log.cc:1196] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e3021c51525846e199f279f988248af0
I20260811 18:57:40.562091 4169 ts_tablet_manager.cc:1939] T e3021c51525846e199f279f988248af0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:40.562084 3554 catalog_manager.cc:5495] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:40.563891 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:40.564260 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 3a994b3f4d7d4c92946ad8d595e6ad2b (table default.Table_079d99e744054f2a8a2d6ac002aca4e3 [id=9c62165c055f4acea600e53873f88772]). Expected version 2 got 0
I20260811 18:57:40.564713 3554 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet faf0594d55844ae49c99247ecd8b810f (table default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b]). Expected version 3 got 0
I20260811 18:57:40.566551 4163 tablet_bootstrap.cc:492] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.566728 4163 ts_tablet_manager.cc:1397] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.076s user 0.008s sys 0.011s
I20260811 18:57:40.567422 4163 raft_consensus.cc:348] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.567649 4163 raft_consensus.cc:374] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.567713 4163 raft_consensus.cc:729] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.567896 4163 consensus_queue.cc:260] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.570134 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 108372us
I20260811 18:57:40.577679 3871 tablet_service.cc:1467] Processing CreateTablet for tablet d8c6b62d2a684869b0ad31a1b25c7ebf (DEFAULT_TABLE table=default.Table_bf1e0bf055c44f45835974098ec141f8 [id=8f5e99a55df34267a0b32710772f1978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.578990 4163 ts_tablet_manager.cc:1428] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260811 18:57:40.582372 5701 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.582523 5701 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.583110 5701 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:40.590243 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c6b62d2a684869b0ad31a1b25c7ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.591312 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_298b25b6ed544a9fb0118e1af056b525" } new_table_name: "default.Table_df6343d393304ce6bd7043ce8b51cc50" modify_external_catalogs: true
I20260811 18:57:40.591789 4162 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.591933 4162 ts_tablet_manager.cc:1397] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.229s user 0.015s sys 0.024s
I20260811 18:57:40.592656 4162 raft_consensus.cc:348] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.592892 4162 raft_consensus.cc:374] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.592947 4162 raft_consensus.cc:729] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.593144 4162 consensus_queue.cc:260] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.593567 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6936ad30157d4323baa32612df13a29d" } new_table_name: "default.Table_70318db01b604cd79403f7bf85270fce" modify_external_catalogs: true
I20260811 18:57:40.593712 4162 ts_tablet_manager.cc:1428] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:40.594033 4162 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.601235 4162 tablet_bootstrap.cc:654] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.603937 5734 raft_consensus.cc:482] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.604216 5734 raft_consensus.cc:504] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.605386 4162 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.605578 4162 ts_tablet_manager.cc:1397] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260811 18:57:40.606931 4162 raft_consensus.cc:348] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.607196 4162 raft_consensus.cc:374] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.607272 4162 raft_consensus.cc:729] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.607484 4162 consensus_queue.cc:260] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.608083 4162 ts_tablet_manager.cc:1428] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:40.608394 4162 tablet_bootstrap.cc:492] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:40.609248 3678 leader_election.cc:334] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.609771 5734 leader_election.cc:288] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.590822 4163 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:40.607118 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1e244addf5462499fcadaa3332cba1" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.610257 4162 tablet_bootstrap.cc:654] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:40.610711 3676 leader_election.cc:341] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.610831 3676 leader_election.cc:302] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.611446 5735 tablet.cc:1722] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:40.613119 4163 tablet_bootstrap.cc:654] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.616297 3547 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_f94d755a69ab49dcbbc91974db72cb77" } new_table_name: "default.Table_2a545f0339f848b6a536ac32accb3d2b" modify_external_catalogs: true
I20260811 18:57:40.616786 5733 raft_consensus.cc:2726] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.618453 3555 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_25c8439e034242f7bc3d1c546188fc8e" } new_table_name: "default.Table_f9902d67710648848c10bb18357257a6" modify_external_catalogs: true
I20260811 18:57:40.628548 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 72f11748cf984942ba99bda4ad6605ab
I20260811 18:57:40.629180 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet cf6c8d542ae2489ba466ffadd518db12 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:40.629812 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_59d30e9c40554addab5dc9b7c1d11432"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.630414 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59d30e9c40554addab5dc9b7c1d11432 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.646085 4162 tablet_bootstrap.cc:492] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.646397 4162 ts_tablet_manager.cc:1397] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.003s
I20260811 18:57:40.647333 4162 raft_consensus.cc:348] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.647660 4162 raft_consensus.cc:374] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.647805 4162 raft_consensus.cc:729] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.648126 4162 consensus_queue.cc:260] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.648916 4162 ts_tablet_manager.cc:1428] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:40.649854 4162 tablet_bootstrap.cc:492] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.651999 3553 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_9ddf3bbaa80e4104bba8ebbcbbe77f28" } new_table_name: "default.Table_7fcad849494e46c1bcef25d34ef0cd78" modify_external_catalogs: true
W20260811 18:57:40.664992 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 81393.2us
W20260811 18:57:40.665505 3824 consensus_peers.cc:487] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4c7f500a5444bae9a015a5e903c0622. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.666559 5733 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.666831 5733 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.669538 4162 tablet_bootstrap.cc:654] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.669590 5733 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.673803 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.674124 3887 raft_consensus.cc:2454] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 0.
I20260811 18:57:40.674773 3676 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:40.675064 5733 raft_consensus.cc:2781] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.675205 5733 raft_consensus.cc:482] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.675346 5733 raft_consensus.cc:3037] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:57:40.677346 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.680563 3550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6" } modify_external_catalogs: true
I20260811 18:57:40.680809 3550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_e4ca2c0d65cd4c65a2580b79b4122cf6" } modify_external_catalogs: true
I20260811 18:57:40.689117 5593 raft_consensus.cc:482] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.689321 5593 raft_consensus.cc:504] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.689882 5593 leader_election.cc:288] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:40.690541 5630 consensus_queue.cc:1035] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:40.694589 3824 leader_election.cc:341] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 6b4de110bd3348f0b0dae21c9f2b6a19
W20260811 18:57:40.696766 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1922c7317c24aa984945aaaa6ce8003 consensus state has no leader
I20260811 18:57:40.695129 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1036) took 4814 ms. Trace:
I20260811 18:57:40.699504 4169 rpcz_store.cc:276] 0811 18:57:35.880779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.881029 (+ 250us) service_pool.cc:227] Handling call
0811 18:57:40.332842 (+4451813us) ts_tablet_manager.cc:1145] Deleting tablet e3021c51525846e199f279f988248af0
0811 18:57:40.332854 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.448406 (+115552us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.561256 (+112850us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:40.695085 (+133829us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4451518}
I20260811 18:57:40.699913 4169 tablet_replica.cc:330] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:40.703034 4163 tablet_bootstrap.cc:492] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:40.703213 4163 ts_tablet_manager.cc:1397] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.112s user 0.003s sys 0.006s
I20260811 18:57:40.704037 4163 raft_consensus.cc:348] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.704291 4163 raft_consensus.cc:374] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.704363 4163 raft_consensus.cc:729] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.704571 4163 consensus_queue.cc:260] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.708248 4163 ts_tablet_manager.cc:1428] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:57:40.708621 4163 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:40.719354 3547 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25266bc7788f49f8af8eb9a948e61c8d consensus state has no leader
I20260811 18:57:40.719807 3547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38" } modify_external_catalogs: true
I20260811 18:57:40.720011 3547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_6f13efa6fe524f5bbd7154aa2f991d38" } modify_external_catalogs: true
W20260811 18:57:40.721551 3555 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae75d6652c704e1995966c6dd4c06307 consensus state has no leader
W20260811 18:57:40.723438 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9948c33ed58b4160932be82ef9615ba5 consensus state has no leader
I20260811 18:57:40.724624 3546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c925e9ac591346b1a5142f338c68bcc6" } modify_external_catalogs: true
I20260811 18:57:40.724819 3546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c925e9ac591346b1a5142f338c68bcc6" } modify_external_catalogs: true
I20260811 18:57:40.726410 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_22ffcd40375b45aab1616fa1fb6f030d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.727078 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22ffcd40375b45aab1616fa1fb6f030d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.729166 3550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet b50efab3c782447cad8e45b1fa51aa1f
I20260811 18:57:40.734637 4169 raft_consensus.cc:2229] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.734966 4169 raft_consensus.cc:2258] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.735913 4169 ts_tablet_manager.cc:1905] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.752440 3554 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_59c9cdf2f7424fa7980a53799ff004d3 [id=f5844f91d9ef446bbd4d34bdea7cba8d] alter complete (version 1)
I20260811 18:57:40.688943 5733 raft_consensus.cc:504] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.721388 4162 tablet_bootstrap.cc:492] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:40.768213 4162 ts_tablet_manager.cc:1397] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.118s user 0.009s sys 0.004s
I20260811 18:57:40.769117 4162 raft_consensus.cc:348] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.769200 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2599dbd65a814601928373ab9910e3ca" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24"
I20260811 18:57:40.769349 4162 raft_consensus.cc:374] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.769404 4162 raft_consensus.cc:729] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.769444 3887 raft_consensus.cc:3037] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.769652 4162 consensus_queue.cc:260] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:40.723304 3746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:40.770314 3746 raft_consensus.cc:2454] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:40.772637 3887 raft_consensus.cc:2454] T 2599dbd65a814601928373ab9910e3ca P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c4bdbd811417ab011a7131f17d283 in term 1.
I20260811 18:57:40.705665 5734 raft_consensus.cc:482] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.773872 5734 raft_consensus.cc:504] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.775307 5734 leader_election.cc:288] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.760870 5407 raft_consensus.cc:482] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.778431 5407 raft_consensus.cc:504] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.779078 5407 leader_election.cc:288] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.765305 5641 raft_consensus.cc:482] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.764583 5728 tablet.cc:1722] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260811 18:57:40.779400 5641 raft_consensus.cc:504] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.780015 5641 leader_election.cc:288] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.778023 3678 leader_election.cc:334] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.780483 3678 leader_election.cc:334] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.783810 3678 leader_election.cc:334] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:40.784041 3678 leader_election.cc:334] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.784823 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0116997a1c64e658045c319f1c45861" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.784932 3676 leader_election.cc:302] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.785287 5734 raft_consensus.cc:2781] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.785346 3888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe24bb6edb944008ff9cc73faa7374c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:40.785663 3676 leader_election.cc:341] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.785794 3676 leader_election.cc:302] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.786187 5641 raft_consensus.cc:2726] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.785653 5734 raft_consensus.cc:686] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Running, Role: LEADER
I20260811 18:57:40.786892 5734 consensus_queue.cc:237] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:40.789413 3676 leader_election.cc:341] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.789541 3676 leader_election.cc:302] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:40.789599 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf7720e31d84ae2ab592da1e7a3132a" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.789956 5641 raft_consensus.cc:2726] T 8fe24bb6edb944008ff9cc73faa7374c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.790189 5733 leader_election.cc:288] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
W20260811 18:57:40.790298 3676 leader_election.cc:341] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:40.790405 3676 leader_election.cc:302] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:40.797078 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 222712us
I20260811 18:57:40.804664 4163 tablet_bootstrap.cc:654] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.806788 5641 raft_consensus.cc:2726] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.815836 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_525108ad39d242bc818b5bb86a5c0103"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.816668 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_525108ad39d242bc818b5bb86a5c0103 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.818450 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_beb35f174ef644f5994dd0282f0d4ee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.818984 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_beb35f174ef644f5994dd0282f0d4ee3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.820214 3549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_65f6fe5195b443fcbd5f077f1a147107"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.820669 3549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65f6fe5195b443fcbd5f077f1a147107 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.821846 3552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_58f2aeab56c6446184cfeddcb1134314" } modify_external_catalogs: true
I20260811 18:57:40.822029 3552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_58f2aeab56c6446184cfeddcb1134314" } modify_external_catalogs: true
I20260811 18:57:40.825174 3547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet bf1c037a37504b3fad9612f5a734dd33
I20260811 18:57:40.816634 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 79539e937f81437ab3b12b8465f12d97) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.826810 3546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet a7946503bec9446d8deb02448eb231d5
I20260811 18:57:40.832841 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet e18242f762d8407db0e2f8716890feb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.833756 3851 tablet_service.cc:1467] Processing CreateTablet for tablet a54921b815f2408c96bdf18e243041d5 (DEFAULT_TABLE table=default.Table_23655b2975534b81b076fd031ea5022a [id=e040f0cd609644928bb9690a053fa426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.834780 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.840480 3859 tablet_service.cc:1467] Processing CreateTablet for tablet f891108d44c44296934924941981b568 (DEFAULT_TABLE table=default.Table_4b752724648a499980ad71fd596a731f [id=78f3a4a7d6ad41278417b8eef697d0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.845278 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 0bac7ac254f4471397580dc510648dfc (DEFAULT_TABLE table=default.Table_69847be1563a4aad8d093d935864f531 [id=6a7e6156c0c247f68bcdb76ed1eb6365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.849972 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 756062a54d144339b91f8c68d9edceab (DEFAULT_TABLE table=default.Table_e142d545d8b249f3835bce3bbef9f94e [id=51b0aa68c2e749768c886732760d4f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.868304 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54921b815f2408c96bdf18e243041d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.868907 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f891108d44c44296934924941981b568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.869251 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bac7ac254f4471397580dc510648dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.869573 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756062a54d144339b91f8c68d9edceab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.878120 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_3aa1c735a06c46e9833fdb958cac274d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.878929 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3aa1c735a06c46e9833fdb958cac274d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.881009 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_6f3b91b31a884708a27160ebfca75cab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.881682 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f3b91b31a884708a27160ebfca75cab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.886219 5733 consensus_queue.cc:1035] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.892804 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" } modify_external_catalogs: true
W20260811 18:57:40.912899 3678 consensus_peers.cc:487] T d73a98bd35bd444195fd32fe6c67a04f P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:40.919169 3862 tablet_service.cc:1467] Processing CreateTablet for tablet ce6a80313c854c17913db64aeab9acf3 (DEFAULT_TABLE table=default.Table_c4b27a9eb0994efe9f4176f74c16a62e [id=6e29d4572cd64f8cb70940f4119296e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.919744 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6a80313c854c17913db64aeab9acf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.935029 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b50efab3c782447cad8e45b1fa51aa1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.935449 3867 tablet_service.cc:1467] Processing CreateTablet for tablet a07f682b68114c2b9cb0df27a94406ec (DEFAULT_TABLE table=default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.936100 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07f682b68114c2b9cb0df27a94406ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.943498 4162 ts_tablet_manager.cc:1428] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.175s user 0.004s sys 0.007s
I20260811 18:57:40.950555 4163 tablet_bootstrap.cc:492] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
W20260811 18:57:40.950781 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 92904.8us
I20260811 18:57:40.958736 4162 tablet_bootstrap.cc:492] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:40.967087 4163 ts_tablet_manager.cc:1397] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.259s user 0.005s sys 0.000s
I20260811 18:57:40.962435 5742 raft_consensus.cc:482] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.968542 5742 raft_consensus.cc:504] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.968181 5630 raft_consensus.cc:482] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.970994 5630 raft_consensus.cc:504] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.971894 5630 leader_election.cc:288] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:40.972911 3678 leader_election.cc:334] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:40.974181 4163 raft_consensus.cc:348] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.979110 5742 leader_election.cc:288] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:40.986301 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e3021c51525846e199f279f988248af0 (table default.Table_e394e4379aae4ac8b877d52273a6bf37 [id=4f31c3c9a50941fba7ad9e1d390c85a7]) successfully deleted
I20260811 18:57:40.998678 4163 raft_consensus.cc:374] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.998947 4163 raft_consensus.cc:729] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.999325 4163 consensus_queue.cc:260] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:40.976478 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0540e4e43ea34fbfbc66b6035c60684d" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:40.978869 3859 tablet_service.cc:1467] Processing CreateTablet for tablet bba6561b8c4b4c728db18446a9a91f30 (DEFAULT_TABLE table=default.Table_5affb8acc7e5466ba5ed988728ba50cb [id=0a26d099a1324a9aa8005962a8d9bd79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.000206 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba6561b8c4b4c728db18446a9a91f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.982708 3852 tablet_service.cc:1514] Processing DeleteTablet for tablet bf1c037a37504b3fad9612f5a734dd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:41.003258 3676 leader_election.cc:341] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.003432 3676 leader_election.cc:302] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.009931 5742 raft_consensus.cc:2726] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:40.983560 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 997be6173a194bbe91c6c0b9046d1a0e (DEFAULT_TABLE table=default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae [id=7586915510704916afd712ab81f9f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.984809 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet a7946503bec9446d8deb02448eb231d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.985358 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 01fe8e896b2246fabd2b6fae396bf085 (DEFAULT_TABLE table=default.Table_f83d8784174f40e29b96eba2de396065 [id=fdb879cbb1cb4f498f94d14048b87e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.010891 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01fe8e896b2246fabd2b6fae396bf085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.985924 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 81e467ea40364946b7c0bc4b5592996b (DEFAULT_TABLE table=default.Table_2486985deae44613a5f8d342c8236784 [id=74d8ef880ca24497ace800c16e041dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.014761 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e467ea40364946b7c0bc4b5592996b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.017249 3361 master-stress-test.cc:578] Killing test thread 1/25
W20260811 18:57:41.024854 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ae02c9b0b066474894546829aed304e3 (table default.Table_76b86d4f079c45b1a7f34a195fc97dfb [id=137cef1d67054dc6be05fcef9b22c375], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ae02c9b0b066474894546829aed304e3 consensus state has no leader
I20260811 18:57:40.986475 3871 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f11748cf984942ba99bda4ad6605ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:40.975323 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 86f215c21d184dd6ba828fa31286cce1 (DEFAULT_TABLE table=default.Table_d8f35609e86b47aa9a8ab711834da08f [id=72dfc9b4df824dcd92241af7ab88c68e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.029963 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f215c21d184dd6ba828fa31286cce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.048406 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 997be6173a194bbe91c6c0b9046d1a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.067209 4162 tablet_bootstrap.cc:654] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.069007 4163 ts_tablet_manager.cc:1428] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.096s user 0.002s sys 0.000s
I20260811 18:57:41.069476 4163 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.071363 4163 tablet_bootstrap.cc:654] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.050889 3824 leader_election.cc:334] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.072559 3824 leader_election.cc:302] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.072957 5630 raft_consensus.cc:2726] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.086077 4169 ts_tablet_manager.cc:1918] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.086373 4169 log.cc:1196] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/968fa8b8912b40a8bdba77db1d39c5c0
I20260811 18:57:41.086845 4169 ts_tablet_manager.cc:1939] T 968fa8b8912b40a8bdba77db1d39c5c0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.086078 3550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_88ee0abdcd9943568a07d73017ff598a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.087409 3550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88ee0abdcd9943568a07d73017ff598a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.087821 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_60eefdf4cac142ff98c75bfd637b21af" } modify_external_catalogs: true
W20260811 18:57:41.119937 3824 leader_election.cc:341] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:41.120177 3824 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.120416 5630 raft_consensus.cc:2781] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.120493 5630 raft_consensus.cc:482] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.120563 5630 raft_consensus.cc:3037] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.123310 5630 raft_consensus.cc:504] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.123991 5630 leader_election.cc:288] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.128211 3548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_b149c08055014e08aad5ad5be387047b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.128815 3548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b149c08055014e08aad5ad5be387047b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.129477 3546 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (table default.Table_6c8484897dd046ab8499d1069356cdcd [id=221d836949c341a09235288dc5802137]). Expected version 3 got 0
I20260811 18:57:41.129726 3546 catalog_manager.cc:5495] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65). New cstate: current_term: 1 leader_uuid: "bd8c4bdbd811417ab011a7131f17d283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:41.144066 5630 raft_consensus.cc:482] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.144272 5630 raft_consensus.cc:504] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.144912 5630 leader_election.cc:288] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.148226 3553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_9c66e99a00ba4953b1faed44821e14ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.148929 3553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c66e99a00ba4953b1faed44821e14ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.150770 3549 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_aadee0d920fc4be5aa7f965bc9db93e4" } new_table_name: "default.Table_1a248177f5ce4689a09bdb2d292adae1" modify_external_catalogs: true
I20260811 18:57:41.151306 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 76e71af5ee8b463eb158ac9ae76ff252
I20260811 18:57:41.152717 3552 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_1612fbcb897947ee843430ed61c2e50e" } new_table_name: "default.Table_cef814687d014504b5a7a569342e4bb4" modify_external_catalogs: true
I20260811 18:57:41.154788 5630 raft_consensus.cc:482] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.154959 5630 raft_consensus.cc:504] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.155576 5630 leader_election.cc:288] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.169507 4162 tablet_bootstrap.cc:492] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.170500 4162 ts_tablet_manager.cc:1397] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.212s user 0.004s sys 0.000s
I20260811 18:57:41.171536 4162 raft_consensus.cc:348] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.172752 4162 raft_consensus.cc:374] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.173117 4162 raft_consensus.cc:729] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.158317 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1037) took 5220 ms. Trace:
I20260811 18:57:41.175781 4169 rpcz_store.cc:276] 0811 18:57:35.938786 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:35.939004 (+ 218us) service_pool.cc:227] Handling call
0811 18:57:40.699843 (+4760839us) ts_tablet_manager.cc:1145] Deleting tablet 968fa8b8912b40a8bdba77db1d39c5c0
0811 18:57:40.699855 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:40.932192 (+232337us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.085957 (+153765us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.158287 (+ 72330us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70432,"tablet-delete.queue_time_us":4760561}
I20260811 18:57:41.178586 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1049) took 5044 ms. Trace:
I20260811 18:57:41.180433 4169 rpcz_store.cc:276] 0811 18:57:36.134037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.134647 (+ 610us) service_pool.cc:227] Handling call
0811 18:57:41.178416 (+5043769us) ts_tablet_manager.cc:1145] Deleting tablet 1700ee40ac0d40198c5b8a161a90b6b6
0811 18:57:41.178433 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.178562 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5043504}
I20260811 18:57:41.182070 3361 master-stress-test.cc:580] Killed test thread 1/25
I20260811 18:57:41.182070 5742 raft_consensus.cc:482] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.182224 3361 master-stress-test.cc:578] Killing test thread 2/25
I20260811 18:57:41.182368 3361 master-stress-test.cc:580] Killed test thread 2/25
I20260811 18:57:41.182456 3361 master-stress-test.cc:578] Killing test thread 3/25
I20260811 18:57:41.182585 5742 raft_consensus.cc:504] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.184267 5742 leader_election.cc:288] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.183360 4162 consensus_queue.cc:260] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.186609 3678 leader_election.cc:334] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.175094 5630 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.179673 5593 consensus_queue.cc:1035] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.187844 5630 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.188462 5630 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.180065 3824 leader_election.cc:341] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: e18242f762d8407db0e2f8716890feb4
W20260811 18:57:41.189237 3824 leader_election.cc:334] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.191891 3715 tablet_service.cc:1467] Processing CreateTablet for tablet 0fde489a381549f394a78e98c8cade0f (DEFAULT_TABLE table=default.Table_8e26f75764214905965fa0f514cb0028 [id=3aa9faf1a865494e9e6b33917e0cd501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.192067 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 26f55fe6a9e840a4a5e7ba6bf727c779 (DEFAULT_TABLE table=default.Table_724b9320db2141f39b2511549a1aa8d4 [id=de7aea74dccc46f68bc416ad0f023c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.192525 3715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fde489a381549f394a78e98c8cade0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.192932 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 625daea5324b4494996a51f0f9e9bcb7 (DEFAULT_TABLE table=default.Table_a6af8048b8f14af899345d8213953792 [id=677b3b20925544f28c2fb9ae18878c69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.195566 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 625daea5324b4494996a51f0f9e9bcb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.196892 3361 master-stress-test.cc:580] Killed test thread 3/25
I20260811 18:57:41.197008 3361 master-stress-test.cc:578] Killing test thread 4/25
I20260811 18:57:41.199318 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f55fe6a9e840a4a5e7ba6bf727c779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.207298 4162 ts_tablet_manager.cc:1428] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.036s user 0.001s sys 0.001s
I20260811 18:57:41.210968 3718 tablet_service.cc:1467] Processing CreateTablet for tablet eea325ca357146fcaa408d87803da349 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.215816 3713 tablet_service.cc:1467] Processing CreateTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 (DEFAULT_TABLE table=default.Table_9042bc231f0841a4a370be5e6251dd6e [id=044362a3a169467396a6be8959d1d07b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.216890 3710 tablet_service.cc:1467] Processing CreateTablet for tablet 929586965ce6468c82950102cfed41da (DEFAULT_TABLE table=default.Table_bce1d6623de1467198e1dfaf763f08cf [id=a85df4c40f944b598dcd718a4e9a3a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.217549 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet e544b4b1592c4d51a2ed0d6461edc75e with delete_type TABLET_DATA_DELETED (replaced by tablet a07f682b68114c2b9cb0df27a94406ec) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.218809 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet ae96e10a2ed4407c9d42a359d82d389e
I20260811 18:57:41.221726 3547 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38308:
name: "default.Table_bef031fd83e44076810e019e0038f70f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:41.222424 3547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bef031fd83e44076810e019e0038f70f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:41.223732 3551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_4a65337aa3b94589a51dd41812870aca" } new_table_name: "default.Table_73a91eab61b94f63a4d5123a9e446693" modify_external_catalogs: true
I20260811 18:57:41.224882 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cef27540ff46aebc9b6f4760ae0db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.225129 3555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c12987db235f4f4fb949580479275ec2" } modify_external_catalogs: true
I20260811 18:57:41.225312 3555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_c12987db235f4f4fb949580479275ec2" } modify_external_catalogs: true
I20260811 18:57:41.225932 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet bc38e6760754449fbd6209d765cce70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.226781 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet ab03996c5c624e5eb17267b124e27f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.227322 3548 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_61e3f65a0ee948fab69aa4add687f2bc" } new_table_name: "default.Table_c8f1686d290b4930adcdda63173c77bc" modify_external_catalogs: true
I20260811 18:57:41.227797 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 441feddae43741678ef0f64888b5c6f3 (DEFAULT_TABLE table=default.Table_b081a3f48c9048ecb88f1bd4571f934b [id=54b7f379f89e4c009585c94a5f7e1a55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.228272 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 6688ed13d1334ed98503b21a937facd2 (DEFAULT_TABLE table=default.Table_644eac4afde147dea1b6dc8cd69dc719 [id=583ba00a08184bc38ff31f87596e3788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.228773 3550 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38308:
table { table_name: "default.Table_078a10661db44f05a3516fe2dd35d154" } new_table_name: "default.Table_d82e6143f45747ddafae8a1e2651b9a0" modify_external_catalogs: true
I20260811 18:57:41.229921 3706 tablet_service.cc:1467] Processing CreateTablet for tablet 5d1b94cb25b84ae1a4c22801b1a98094 (DEFAULT_TABLE table=default.Table_04519ba2021244b6922bea0e5b79ecb6 [id=02744826873c4079a8d983c8e0d71c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.229983 3721 tablet_service.cc:1467] Processing CreateTablet for tablet ab9fe26e8e1e4c3886af83ac3e6956f5 (DEFAULT_TABLE table=default.Table_7e6c466c96e8465c9677dcaddee8819a [id=684f7399ba074fc5bbea2769d06cc682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.183887 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b203df4124364737839bf4c1ae072988" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.230515 3719 tablet_service.cc:1467] Processing CreateTablet for tablet b9fbac57c9c44a028f5b258593aff86d (DEFAULT_TABLE table=default.Table_51582df654e7406dbc574ba8de215baf [id=0eb328384d824024bcf7782bc4cf5b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.230947 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.231005 3676 leader_election.cc:341] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.231161 3676 leader_election.cc:302] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.231601 3824 leader_election.cc:302] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.231633 3708 tablet_service.cc:1514] Processing DeleteTablet for tablet f389d2b7ff354ac4bab5d68b9ff201c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.232051 3824 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.232498 3708 tablet_service.cc:1467] Processing CreateTablet for tablet b3ef682132a04b338b03a4387999d6b9 (DEFAULT_TABLE table=default.Table_4fd6bf06dd964e719ce1470aec9f60bc [id=54c1a4605328449dbb0a4928e192e271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.232781 5630 raft_consensus.cc:2726] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.232841 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.233352 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.233830 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.167547 4163 tablet_bootstrap.cc:492] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.235615 4163 ts_tablet_manager.cc:1397] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.166s user 0.004s sys 0.000s
I20260811 18:57:41.236449 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.236867 4163 raft_consensus.cc:348] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.237188 4163 raft_consensus.cc:374] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.237319 4163 raft_consensus.cc:729] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.237613 4163 consensus_queue.cc:260] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.238698 4163 ts_tablet_manager.cc:1428] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:41.239214 5742 raft_consensus.cc:2726] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.190683 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1051) took 5055 ms. Trace:
I20260811 18:57:41.206441 5593 raft_consensus.cc:482] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.239630 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1fc828f73642c88171d6ac827de90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.239591 5593 raft_consensus.cc:504] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.240204 5593 leader_election.cc:288] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.240304 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet b9fbac57c9c44a028f5b258593aff86d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.240463 4169 rpcz_store.cc:276] 0811 18:57:36.134871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.134917 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:41.190523 (+5055606us) ts_tablet_manager.cc:1145] Deleting tablet 38a4783168554053aa953dba5e893b6b
0811 18:57:41.190538 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.190664 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5055422}
I20260811 18:57:41.242857 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet d65cb8d901c04d8ba2b7a40efc7fa3af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.243453 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.245362 3553 master_service.cc:946] ReplaceTablet: received request to replace tablet 58bc967770de4ed486b638fb6b174db8 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.245787 5642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: acquiring CA information for follower catalog manager: success
I20260811 18:57:41.247128 3361 master-stress-test.cc:580] Killed test thread 4/25
I20260811 18:57:41.247304 3361 master-stress-test.cc:578] Killing test thread 5/25
I20260811 18:57:41.247270 3716 tablet_service.cc:1514] Processing DeleteTablet for tablet fdab1c3c431c4dceb08b00eabcfa1ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.247396 3361 master-stress-test.cc:580] Killed test thread 5/25
I20260811 18:57:41.247473 3361 master-stress-test.cc:578] Killing test thread 6/25
I20260811 18:57:41.249136 4163 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.251129 4163 tablet_bootstrap.cc:654] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.252120 3549 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ccf89d0b4e84205ad1d02c7c985a9d3 consensus state has no leader
I20260811 18:57:41.252399 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet 54fa32f8fa1144c4bd9dcd00a550001a from {username='slave'} at 127.0.0.1:38308
W20260811 18:57:41.254979 3552 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c575ba3060a140759484bea1462ffd73 has no consensus state
I20260811 18:57:41.255266 3552 master_service.cc:946] ReplaceTablet: received request to replace tablet 1f9621bbd0f94223a5d36bc01487f615 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.255551 3740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c29af3615e74722a809d3e5ede9b9a1" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.255795 3740 raft_consensus.cc:3037] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.255826 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 968fa8b8912b40a8bdba77db1d39c5c0 (table default.Table_1074819ead5e452d8a479310453e4b9a [id=ba2bd0b04e9d4bf1b6640d3dd63e7562]) successfully deleted
I20260811 18:57:41.257097 3739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.257476 3739 raft_consensus.cc:2454] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:41.260984 3740 raft_consensus.cc:2454] T 2c29af3615e74722a809d3e5ede9b9a1 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.264325 3546 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_c33b8642f3da4f97830feaf6f6a67ea8 [id=5a23d06641cb447fb634032bd768498b] alter complete (version 3)
W20260811 18:57:41.275591 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1700ee40ac0d40198c5b8a161a90b6b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1700ee40ac0d40198c5b8a161a90b6b6 already in progress: creating tablet
W20260811 18:57:41.275913 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 38a4783168554053aa953dba5e893b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38a4783168554053aa953dba5e893b6b already in progress: creating tablet
I20260811 18:57:41.277542 3549 master_service.cc:946] ReplaceTablet: received request to replace tablet eea325ca357146fcaa408d87803da349 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.278021 3361 master-stress-test.cc:580] Killed test thread 6/25
I20260811 18:57:41.278196 3361 master-stress-test.cc:578] Killing test thread 7/25
I20260811 18:57:41.278307 3361 master-stress-test.cc:580] Killed test thread 7/25
I20260811 18:57:41.278424 3361 master-stress-test.cc:578] Killing test thread 8/25
I20260811 18:57:41.278527 3361 master-stress-test.cc:580] Killed test thread 8/25
I20260811 18:57:41.278635 3361 master-stress-test.cc:578] Killing test thread 9/25
I20260811 18:57:41.278730 3361 master-stress-test.cc:580] Killed test thread 9/25
I20260811 18:57:41.278808 3361 master-stress-test.cc:578] Killing test thread 10/25
I20260811 18:57:41.278889 3361 master-stress-test.cc:580] Killed test thread 10/25
I20260811 18:57:41.278972 3361 master-stress-test.cc:578] Killing test thread 11/25
W20260811 18:57:41.282564 3551 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a62cdd333ad8494897ae8ec5afd5af74 consensus state has no leader
I20260811 18:57:41.283800 3361 master-stress-test.cc:580] Killed test thread 11/25
I20260811 18:57:41.283941 3361 master-stress-test.cc:578] Killing test thread 12/25
I20260811 18:57:41.284019 3361 master-stress-test.cc:580] Killed test thread 12/25
I20260811 18:57:41.284081 3361 master-stress-test.cc:578] Killing test thread 13/25
I20260811 18:57:41.284183 3361 master-stress-test.cc:580] Killed test thread 13/25
I20260811 18:57:41.284260 3361 master-stress-test.cc:578] Killing test thread 14/25
I20260811 18:57:41.253858 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1052) took 5109 ms. Trace:
I20260811 18:57:41.284504 4169 rpcz_store.cc:276] 0811 18:57:36.144056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.308615 (+164559us) service_pool.cc:227] Handling call
0811 18:57:41.240865 (+4932250us) ts_tablet_manager.cc:1145] Deleting tablet 374c94a12e374d0e82180421c80d92d0
0811 18:57:41.253686 (+ 12821us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.253831 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4931905}
I20260811 18:57:41.285113 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1062) took 4935 ms. Trace:
I20260811 18:57:41.285225 4169 rpcz_store.cc:276] 0811 18:57:36.349751 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.349811 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:41.284939 (+4935128us) ts_tablet_manager.cc:1145] Deleting tablet 60091bd891a94cc4aac6ef47673a4ee5
0811 18:57:41.284958 (+ 19us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.285090 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4932595}
I20260811 18:57:41.285635 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1063) took 4935 ms. Trace:
I20260811 18:57:41.285710 4169 rpcz_store.cc:276] 0811 18:57:36.349867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.351599 (+ 1732us) service_pool.cc:227] Handling call
0811 18:57:41.285496 (+4933897us) ts_tablet_manager.cc:1145] Deleting tablet fe080f2071964da6b54f1ee5af98d918
0811 18:57:41.285510 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.285619 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4930726}
I20260811 18:57:41.285996 4169 tablet_replica.cc:330] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.286212 4169 raft_consensus.cc:2229] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.258644 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.274540 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet ae96e10a2ed4407c9d42a359d82d389e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.286577 4169 raft_consensus.cc:2258] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.287227 4169 ts_tablet_manager.cc:1905] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.291980 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 73f05e1d5cb947fe9dbcc2e876a23788 (table default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba]). Expected version 2 got 0
I20260811 18:57:41.292444 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 4f3c4536fc7440db83851fc8d7b3de10 (table default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187]). Expected version 2 got 0
I20260811 18:57:41.292739 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:41.293010 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 0bd39dcc0bed4623a06f00a56b740dba (table default.Table_28ed0eb238674836950a5639ebcbc6b9 [id=f312f8bcd459400b92a33532675d3991]). Expected version 2 got 0
I20260811 18:57:41.293577 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:41.294140 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 452ee450778e4d33adfa7e2745b5d970 (table default.Table_4d6c4d770dc949e78c1707ddbab71740 [id=48f45fcc06e0418185841260a06cb98c]) successfully deleted
I20260811 18:57:41.295084 3555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet dfb6ac49f1d4403dad38a3125264088a
W20260811 18:57:41.296679 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 03c50dcffac54c7bac210eeaed340ca6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03c50dcffac54c7bac210eeaed340ca6 already in progress: creating tablet
W20260811 18:57:41.297286 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 73c519528f75423195b6bdf6683fd91b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c519528f75423195b6bdf6683fd91b already in progress: creating tablet
W20260811 18:57:41.297538 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b already in progress: creating tablet
W20260811 18:57:41.282248 3824 leader_election.cc:334] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.298501 3824 leader_election.cc:302] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.298516 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 3ad5f39f59b8447cb3f15b9646e4b852 (table default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]) successfully deleted
W20260811 18:57:41.298753 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7cb3e997b4454598bdce102e92296887 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7cb3e997b4454598bdce102e92296887 already in progress: creating tablet
I20260811 18:57:41.298817 5593 raft_consensus.cc:2726] T e18242f762d8407db0e2f8716890feb4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.299037 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 8f9b279c6c7344f1931b81e10248b1b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f9b279c6c7344f1931b81e10248b1b6 already in progress: creating tablet
W20260811 18:57:41.299230 3824 consensus_peers.cc:487] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:41.299353 3824 consensus_peers.cc:487] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:41.299360 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ea579c33f8b14e9fa7fa389ce7f45931 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea579c33f8b14e9fa7fa389ce7f45931 already in progress: creating tablet
W20260811 18:57:41.299546 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 7f4e1fc5686944dfa3eb57702c810eb8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f4e1fc5686944dfa3eb57702c810eb8 already in progress: creating tablet
W20260811 18:57:41.299759 3548 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1474691d8bec4698981ff71ef23ff857 consensus state has no leader
I20260811 18:57:41.300526 3824 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.300828 5593 raft_consensus.cc:2781] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.300910 5593 raft_consensus.cc:482] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.300974 5593 raft_consensus.cc:3037] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.300978 3548 master_service.cc:946] ReplaceTablet: received request to replace tablet c44eee516d634d9a986107b55410dfc3 from {username='slave'} at 127.0.0.1:38308
I20260811 18:57:41.303869 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea325ca357146fcaa408d87803da349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.304107 5593 raft_consensus.cc:504] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.304764 5593 leader_election.cc:288] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.306891 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929586965ce6468c82950102cfed41da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307292 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 441feddae43741678ef0f64888b5c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307593 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6688ed13d1334ed98503b21a937facd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.307842 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9fe26e8e1e4c3886af83ac3e6956f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.308038 5757 tablet_replica.cc:330] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.308315 5757 raft_consensus.cc:2229] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:41.308714 5757 raft_consensus.cc:2258] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.309324 5757 ts_tablet_manager.cc:1905] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.308059 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d1b94cb25b84ae1a4c22801b1a98094. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.310448 3550 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45e040f8120a4edfba89c1a8d55deb4a has no consensus state
I20260811 18:57:41.314510 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ef682132a04b338b03a4387999d6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.317816 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cef27540ff46aebc9b6f4760ae0db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.318210 4169 ts_tablet_manager.cc:1918] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.322516 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9fbac57c9c44a028f5b258593aff86d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.321542 5742 consensus_queue.cc:1035] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.324677 3361 master-stress-test.cc:580] Killed test thread 14/25
I20260811 18:57:41.324790 3361 master-stress-test.cc:578] Killing test thread 15/25
I20260811 18:57:41.324860 3361 master-stress-test.cc:580] Killed test thread 15/25
I20260811 18:57:41.324921 3361 master-stress-test.cc:578] Killing test thread 16/25
I20260811 18:57:41.324988 3361 master-stress-test.cc:580] Killed test thread 16/25
I20260811 18:57:41.325062 3361 master-stress-test.cc:578] Killing test thread 17/25
I20260811 18:57:41.325133 3361 master-stress-test.cc:580] Killed test thread 17/25
I20260811 18:57:41.325194 3361 master-stress-test.cc:578] Killing test thread 18/25
I20260811 18:57:41.325253 3361 master-stress-test.cc:580] Killed test thread 18/25
I20260811 18:57:41.325316 3361 master-stress-test.cc:578] Killing test thread 19/25
I20260811 18:57:41.325389 3361 master-stress-test.cc:580] Killed test thread 19/25
I20260811 18:57:41.325449 3361 master-stress-test.cc:578] Killing test thread 20/25
I20260811 18:57:41.325511 3361 master-stress-test.cc:580] Killed test thread 20/25
I20260811 18:57:41.325570 3361 master-stress-test.cc:578] Killing test thread 21/25
I20260811 18:57:41.325632 3361 master-stress-test.cc:580] Killed test thread 21/25
I20260811 18:57:41.325693 3361 master-stress-test.cc:578] Killing test thread 22/25
I20260811 18:57:41.327144 4162 tablet_bootstrap.cc:492] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.327888 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 2 got 0
I20260811 18:57:41.337205 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 58bc967770de4ed486b638fb6b174db8
I20260811 18:57:41.337733 3553 catalog_manager.cc:6565] ReplaceTablet: tablet 58bc967770de4ed486b638fb6b174db8 deleted and replaced by tablet f336fdd55965432f83b2d1c436fdf1fa
I20260811 18:57:41.343441 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet eea325ca357146fcaa408d87803da349
I20260811 18:57:41.343839 3549 catalog_manager.cc:6565] ReplaceTablet: tablet eea325ca357146fcaa408d87803da349 deleted and replaced by tablet a4e0b84bdb134a459d2d9afd046326c2
I20260811 18:57:41.344518 3548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet c44eee516d634d9a986107b55410dfc3
I20260811 18:57:41.345045 3548 catalog_manager.cc:6565] ReplaceTablet: tablet c44eee516d634d9a986107b55410dfc3 deleted and replaced by tablet 5faabb319fc84f24ab548a707be284f1
I20260811 18:57:41.345803 3361 master-stress-test.cc:580] Killed test thread 22/25
I20260811 18:57:41.345944 3361 master-stress-test.cc:578] Killing test thread 23/25
I20260811 18:57:41.341475 4162 tablet_bootstrap.cc:654] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.299768 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d21973bc8e514a089c2beb24a686060f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d21973bc8e514a089c2beb24a686060f already in progress: creating tablet
W20260811 18:57:41.348656 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 48896us
W20260811 18:57:41.348932 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 2f2f5ce2b1d14a9a988cff644e196088 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f2f5ce2b1d14a9a988cff644e196088 already in progress: creating tablet
W20260811 18:57:41.349256 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f2b66196716a4ba3893b6b8bfec2b8c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2b66196716a4ba3893b6b8bfec2b8c0 already in progress: creating tablet
W20260811 18:57:41.349512 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a3973dfdaac746a6af7a402277d2711b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3973dfdaac746a6af7a402277d2711b already in progress: creating tablet
W20260811 18:57:41.349730 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 15dd14e2b4bd4a08b5b08f532a55291b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15dd14e2b4bd4a08b5b08f532a55291b already in progress: creating tablet
W20260811 18:57:41.349941 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 75ceba7bfc8240268b16c789f07813d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75ceba7bfc8240268b16c789f07813d1 already in progress: creating tablet
W20260811 18:57:41.350165 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0495c5b770b34d54afb1379a6e36204c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0495c5b770b34d54afb1379a6e36204c already in progress: creating tablet
W20260811 18:57:41.350411 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5a75a480f38047d880b0c1fa51403786 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a75a480f38047d880b0c1fa51403786 already in progress: creating tablet
W20260811 18:57:41.350641 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d32212c817c94fad8379462b027e7d3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32212c817c94fad8379462b027e7d3d already in progress: creating tablet
I20260811 18:57:41.350899 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e18242f762d8407db0e2f8716890feb4 (table default.Table_4981573f7a434b5da77a06d251ea684c [id=9ac3355276b6491b84bbc7d30235db66]) successfully deleted
W20260811 18:57:41.351626 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 3195082643264e0d90f70e1007ab9b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3195082643264e0d90f70e1007ab9b24 already in progress: creating tablet
W20260811 18:57:41.352082 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 429eb5824f3f4b2b8da6e17ba7c69b92 already in progress: creating tablet
W20260811 18:57:41.352625 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f89eebbc40234794a081734f834294ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89eebbc40234794a081734f834294ab already in progress: creating tablet
W20260811 18:57:41.353009 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a5bbd3274980470b9fe71457b49bdd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5bbd3274980470b9fe71457b49bdd62 already in progress: creating tablet
W20260811 18:57:41.353428 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b7b38d3b9570467a9128bd7fe73acf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b38d3b9570467a9128bd7fe73acf2f already in progress: creating tablet
I20260811 18:57:41.353557 3361 master-stress-test.cc:580] Killed test thread 23/25
I20260811 18:57:41.353699 3361 master-stress-test.cc:578] Killing test thread 24/25
I20260811 18:57:41.353888 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 42405aec21e3405486b6c658438b5fb2 (table default.Table_7d0c2a4e217c4217aa3fe24ec0738635 [id=3e392b51ee7048499231c67cbb3f5a08]) successfully deleted
I20260811 18:57:41.343853 5757 ts_tablet_manager.cc:1918] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:41.354452 5757 log.cc:1196] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:41.354527 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet eb8f5b87062349c4b19ec99fa0175f24 (table default.Table_9f42592ab70540e59f42f7e97293a058 [id=9a1945e28f2a45ffa40293f6c8ceb9e0]) successfully deleted
I20260811 18:57:41.354931 5757 ts_tablet_manager.cc:1939] T e544b4b1592c4d51a2ed0d6461edc75e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.355628 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet f75dc515d7f3473b8e43f20255a70e3e (table default.Table_b717724905d74a6a865cd6eb35c3a4d8 [id=18dc0706e9a141aca738f8dd078c4793]) successfully deleted
I20260811 18:57:41.357508 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet d37e81d376e14907b13746cca831a96b (table default.Table_5054093f72194ee8a604b8aa7fb6cf67 [id=ea12d5afbba64e85b677dac3bc5fb285]) successfully deleted
I20260811 18:57:41.359300 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet bfea0cd619024204b9bff1e2dd62b7b0 (table default.Table_911b59bd767843d39233f5c65932fb5d [id=17fdd095e10d41fe8033aad4d1034af5]) successfully deleted
I20260811 18:57:41.359905 5757 tablet_replica.cc:330] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
W20260811 18:57:41.360028 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 69059a0667234a408cc1c09585df63bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69059a0667234a408cc1c09585df63bc already in progress: creating tablet
W20260811 18:57:41.360679 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 82b04e37fc264fc08e0ed9517356a17d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82b04e37fc264fc08e0ed9517356a17d already in progress: creating tablet
W20260811 18:57:41.361255 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d11b889e9d044a049d0586382c263951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d11b889e9d044a049d0586382c263951 already in progress: creating tablet
W20260811 18:57:41.361763 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet e9f0e443ab84424dac307b745b496986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9f0e443ab84424dac307b745b496986 already in progress: creating tablet
W20260811 18:57:41.362231 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 6710084f21734dea9c7ce1f43d1bf587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6710084f21734dea9c7ce1f43d1bf587 already in progress: creating tablet
I20260811 18:57:41.362717 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]) successfully deleted
I20260811 18:57:41.362977 5757 raft_consensus.cc:2229] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:41.363250 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 31bb11617f1548f88f534289706edb06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31bb11617f1548f88f534289706edb06 already in progress: creating tablet
I20260811 18:57:41.363519 5757 raft_consensus.cc:2258] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:41.363708 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 76506ad179a9408c9e4cf8e4275ebddd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76506ad179a9408c9e4cf8e4275ebddd already in progress: creating tablet
I20260811 18:57:41.364215 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8304ac7709ca4392baec4cfa78ef5b41 (table default.Table_6f62a01a774b4295a6dc3352d22d1ba4 [id=3ecfeda772b048aeaa77983d91634bd3]) successfully deleted
I20260811 18:57:41.364270 5757 ts_tablet_manager.cc:1905] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:41.364722 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eb43127dda814a718b338906fc2a3c47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb43127dda814a718b338906fc2a3c47 already in progress: creating tablet
I20260811 18:57:41.364912 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b4dc2df4c6464d71bb13fe25e0046758 (table default.Table_d84e9ae317424a768d54ab23dc68d514 [id=73a97b63e5db407a94ff739f295cc859]) successfully deleted
W20260811 18:57:41.365109 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ee5b7d1e32514ad48177371582e85241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5b7d1e32514ad48177371582e85241 already in progress: creating tablet
W20260811 18:57:41.365285 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4451cc4d7c5940cdb41c021ae486a5a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4451cc4d7c5940cdb41c021ae486a5a9 already in progress: creating tablet
I20260811 18:57:41.365433 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ffa8a4a1c0cb4834be501bdd7d468472 (table default.Table_a15aed3f339749a7ab05c1774bc21112 [id=dac94de26b1c4d3fbef5a6b71f7e70e1]) successfully deleted
W20260811 18:57:41.365607 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c590c98e39f84ea09b26bbe8640ca93d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c590c98e39f84ea09b26bbe8640ca93d already in progress: creating tablet
W20260811 18:57:41.365782 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet cb54beb34a654fbdb9c7632901bb0b9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb54beb34a654fbdb9c7632901bb0b9e already in progress: creating tablet
W20260811 18:57:41.367524 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 09366eccf38a4876840cc5ddddbbb103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09366eccf38a4876840cc5ddddbbb103 already in progress: creating tablet
W20260811 18:57:41.367985 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 78868ff1f8024163ae1d782151100cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78868ff1f8024163ae1d782151100cd4 already in progress: creating tablet
W20260811 18:57:41.368688 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 1ccd56a8b7cc454ea3f2f08b59374521 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ccd56a8b7cc454ea3f2f08b59374521 already in progress: creating tablet
W20260811 18:57:41.369079 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 already in progress: creating tablet
W20260811 18:57:41.369798 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fa95131ce50943e3b0c6b5bdd5a30939 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa95131ce50943e3b0c6b5bdd5a30939 already in progress: creating tablet
I20260811 18:57:41.372710 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]) successfully deleted
W20260811 18:57:41.373425 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f9cef27540ff46aebc9b6f4760ae0db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9cef27540ff46aebc9b6f4760ae0db4 already in progress: creating tablet
W20260811 18:57:41.374121 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bc38e6760754449fbd6209d765cce70c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc38e6760754449fbd6209d765cce70c already in progress: creating tablet
W20260811 18:57:41.374475 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ab03996c5c624e5eb17267b124e27f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab03996c5c624e5eb17267b124e27f45 already in progress: creating tablet
I20260811 18:57:41.378588 3713 tablet_service.cc:1467] Processing CreateTablet for tablet c82456373d974f35846ade45e2398bbd (DEFAULT_TABLE table=default.Table_4b90b4ead71940e58e93af4a9804a7d8 [id=ac620652bd4b404da86f7dda6ed307d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.379074 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82456373d974f35846ade45e2398bbd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.383009 3678 consensus_peers.cc:487] T 473040a90e6941c38beb697316d2a27b P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:41.392637 5407 raft_consensus.cc:482] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.392809 5407 raft_consensus.cc:504] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.393420 5407 leader_election.cc:288] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.394935 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
W20260811 18:57:41.395627 3678 leader_election.cc:334] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.333110 4163 tablet_bootstrap.cc:492] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.402712 4163 ts_tablet_manager.cc:1397] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.154s user 0.004s sys 0.000s
I20260811 18:57:41.402885 5757 ts_tablet_manager.cc:1918] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.403004 5757 log.cc:1196] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/38e24f8d4522466d81605ea37fa7064a
I20260811 18:57:41.403311 5757 ts_tablet_manager.cc:1939] T 38e24f8d4522466d81605ea37fa7064a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.403658 4163 raft_consensus.cc:348] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.403956 4163 raft_consensus.cc:374] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.404022 4163 raft_consensus.cc:729] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.404301 4163 consensus_queue.cc:260] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.405100 4163 ts_tablet_manager.cc:1428] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:41.405491 4163 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:41.319404 3824 leader_election.cc:341] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 19243ab3d6744aaa87a133f1462cd2fa
W20260811 18:57:41.407188 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 87794.3us
W20260811 18:57:41.407471 3824 leader_election.cc:341] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:41.409142 3824 leader_election.cc:302] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.409271 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:41.421993 3713 tablet_service.cc:1467] Processing CreateTablet for tablet defce8dba232424ab74ebae42d6a078a (DEFAULT_TABLE table=default.Table_7a85117c40464174beed058911301729 [id=728fa9eb1e2947c8994d451b9a80eaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.423180 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e556811ad1643f1baa3bbf4edebd157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.423386 4162 tablet_bootstrap.cc:492] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.423517 4162 ts_tablet_manager.cc:1397] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.096s user 0.011s sys 0.002s
I20260811 18:57:41.423591 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 9d09471428a5403da8b3bbf0f1597ed3 (DEFAULT_TABLE table=default.Table_e894321d446d4d6da9e866788ff2b1e7 [id=fd4705a377a94d3eb676f4dd0f37e096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.423964 5593 raft_consensus.cc:2781] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.424111 5593 raft_consensus.cc:686] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.424294 4162 raft_consensus.cc:348] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.424357 5593 consensus_queue.cc:237] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.424746 4162 raft_consensus.cc:374] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.424861 4162 raft_consensus.cc:729] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.425129 4162 consensus_queue.cc:260] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.425740 4162 ts_tablet_manager.cc:1428] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:41.417482 4163 tablet_bootstrap.cc:654] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.394966 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b2ca385374abeb709fccbdf6fa0ba" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.427229 3676 leader_election.cc:341] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.427353 3676 leader_election.cc:302] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.427747 5407 raft_consensus.cc:2726] T 6f6b2ca385374abeb709fccbdf6fa0ba P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.420962 5764 tablet.cc:1717] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24: Skipping requested alter to schema version 3, tablet already version 3
I20260811 18:57:41.428687 5764 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37348 (request call id 1241) took 1144 ms. Trace:
I20260811 18:57:41.428810 5764 rpcz_store.cc:276] 0811 18:57:40.283830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.286527 (+ 2697us) service_pool.cc:227] Handling call
0811 18:57:41.428669 (+1142142us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0811 18:57:40.286967 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0811 18:57:40.287071 (+ 104us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0811 18:57:40.803344 (+516273us) alter_schema_op.cc:65] Acquired schema lock
0811 18:57:40.803354 (+ 10us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0811 18:57:40.803405 (+ 51us) alter_schema_op.cc:126] START. Timestamp: 1786474660803359
0811 18:57:40.803408 (+ 3us) op_driver.cc:331] REPLICATION: starting
0811 18:57:40.803666 (+ 258us) log.cc:842] Serialized 173 byte log entry
0811 18:57:41.408587 (+604921us) op_driver.cc:444] REPLICATION: finished
0811 18:57:41.420918 (+ 12331us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0811 18:57:41.428355 (+ 7437us) log.cc:842] Serialized 94 byte log entry
0811 18:57:41.428614 (+ 259us) alter_schema_op.cc:71] Released schema lock
0811 18:57:41.428630 (+ 16us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12235,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":18090,"prepare.run_wall_time_us":517205,"raft.queue_time_us":107,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":5,"replication_time_us":605148,"thread_start_us":12637,"threads_started":1}]]}
I20260811 18:57:41.430613 4162 tablet_bootstrap.cc:492] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:41.431324 3824 leader_election.cc:341] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:41.431938 5757 tablet_replica.cc:330] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.432231 5757 raft_consensus.cc:2229] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.432801 5757 raft_consensus.cc:2258] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.434134 5757 ts_tablet_manager.cc:1905] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.437953 5764 tablet.cc:1722] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.446206 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bfb54175c834517b7af291c942333cd with delete_type TABLET_DATA_DELETED (replaced by tablet 13af4d419ae542cc961c75f28efd0204) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.446537 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defce8dba232424ab74ebae42d6a078a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.446800 4163 tablet_bootstrap.cc:492] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.446842 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb6ac49f1d4403dad38a3125264088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447261 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 58bc967770de4ed486b638fb6b174db8 with delete_type TABLET_DATA_DELETED (replaced by tablet f336fdd55965432f83b2d1c436fdf1fa) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447288 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d09471428a5403da8b3bbf0f1597ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.448920 4162 tablet_bootstrap.cc:654] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.449748 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa8a4a1c0cb4834be501bdd7d468472 with delete_type TABLET_DATA_DELETED (replaced by tablet 07a78361bbc44fcf86985f9f20c5665b) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.450361 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet eea325ca357146fcaa408d87803da349 with delete_type TABLET_DATA_DELETED (replaced by tablet a4e0b84bdb134a459d2d9afd046326c2) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.450780 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet c44eee516d634d9a986107b55410dfc3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5faabb319fc84f24ab548a707be284f1) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.451131 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet f75dc515d7f3473b8e43f20255a70e3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:41.447108 4163 ts_tablet_manager.cc:1397] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.006s
I20260811 18:57:41.453941 4163 raft_consensus.cc:348] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.460562 5757 ts_tablet_manager.cc:1918] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.463590 4163 raft_consensus.cc:374] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.463793 4163 raft_consensus.cc:729] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.464085 4163 consensus_queue.cc:260] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.465029 4163 ts_tablet_manager.cc:1428] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260811 18:57:41.465390 4163 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.462316 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 with delete_type TABLET_DATA_DELETED (replaced by tablet fe48bbab3d7e4f7591883296a9996db7) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.466720 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 8794dcb46ccc442e9271fc33453e4800 (DEFAULT_TABLE table=default.Table_f614f2bd34c949cd94f9493a086dd8c7 [id=9b6853170f7b4ebb9c266f96c7d99e10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.466835 5757 log.cc:1196] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/9ec4bba841dd446aa3db751358ce7f32
I20260811 18:57:41.467163 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8794dcb46ccc442e9271fc33453e4800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.467248 5757 ts_tablet_manager.cc:1939] T 9ec4bba841dd446aa3db751358ce7f32 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.467324 4163 tablet_bootstrap.cc:654] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.464483 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d93073a69c143ff9460248848a108d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:39 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.318331 4169 log.cc:1196] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/fc3f120758c6405dafd2ea0bf54c81d8
I20260811 18:57:41.470433 4169 ts_tablet_manager.cc:1939] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.471045 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1064) took 5119 ms. Trace:
I20260811 18:57:41.471242 4169 rpcz_store.cc:276] 0811 18:57:36.351368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.353947 (+ 2579us) service_pool.cc:227] Handling call
0811 18:57:41.285949 (+4932002us) ts_tablet_manager.cc:1145] Deleting tablet fc3f120758c6405dafd2ea0bf54c81d8
0811 18:57:41.285960 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.315087 (+ 29127us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.318114 (+ 3027us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.471018 (+152904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4930927}
I20260811 18:57:41.471879 4169 tablet_replica.cc:330] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.472165 4169 raft_consensus.cc:2229] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.472385 4169 raft_consensus.cc:2258] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.473008 4169 ts_tablet_manager.cc:1905] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.474051 4162 tablet_bootstrap.cc:492] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.474267 4162 ts_tablet_manager.cc:1397] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.005s
W20260811 18:57:41.474706 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 374c94a12e374d0e82180421c80d92d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374c94a12e374d0e82180421c80d92d0 already in progress: creating tablet
W20260811 18:57:41.475030 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 60091bd891a94cc4aac6ef47673a4ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60091bd891a94cc4aac6ef47673a4ee5 already in progress: creating tablet
I20260811 18:57:41.475085 4162 raft_consensus.cc:348] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.475293 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fe080f2071964da6b54f1ee5af98d918 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe080f2071964da6b54f1ee5af98d918 already in progress: creating tablet
I20260811 18:57:41.475260 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b355a0d3984012be4e9406998fd4a0" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.475318 4162 raft_consensus.cc:374] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.475420 4162 raft_consensus.cc:729] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.475430 3741 raft_consensus.cc:3037] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.478925 3741 raft_consensus.cc:2454] T c0b355a0d3984012be4e9406998fd4a0 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
W20260811 18:57:41.478966 3824 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.479336 3824 leader_election.cc:334] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.479665 3824 leader_election.cc:302] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.479736 4163 tablet_bootstrap.cc:492] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.479938 4163 ts_tablet_manager.cc:1397] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.004s
I20260811 18:57:41.480773 4163 raft_consensus.cc:348] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481122 4163 raft_consensus.cc:374] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.481178 4163 raft_consensus.cc:729] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.475675 4162 consensus_queue.cc:260] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481369 4163 consensus_queue.cc:260] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.481925 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 56542f3afe304feb888f101961d855c7 (DEFAULT_TABLE table=default.Table_9919772109274064aee942f6a7f1220f [id=75a15771a07045039c110309e3dc0beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.482301 4162 ts_tablet_manager.cc:1428] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 18:57:41.482473 4163 ts_tablet_manager.cc:1428] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 18:57:41.482906 4163 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.482622 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56542f3afe304feb888f101961d855c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.483597 4162 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.485419 4162 tablet_bootstrap.cc:654] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.486012 4163 tablet_bootstrap.cc:654] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.485966 5757 tablet_replica.cc:330] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.492501 3718 tablet_service.cc:1467] Processing CreateTablet for tablet d8c6b62d2a684869b0ad31a1b25c7ebf (DEFAULT_TABLE table=default.Table_bf1e0bf055c44f45835974098ec141f8 [id=8f5e99a55df34267a0b32710772f1978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.492642 5757 raft_consensus.cc:2229] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.492810 5757 raft_consensus.cc:2258] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.492962 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c6b62d2a684869b0ad31a1b25c7ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.493182 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a54921b815f2408c96bdf18e243041d5 (DEFAULT_TABLE table=default.Table_23655b2975534b81b076fd031ea5022a [id=e040f0cd609644928bb9690a053fa426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.493433 5757 ts_tablet_manager.cc:1905] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.493544 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54921b815f2408c96bdf18e243041d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.503337 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 71a7eda2248c4f708bdc772ea3598dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.508503 4162 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.508844 4162 ts_tablet_manager.cc:1397] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20260811 18:57:41.509920 5407 raft_consensus.cc:482] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.510450 5407 raft_consensus.cc:504] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.511068 5407 leader_election.cc:288] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.512342 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 79539e937f81437ab3b12b8465f12d97) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.512969 3678 leader_election.cc:334] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.513145 4162 raft_consensus.cc:348] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.513484 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4059e5907b1e488ab9e1d04e563f96cd" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.513301 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.516253 4162 raft_consensus.cc:374] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 18:57:41.514181 3676 leader_election.cc:341] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.516459 3676 leader_election.cc:302] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.516867 5407 raft_consensus.cc:2726] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.513548 5772 tablet.cc:1717] T faf0594d55844ae49c99247ecd8b810f P bd8c4bdbd811417ab011a7131f17d283: Skipping requested alter to schema version 3, tablet already version 3
I20260811 18:57:41.517684 5772 tablet.cc:1722] T eb82243777a9494b96959ced0c24de5a P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.518299 4162 raft_consensus.cc:729] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.521845 3723 tablet_service.cc:1467] Processing CreateTablet for tablet f891108d44c44296934924941981b568 (DEFAULT_TABLE table=default.Table_4b752724648a499980ad71fd596a731f [id=78f3a4a7d6ad41278417b8eef697d0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.522296 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f891108d44c44296934924941981b568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.522225 4162 consensus_queue.cc:260] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.523173 4163 tablet_bootstrap.cc:492] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.523398 4163 ts_tablet_manager.cc:1397] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.004s
I20260811 18:57:41.524255 4163 raft_consensus.cc:348] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.524959 4162 ts_tablet_manager.cc:1428] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260811 18:57:41.527632 4162 tablet_bootstrap.cc:492] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.533804 4162 tablet_bootstrap.cc:654] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.527287 5757 ts_tablet_manager.cc:1918] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.537039 5757 log.cc:1196] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b20872d80c9545e5822bfc82d496ce97
I20260811 18:57:41.537508 5757 ts_tablet_manager.cc:1939] T b20872d80c9545e5822bfc82d496ce97 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.539764 5757 tablet_replica.cc:330] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.540766 5757 raft_consensus.cc:2229] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.541009 5757 raft_consensus.cc:2258] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.541628 5757 ts_tablet_manager.cc:1905] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.550498 4163 raft_consensus.cc:374] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.550733 4163 raft_consensus.cc:729] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.551133 4163 consensus_queue.cc:260] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.551921 4163 ts_tablet_manager.cc:1428] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260811 18:57:41.557602 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 0bac7ac254f4471397580dc510648dfc (DEFAULT_TABLE table=default.Table_69847be1563a4aad8d093d935864f531 [id=6a7e6156c0c247f68bcdb76ed1eb6365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.558785 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 756062a54d144339b91f8c68d9edceab (DEFAULT_TABLE table=default.Table_e142d545d8b249f3835bce3bbef9f94e [id=51b0aa68c2e749768c886732760d4f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.560889 3706 tablet_service.cc:1467] Processing CreateTablet for tablet ce6a80313c854c17913db64aeab9acf3 (DEFAULT_TABLE table=default.Table_c4b27a9eb0994efe9f4176f74c16a62e [id=6e29d4572cd64f8cb70940f4119296e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.528234 5593 raft_consensus.cc:2726] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.528033 3824 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.72.67:36027, user_credentials={real_user=slave}} blocked reactor thread for 48698.8us
I20260811 18:57:41.559700 5767 raft_consensus.cc:482] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.564857 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f11748cf984942ba99bda4ad6605ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.566761 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet b50efab3c782447cad8e45b1fa51aa1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.567451 3719 tablet_service.cc:1467] Processing CreateTablet for tablet 997be6173a194bbe91c6c0b9046d1a0e (DEFAULT_TABLE table=default.Table_2a4d5ad618dd4fb1a5bde684ecb0e2ae [id=7586915510704916afd712ab81f9f06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.567642 3723 tablet_service.cc:1467] Processing CreateTablet for tablet a07f682b68114c2b9cb0df27a94406ec (DEFAULT_TABLE table=default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.568830 3711 tablet_service.cc:1467] Processing CreateTablet for tablet 86f215c21d184dd6ba828fa31286cce1 (DEFAULT_TABLE table=default.Table_d8f35609e86b47aa9a8ab711834da08f [id=72dfc9b4df824dcd92241af7ab88c68e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.569764 3708 tablet_service.cc:1467] Processing CreateTablet for tablet bba6561b8c4b4c728db18446a9a91f30 (DEFAULT_TABLE table=default.Table_5affb8acc7e5466ba5ed988728ba50cb [id=0a26d099a1324a9aa8005962a8d9bd79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.570088 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 01fe8e896b2246fabd2b6fae396bf085 (DEFAULT_TABLE table=default.Table_f83d8784174f40e29b96eba2de396065 [id=fdb879cbb1cb4f498f94d14048b87e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.570789 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.571409 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1e244addf5462499fcadaa3332cba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:19 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.563108 5767 raft_consensus.cc:504] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.571957 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 756062a54d144339b91f8c68d9edceab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572158 3708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba6561b8c4b4c728db18446a9a91f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572296 5767 leader_election.cc:288] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.562652 4163 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.572569 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01fe8e896b2246fabd2b6fae396bf085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.572752 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 997be6173a194bbe91c6c0b9046d1a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.573048 3706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6a80313c854c17913db64aeab9acf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.573335 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet bf1c037a37504b3fad9612f5a734dd33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.573665 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a07f682b68114c2b9cb0df27a94406ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.563242 3824 leader_election.cc:334] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.574751 3824 leader_election.cc:302] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:41.574914 3824 raft_consensus.cc:2670] T fc3f120758c6405dafd2ea0bf54c81d8 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:41.575492 3824 leader_election.cc:334] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.575702 3824 leader_election.cc:302] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.575917 5767 raft_consensus.cc:2726] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.576198 3824 leader_election.cc:334] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.565064 5593 raft_consensus.cc:482] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd8c4bdbd811417ab011a7131f17d283)
I20260811 18:57:41.579388 5593 raft_consensus.cc:504] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } }
I20260811 18:57:41.579718 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet fc3f120758c6405dafd2ea0bf54c81d8 (table default.Table_b7f48893a8614cacb6393af837f050f3 [id=c98fce30aeef4de0ae0d79aa40ddd957]) successfully deleted
I20260811 18:57:41.580191 5593 leader_election.cc:288] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.580600 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bac7ac254f4471397580dc510648dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.581697 3824 leader_election.cc:302] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.581996 5767 raft_consensus.cc:2781] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.582140 5767 raft_consensus.cc:686] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.582521 5767 consensus_queue.cc:237] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.584715 5630 raft_consensus.cc:482] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.584887 5630 raft_consensus.cc:504] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.585530 5630 leader_election.cc:288] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.585644 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet a7946503bec9446d8deb02448eb231d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.586561 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f215c21d184dd6ba828fa31286cce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.586903 5407 raft_consensus.cc:482] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.587036 5407 raft_consensus.cc:504] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.587675 5407 leader_election.cc:288] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.589289 3718 tablet_service.cc:1467] Processing CreateTablet for tablet 81e467ea40364946b7c0bc4b5592996b (DEFAULT_TABLE table=default.Table_2486985deae44613a5f8d342c8236784 [id=74d8ef880ca24497ace800c16e041dfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.589397 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f287040b879642448367abe2b1c73cd9" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.589980 5407 raft_consensus.cc:482] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.590207 5407 raft_consensus.cc:504] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.590909 5407 leader_election.cc:288] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.592418 3718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e467ea40364946b7c0bc4b5592996b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:41.592371 3678 leader_election.cc:334] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.593921 3676 leader_election.cc:341] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.594069 3676 leader_election.cc:302] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.594930 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae7da0868b114052ba421801b76fb33c" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.595603 3678 leader_election.cc:334] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.596216 3676 leader_election.cc:341] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.596346 3676 leader_election.cc:302] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.596746 5733 raft_consensus.cc:2726] T f287040b879642448367abe2b1c73cd9 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.596832 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet 4059e5907b1e488ab9e1d04e563f96cd with delete_type TABLET_DATA_DELETED (replaced by tablet 99bb8e934e6c44ca88640715b11e6bef) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.596907 5733 raft_consensus.cc:2726] T ae7da0868b114052ba421801b76fb33c P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.599030 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet 0540e4e43ea34fbfbc66b6035c60684d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.599365 3706 tablet_service.cc:1514] Processing DeleteTablet for tablet b203df4124364737839bf4c1ae072988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.600651 4162 tablet_bootstrap.cc:492] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.600859 4162 ts_tablet_manager.cc:1397] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.073s user 0.009s sys 0.004s
I20260811 18:57:41.601840 4162 raft_consensus.cc:348] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.602147 4162 raft_consensus.cc:374] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.602272 4162 raft_consensus.cc:729] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.602602 4162 consensus_queue.cc:260] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.604127 4162 ts_tablet_manager.cc:1428] T 9722572533de436cb9149e6230570382 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:41.608428 4162 tablet_bootstrap.cc:492] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.596354 5757 ts_tablet_manager.cc:1918] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.608973 5757 log.cc:1196] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/e14f8717bcd94d6d88295fd8be130ca5
I20260811 18:57:41.609474 5757 ts_tablet_manager.cc:1939] T e14f8717bcd94d6d88295fd8be130ca5 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.610661 5757 tablet_replica.cc:330] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.610857 4162 tablet_bootstrap.cc:654] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.611763 5757 raft_consensus.cc:2229] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.613667 5757 raft_consensus.cc:2258] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.614406 5757 ts_tablet_manager.cc:1905] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.619944 4163 tablet_bootstrap.cc:654] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.625466 5757 ts_tablet_manager.cc:1918] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.626644 4162 tablet_bootstrap.cc:492] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.626956 4162 ts_tablet_manager.cc:1397] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.005s
I20260811 18:57:41.627923 4162 raft_consensus.cc:348] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.628242 4162 raft_consensus.cc:374] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.628377 4162 raft_consensus.cc:729] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.628706 4162 consensus_queue.cc:260] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.629570 4162 ts_tablet_manager.cc:1428] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:41.629791 5757 log.cc:1196] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/ebf7720e31d84ae2ab592da1e7a3132a
I20260811 18:57:41.634119 5757 ts_tablet_manager.cc:1939] T ebf7720e31d84ae2ab592da1e7a3132a P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.630162 4162 tablet_bootstrap.cc:492] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.636055 4169 ts_tablet_manager.cc:1918] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.636422 4169 log.cc:1196] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/48d32bb4eca047e4ba786c04b8bebf8b
I20260811 18:57:41.636829 4169 ts_tablet_manager.cc:1939] T 48d32bb4eca047e4ba786c04b8bebf8b P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.637773 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1066) took 5277 ms. Trace:
I20260811 18:57:41.637902 4169 rpcz_store.cc:276] 0811 18:57:36.360446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.360888 (+ 442us) service_pool.cc:227] Handling call
0811 18:57:41.471827 (+5110939us) ts_tablet_manager.cc:1145] Deleting tablet 48d32bb4eca047e4ba786c04b8bebf8b
0811 18:57:41.471839 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.563939 (+ 92100us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.635913 (+ 71974us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.637746 (+ 1833us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5110508}
I20260811 18:57:41.637863 4162 tablet_bootstrap.cc:654] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.638942 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260811 18:57:41.639079 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:41.643203 5757 tablet_replica.cc:330] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.643502 5757 raft_consensus.cc:2229] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.644155 5757 raft_consensus.cc:2258] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.644888 5757 ts_tablet_manager.cc:1905] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.650216 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e71af5ee8b463eb158ac9ae76ff252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.652412 5757 ts_tablet_manager.cc:1918] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:41.653496 3824 leader_election.cc:334] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.657958 5757 log.cc:1196] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:41.658739 5757 ts_tablet_manager.cc:1939] T cd1fc828f73642c88171d6ac827de90e P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.662767 5757 tablet_replica.cc:330] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.663261 5757 raft_consensus.cc:2229] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.663753 5757 raft_consensus.cc:2258] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.664572 5757 ts_tablet_manager.cc:1905] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.668593 5764 tablet.cc:1722] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:41.668864 3824 leader_election.cc:341] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 0bfb54175c834517b7af291c942333cd
W20260811 18:57:41.672037 3824 leader_election.cc:341] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 3ad5f39f59b8447cb3f15b9646e4b852
W20260811 18:57:41.672259 3824 leader_election.cc:341] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: f75dc515d7f3473b8e43f20255a70e3e
I20260811 18:57:41.672793 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1067) took 5312 ms. Trace:
I20260811 18:57:41.672971 4169 rpcz_store.cc:276] 0811 18:57:36.360557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.360672 (+ 115us) service_pool.cc:227] Handling call
0811 18:57:41.638275 (+5277603us) ts_tablet_manager.cc:1145] Deleting tablet 03c50dcffac54c7bac210eeaed340ca6
0811 18:57:41.638287 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.672763 (+ 34476us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5276748}
I20260811 18:57:41.673472 4169 tablet_replica.cc:330] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.673786 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 48d32bb4eca047e4ba786c04b8bebf8b (table default.Table_f3fb616fae4249b4a043ddb879c7db57 [id=907c2df8ce1e4a2e8c61efe0613c7820]) successfully deleted
I20260811 18:57:41.677443 4162 tablet_bootstrap.cc:492] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.677727 4162 ts_tablet_manager.cc:1397] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.013s
I20260811 18:57:41.681708 4162 raft_consensus.cc:348] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.682080 4162 raft_consensus.cc:374] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.682212 4162 raft_consensus.cc:729] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.682577 4162 consensus_queue.cc:260] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.683171 5733 consensus_queue.cc:1035] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:41.683633 4162 ts_tablet_manager.cc:1428] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 18:57:41.684695 5757 ts_tablet_manager.cc:1918] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.684859 5757 log.cc:1196] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4a64df7c295c4d9baded1b63a9980581
I20260811 18:57:41.685398 5757 ts_tablet_manager.cc:1939] T 4a64df7c295c4d9baded1b63a9980581 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.686548 4162 tablet_bootstrap.cc:492] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.691677 4162 tablet_bootstrap.cc:654] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.695935 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b40da8cb7324da9804e6fdafd8e8266: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.697108 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet e902efbb774244d18e3ba165bfb7c695 on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:41.698949 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 71df3738a5cf4b12820dda131ced62e1 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.706097 5642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:41.707721 5757 tablet_replica.cc:330] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.708169 5757 raft_consensus.cc:2229] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.708483 5757 raft_consensus.cc:2258] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.709322 5757 ts_tablet_manager.cc:1905] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.716753 4162 tablet_bootstrap.cc:492] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.717022 4162 ts_tablet_manager.cc:1397] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.011s
I20260811 18:57:41.718211 5757 ts_tablet_manager.cc:1918] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.718171 4162 raft_consensus.cc:348] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.718668 4162 raft_consensus.cc:374] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.718820 4162 raft_consensus.cc:729] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.719205 4162 consensus_queue.cc:260] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.718501 5757 log.cc:1196] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/a0116997a1c64e658045c319f1c45861
I20260811 18:57:41.720047 5757 ts_tablet_manager.cc:1939] T a0116997a1c64e658045c319f1c45861 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.721091 4162 ts_tablet_manager.cc:1428] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:41.676192 4163 tablet_bootstrap.cc:492] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.723898 4163 ts_tablet_manager.cc:1397] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.161s user 0.005s sys 0.001s
W20260811 18:57:41.681638 3824 leader_election.cc:334] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.711344 5593 raft_consensus.cc:482] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.724107 3824 leader_election.cc:302] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.724172 5593 raft_consensus.cc:504] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.724764 5593 leader_election.cc:288] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.724917 5593 raft_consensus.cc:2726] T 0bfb54175c834517b7af291c942333cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.724944 4163 raft_consensus.cc:348] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.725169 4163 raft_consensus.cc:374] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.725267 4163 raft_consensus.cc:729] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
W20260811 18:57:41.725320 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 03c50dcffac54c7bac210eeaed340ca6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03c50dcffac54c7bac210eeaed340ca6 already in progress: creating tablet
I20260811 18:57:41.725471 4163 consensus_queue.cc:260] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.726074 4163 ts_tablet_manager.cc:1428] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:41.726708 4169 raft_consensus.cc:2229] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.726949 4169 raft_consensus.cc:2258] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.727558 4163 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.727655 4169 ts_tablet_manager.cc:1905] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:41.729018 3824 leader_election.cc:334] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.729147 3824 leader_election.cc:302] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:41.729455 3824 leader_election.cc:334] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.729892 5630 raft_consensus.cc:2726] T f75dc515d7f3473b8e43f20255a70e3e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.730867 5630 raft_consensus.cc:482] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:57:41.731369 3678 consensus_peers.cc:487] T 2599dbd65a814601928373ab9910e3ca P bd8c4bdbd811417ab011a7131f17d283 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:41.729553 3824 leader_election.cc:302] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.731026 5630 raft_consensus.cc:504] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.732227 5593 raft_consensus.cc:2726] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:57:41.732331 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:41.732542 5630 leader_election.cc:288] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.734661 3824 leader_election.cc:334] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:41.735526 3824 leader_election.cc:334] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.737623 5772 tablet.cc:1722] T 73f05e1d5cb947fe9dbcc2e876a23788 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260811 18:57:41.747799 3824 leader_election.cc:341] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 059b325df1db4fde9d4b26b559121316
I20260811 18:57:41.751370 3824 leader_election.cc:302] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.751833 5593 raft_consensus.cc:2726] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:41.752022 3824 leader_election.cc:341] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 2c0f6df7a291456d80538e2b76417873
I20260811 18:57:41.752140 3824 leader_election.cc:302] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.752360 5593 raft_consensus.cc:2726] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.749441 4163 tablet_bootstrap.cc:654] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.753261 4169 ts_tablet_manager.cc:1918] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.753474 4169 log.cc:1196] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/d00d943b778741b4aeec076ba46a9c40
I20260811 18:57:41.753713 5764 tablet.cc:1722] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.754590 5757 tablet_replica.cc:330] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.754855 4162 tablet_bootstrap.cc:492] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.754905 5757 raft_consensus.cc:2229] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.755072 5757 raft_consensus.cc:2258] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.756453 5772 tablet.cc:1722] T 4f3c4536fc7440db83851fc8d7b3de10 P bd8c4bdbd811417ab011a7131f17d283: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260811 18:57:41.757334 4162 tablet_bootstrap.cc:654] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.776816 5757 ts_tablet_manager.cc:1905] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.777021 5630 raft_consensus.cc:482] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.777308 5630 raft_consensus.cc:504] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.778072 5630 leader_election.cc:288] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.778043 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet ae96e10a2ed4407c9d42a359d82d389e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:38286
W20260811 18:57:41.780735 3824 leader_election.cc:341] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: 42405aec21e3405486b6c658438b5fb2
W20260811 18:57:41.781010 3824 leader_election.cc:334] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.781152 3824 leader_election.cc:302] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:41.781476 5630 raft_consensus.cc:2726] T 42405aec21e3405486b6c658438b5fb2 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.782042 4169 ts_tablet_manager.cc:1939] T d00d943b778741b4aeec076ba46a9c40 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.783241 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1074) took 4853 ms. Trace:
I20260811 18:57:41.785106 5757 ts_tablet_manager.cc:1918] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.786736 5630 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.790458 5630 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.791162 5630 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.785815 4163 tablet_bootstrap.cc:492] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.791441 4163 ts_tablet_manager.cc:1397] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.004s
I20260811 18:57:41.792364 4163 raft_consensus.cc:348] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.792595 4163 raft_consensus.cc:374] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.792680 4163 raft_consensus.cc:729] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.792919 4163 consensus_queue.cc:260] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.798014 4163 ts_tablet_manager.cc:1428] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:41.803516 4163 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:41.806011 4169 rpcz_store.cc:276] 0811 18:57:36.929690 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.929791 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:41.673408 (+4743617us) ts_tablet_manager.cc:1145] Deleting tablet d00d943b778741b4aeec076ba46a9c40
0811 18:57:41.673426 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.749781 (+ 76355us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.753148 (+ 3367us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.783217 (+ 30069us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4743406}
I20260811 18:57:41.807451 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1075) took 4877 ms. Trace:
W20260811 18:57:41.807802 3824 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.813000 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet d00d943b778741b4aeec076ba46a9c40 (table default.Table_267e0bd672f9419fbd1a46b93d0d8169 [id=7ef504f80694466a804cb34703c396fe]) successfully deleted
I20260811 18:57:41.785601 3718 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb6ac49f1d4403dad38a3125264088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:41 UTC) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.786628 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 58bc967770de4ed486b638fb6b174db8 with delete_type TABLET_DATA_DELETED (replaced by tablet f336fdd55965432f83b2d1c436fdf1fa) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.790522 3711 tablet_service.cc:1514] Processing DeleteTablet for tablet eea325ca357146fcaa408d87803da349 with delete_type TABLET_DATA_DELETED (replaced by tablet a4e0b84bdb134a459d2d9afd046326c2) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.792104 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet c44eee516d634d9a986107b55410dfc3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5faabb319fc84f24ab548a707be284f1) from {username='slave'} at 127.0.0.1:38286
I20260811 18:57:41.793721 4162 tablet_bootstrap.cc:492] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.815361 4162 ts_tablet_manager.cc:1397] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.011s
I20260811 18:57:41.817764 4162 raft_consensus.cc:348] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.795825 5742 raft_consensus.cc:482] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.818511 5742 raft_consensus.cc:504] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.820163 5742 leader_election.cc:288] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.800673 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.820714 3741 raft_consensus.cc:2454] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:41.813970 5734 raft_consensus.cc:482] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.810992 4169 rpcz_store.cc:276] 0811 18:57:36.930284 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.930385 (+ 101us) service_pool.cc:227] Handling call
0811 18:57:41.807302 (+4876917us) ts_tablet_manager.cc:1145] Deleting tablet 73c519528f75423195b6bdf6683fd91b
0811 18:57:41.807316 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.807427 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4876752}
I20260811 18:57:41.824239 5734 raft_consensus.cc:504] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.824937 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1076) took 4894 ms. Trace:
I20260811 18:57:41.822724 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930b6e8839e24ec9a69854349bc3ecf8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.825999 5734 leader_election.cc:288] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.819497 4163 tablet_bootstrap.cc:654] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:41.828738 3676 leader_election.cc:341] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.829514 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d81e93739f44d9e99b6ebc647e1d0e4" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:41.833011 3676 leader_election.cc:341] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.825589 3824 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.833701 5630 raft_consensus.cc:2781] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.833935 5630 raft_consensus.cc:482] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.834107 5630 raft_consensus.cc:3037] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.818282 4162 raft_consensus.cc:374] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.835040 4162 raft_consensus.cc:729] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
W20260811 18:57:41.835537 3678 leader_election.cc:334] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.835774 3678 leader_election.cc:302] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.835626 4162 consensus_queue.cc:260] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.836347 3678 leader_election.cc:334] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.836956 4162 ts_tablet_manager.cc:1428] T 7a0e0d139c924f1dafe42da0bedada72 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260811 18:57:41.837004 5742 raft_consensus.cc:2726] T 1d81e93739f44d9e99b6ebc647e1d0e4 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.825129 4169 rpcz_store.cc:276] 0811 18:57:36.930847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.930910 (+ 63us) service_pool.cc:227] Handling call
0811 18:57:41.824577 (+4893667us) ts_tablet_manager.cc:1145] Deleting tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b
0811 18:57:41.824611 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.824847 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4893448}
I20260811 18:57:41.837867 4169 tablet_replica.cc:330] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:41.838551 4169 raft_consensus.cc:2229] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.838914 4169 raft_consensus.cc:2258] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.840070 4169 ts_tablet_manager.cc:1905] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.841265 5630 raft_consensus.cc:504] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.842909 5630 leader_election.cc:288] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:41.844063 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 73c519528f75423195b6bdf6683fd91b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c519528f75423195b6bdf6683fd91b already in progress: creating tablet
I20260811 18:57:41.836550 3678 leader_election.cc:302] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.845455 5734 raft_consensus.cc:2726] T 930b6e8839e24ec9a69854349bc3ecf8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.847059 4162 tablet_bootstrap.cc:492] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
W20260811 18:57:41.847831 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33fdcd8711fe4c3b9ac43a035ffa0c5b already in progress: creating tablet
I20260811 18:57:41.850260 5757 log.cc:1196] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/8d1e244addf5462499fcadaa3332cba1
I20260811 18:57:41.857270 5792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:41.857770 5757 ts_tablet_manager.cc:1939] T 8d1e244addf5462499fcadaa3332cba1 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.861752 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc07c28ccfd48ee9c33e49442c6047b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.862213 3741 raft_consensus.cc:3037] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.867229 5757 tablet_replica.cc:330] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.867848 5757 raft_consensus.cc:2229] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:41.869150 3824 leader_election.cc:334] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.868666 3741 raft_consensus.cc:2454] T 8dc07c28ccfd48ee9c33e49442c6047b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.873762 5757 raft_consensus.cc:2258] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.875325 5757 ts_tablet_manager.cc:1905] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.876322 3824 leader_election.cc:302] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.877255 5630 raft_consensus.cc:2781] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.875913 4162 tablet_bootstrap.cc:654] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.878873 5630 raft_consensus.cc:686] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.879503 5630 consensus_queue.cc:237] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.881269 5734 raft_consensus.cc:482] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.881587 5734 raft_consensus.cc:504] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.886453 5734 leader_election.cc:288] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:41.859776 5792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:41.889168 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdbcfcd69a24d3f8a0f8383ba4c88e8" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:41.889850 5792 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d85eb97df7824bc9bad9dca349b0c180 (127.3.72.125:38889), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:41.892112 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1649 } } ignore_live_leader: false dest_uuid: "d85eb97df7824bc9bad9dca349b0c180" is_pre_election: true
I20260811 18:57:41.892885 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1649 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
W20260811 18:57:41.893803 3676 leader_election.cc:341] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:41.895349 5237 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cb005c681cf42d4ac278ec73a28e4a9; no voters: 0a82a7352eb644b4a6baf09d8084b652, d85eb97df7824bc9bad9dca349b0c180
I20260811 18:57:41.899791 5757 ts_tablet_manager.cc:1918] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.900266 5757 log.cc:1196] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/4059e5907b1e488ab9e1d04e563f96cd
I20260811 18:57:41.901154 5757 ts_tablet_manager.cc:1939] T 4059e5907b1e488ab9e1d04e563f96cd P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.903774 5757 tablet_replica.cc:330] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.904450 5757 raft_consensus.cc:2229] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.904956 5757 raft_consensus.cc:2258] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.906299 5757 ts_tablet_manager.cc:1905] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.906534 4163 tablet_bootstrap.cc:492] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:41.910946 4163 ts_tablet_manager.cc:1397] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.108s user 0.012s sys 0.001s
I20260811 18:57:41.912084 4163 raft_consensus.cc:348] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.912403 4163 raft_consensus.cc:374] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.912534 4163 raft_consensus.cc:729] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.912855 4163 consensus_queue.cc:260] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:41.914481 3678 leader_election.cc:334] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.914649 3678 leader_election.cc:302] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.915095 5734 raft_consensus.cc:2726] T dcdbcfcd69a24d3f8a0f8383ba4c88e8 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:41.916675 5757 ts_tablet_manager.cc:1918] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.916870 5757 log.cc:1196] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/0540e4e43ea34fbfbc66b6035c60684d
I20260811 18:57:41.917327 5757 ts_tablet_manager.cc:1939] T 0540e4e43ea34fbfbc66b6035c60684d P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
I20260811 18:57:41.918735 5757 tablet_replica.cc:330] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: stopping tablet replica
I20260811 18:57:41.919049 5757 raft_consensus.cc:2229] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:41.919276 5757 raft_consensus.cc:2258] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:41.919891 5757 ts_tablet_manager.cc:1905] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:41.922226 4169 ts_tablet_manager.cc:1918] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260811 18:57:41.922937 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 38e24f8d4522466d81605ea37fa7064a (table default.Table_2058b5fb0d08491a9f2fa354276fc21b [id=8b95050589ce44abb38490c7e4a310b8]) successfully deleted
W20260811 18:57:41.923532 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet f389d2b7ff354ac4bab5d68b9ff201c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f389d2b7ff354ac4bab5d68b9ff201c6 already in progress: creating tablet
I20260811 18:57:41.923965 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 9ec4bba841dd446aa3db751358ce7f32 (table default.Table_4927a23edf074354bdd4cf94e4b74eff [id=9a50dade531e49ae9b74e434d17b78b6]) successfully deleted
I20260811 18:57:41.924584 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b20872d80c9545e5822bfc82d496ce97 (table default.Table_3d490ceda4c745668e9deb77c4a07fc8 [id=50ea0c3a29ed429aa29f8b57e6068449]) successfully deleted
I20260811 18:57:41.925983 4169 log.cc:1196] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/3ad5f39f59b8447cb3f15b9646e4b852
I20260811 18:57:41.926030 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet e14f8717bcd94d6d88295fd8be130ca5 (table default.Table_5f4d9f42e99c454e88964d343262e534 [id=235ac0730a9049e89deccd4419925c83]) successfully deleted
I20260811 18:57:41.926251 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet ebf7720e31d84ae2ab592da1e7a3132a (table default.Table_241ec7c366e84652a6d9c0123d2149ac [id=755f6da3c0ae4e74a8c58225a425f458]) successfully deleted
I20260811 18:57:41.926577 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet cd1fc828f73642c88171d6ac827de90e (table default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]) successfully deleted
I20260811 18:57:41.923022 4163 ts_tablet_manager.cc:1428] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
W20260811 18:57:41.926824 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b9fbac57c9c44a028f5b258593aff86d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fbac57c9c44a028f5b258593aff86d already in progress: creating tablet
W20260811 18:57:41.927117 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d65cb8d901c04d8ba2b7a40efc7fa3af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65cb8d901c04d8ba2b7a40efc7fa3af already in progress: creating tablet
I20260811 18:57:41.927325 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4a64df7c295c4d9baded1b63a9980581 (table default.Table_4a82335ab20e412c93a5ac0938a96012 [id=b13ec9f38f84479c924bba986bd42ec6]) successfully deleted
I20260811 18:57:41.927362 4163 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:41.927538 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet fdab1c3c431c4dceb08b00eabcfa1ceb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdab1c3c431c4dceb08b00eabcfa1ceb already in progress: creating tablet
I20260811 18:57:41.927563 5757 ts_tablet_manager.cc:1918] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:41.927701 5757 log.cc:1196] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-0/wal/wals/b203df4124364737839bf4c1ae072988
W20260811 18:57:41.927735 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 0e556811ad1643f1baa3bbf4edebd157 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e556811ad1643f1baa3bbf4edebd157 already in progress: creating tablet
W20260811 18:57:41.927912 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a4afeb6069e4d3d9457fffcf5ec48f0 already in progress: creating tablet
W20260811 18:57:41.928095 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 5d93073a69c143ff9460248848a108d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d93073a69c143ff9460248848a108d4 already in progress: creating tablet
W20260811 18:57:41.928321 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 71a7eda2248c4f708bdc772ea3598dec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71a7eda2248c4f708bdc772ea3598dec already in progress: creating tablet
I20260811 18:57:41.928366 5757 ts_tablet_manager.cc:1939] T b203df4124364737839bf4c1ae072988 P bd8c4bdbd811417ab011a7131f17d283: Deleting consensus metadata
W20260811 18:57:41.928504 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a49a2ac4cc894d7fb17104d0ea4fa6e7 already in progress: creating tablet
W20260811 18:57:41.928748 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9a1f2bfa62e45a1abe72ab94c9f7eeb already in progress: creating tablet
W20260811 18:57:41.928969 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 72f11748cf984942ba99bda4ad6605ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72f11748cf984942ba99bda4ad6605ab already in progress: creating tablet
I20260811 18:57:41.929996 4163 tablet_bootstrap.cc:654] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.930253 5792 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7cb005c681cf42d4ac278ec73a28e4a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:41.931074 4169 ts_tablet_manager.cc:1939] T 3ad5f39f59b8447cb3f15b9646e4b852 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:41.932530 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1077) took 5000 ms. Trace:
I20260811 18:57:41.935891 4162 tablet_bootstrap.cc:492] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:41.936053 4162 ts_tablet_manager.cc:1397] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.089s user 0.004s sys 0.005s
I20260811 18:57:41.937196 4162 raft_consensus.cc:348] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.937472 4162 raft_consensus.cc:374] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.937528 4162 raft_consensus.cc:729] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.937758 4162 consensus_queue.cc:260] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:41.938290 5767 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:41.938534 5767 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.939334 5767 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.941035 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:41.941291 3741 raft_consensus.cc:2454] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
W20260811 18:57:41.941962 3824 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.942267 4162 ts_tablet_manager.cc:1428] T 2be6762e326448d7b677a5641e2b3103 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:57:41.942299 3824 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:41.942608 5767 raft_consensus.cc:2781] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:41.942734 5767 raft_consensus.cc:482] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:41.942862 4162 tablet_bootstrap.cc:492] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:41.942867 5767 raft_consensus.cc:3037] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.945864 5767 raft_consensus.cc:504] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:41.946624 5767 leader_election.cc:288] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:41.947527 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1af33fd98c6486f975368ec90493db7" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:41.947748 3741 raft_consensus.cc:3037] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.951356 3741 raft_consensus.cc:2454] T e1af33fd98c6486f975368ec90493db7 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:41.955464 4162 tablet_bootstrap.cc:654] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.957298 3824 leader_election.cc:302] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:41.957984 5593 raft_consensus.cc:2781] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.958180 5593 raft_consensus.cc:686] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:41.958525 5593 consensus_queue.cc:237] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
W20260811 18:57:41.987200 3824 leader_election.cc:334] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:41.987805 4169 rpcz_store.cc:276] 0811 18:57:36.932302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.932362 (+ 60us) service_pool.cc:227] Handling call
0811 18:57:41.837736 (+4905374us) ts_tablet_manager.cc:1145] Deleting tablet 3ad5f39f59b8447cb3f15b9646e4b852
0811 18:57:41.837779 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:41.918785 (+ 81006us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.922092 (+ 3307us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:41.932487 (+ 10395us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4905177}
I20260811 18:57:41.989490 3548 catalog_manager.cc:5495] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:41.990007 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 2c29af3615e74722a809d3e5ede9b9a1 (table default.Table_4d1fcb681d504d57b9d288e08be657c2 [id=dd46ddba2d9843a981969eb5db1d4cea]). Expected version 1 got 0
I20260811 18:57:41.996435 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 6ddd2d6468454840818302a8fd9bfb88 (DEFAULT_TABLE table=default.Table_59d30e9c40554addab5dc9b7c1d11432 [id=785355398c254031b55ce65fb12e21c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.997359 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 45e040f8120a4edfba89c1a8d55deb4a (DEFAULT_TABLE table=default.Table_d82e6143f45747ddafae8a1e2651b9a0 [id=d8d665af78f448268e114e574587dcfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.997776 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ddd2d6468454840818302a8fd9bfb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.997923 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 9be08aff2b324689a34748f897c04f50 (DEFAULT_TABLE table=default.Table_61e454a7dd9c41c58ecc0605d0ddf6b0 [id=90c633f7d0b44c30823b6e07b231d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:41.998319 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be08aff2b324689a34748f897c04f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.998610 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ebead0adb7184690b117d775949f5973 (DEFAULT_TABLE table=default.Table_6dab2aa50f394dabbbae787dca84b57a [id=07a603a975f144dd85497bb60920e9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.002857 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebead0adb7184690b117d775949f5973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.996711 5630 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.004124 5630 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.004845 5630 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:41.998790 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2afb2162af4ca8a03481c7df0d8766 (DEFAULT_TABLE table=default.Table_8edb4b4387a444abb3b70122ae70018e [id=f90f0baeb30349a39dab268a3a70e0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.006809 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.007109 3741 raft_consensus.cc:2454] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.007829 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2afb2162af4ca8a03481c7df0d8766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.009574 3824 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
W20260811 18:57:42.009948 3824 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.010403 5630 raft_consensus.cc:2781] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.010541 5630 raft_consensus.cc:482] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.010896 5630 raft_consensus.cc:3037] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.010967 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e040f8120a4edfba89c1a8d55deb4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.011516 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1078) took 5078 ms. Trace:
I20260811 18:57:42.012696 4163 tablet_bootstrap.cc:492] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.012868 4163 ts_tablet_manager.cc:1397] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.086s user 0.000s sys 0.004s
I20260811 18:57:42.013777 4163 raft_consensus.cc:348] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.014036 4163 raft_consensus.cc:374] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.014166 4163 raft_consensus.cc:729] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.014482 4163 consensus_queue.cc:260] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.016506 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 3ad5f39f59b8447cb3f15b9646e4b852 (table default.Table_d8171ab03efd442a9bf4e1176f04cb24 [id=dec975b4da8d4185929ab3b7392c4e38]) successfully deleted
I20260811 18:57:42.011641 4169 rpcz_store.cc:276] 0811 18:57:36.932770 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.932827 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:42.011300 (+5078473us) ts_tablet_manager.cc:1145] Deleting tablet 7cb3e997b4454598bdce102e92296887
0811 18:57:42.011315 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.011432 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5078318}
I20260811 18:57:42.017252 4169 tablet_replica.cc:330] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:42.018010 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7cb3e997b4454598bdce102e92296887 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7cb3e997b4454598bdce102e92296887 already in progress: creating tablet
I20260811 18:57:42.018389 4169 raft_consensus.cc:2229] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.021562 5734 raft_consensus.cc:482] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.021782 5734 raft_consensus.cc:504] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.022630 5734 leader_election.cc:288] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:42.023797 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bc6cce2c19746219b8b0fe54b13dcd6" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
W20260811 18:57:42.024442 3676 leader_election.cc:341] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
W20260811 18:57:42.025674 3678 leader_election.cc:334] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.025825 3678 leader_election.cc:302] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.026249 5734 raft_consensus.cc:2726] T 0bc6cce2c19746219b8b0fe54b13dcd6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:42.031700 3678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 90 similar messages]
I20260811 18:57:42.041286 4169 raft_consensus.cc:2258] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.042639 4169 ts_tablet_manager.cc:1905] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.044024 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 80d3f09aada44148b45897e4a7e30bd1 (DEFAULT_TABLE table=default.Table_93e4cfd44cab49469aa14294ac0ba348 [id=dec80fce066e407d85eaab6390d0c75d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.044951 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 66e70ff397bd493992d2f08a30221b69 (DEFAULT_TABLE table=default.Table_4821250a7fa84b68a74499b0d7a1228d [id=8cac8dcdf645477b926cf3a72a64dd61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.043995 3871 tablet_service.cc:1467] Processing CreateTablet for tablet ff03310a9aca474d8148818b2126be67 (DEFAULT_TABLE table=default.Table_b691c53749bc40a0a829753ba4c7e6aa [id=3742d08328974ae3979f3b27f4758e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.045428 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e70ff397bd493992d2f08a30221b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.045640 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff03310a9aca474d8148818b2126be67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.045908 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b609177908d04a829b1fe744ef0f53e1 (table default.Table_4a427b04935c4d62b0595df0085e3a98 [id=4845fddf6562470ebb7275b67fad076e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b609177908d04a829b1fe744ef0f53e1 consensus state has no leader
I20260811 18:57:42.046160 5734 raft_consensus.cc:482] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.046322 5734 raft_consensus.cc:504] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.046995 5734 leader_election.cc:288] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837)
I20260811 18:57:42.048182 3887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114397bea09444479df113552c22e154" candidate_uuid: "bd8c4bdbd811417ab011a7131f17d283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" is_pre_election: true
I20260811 18:57:42.048621 5768 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.048810 5768 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.049443 5768 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.049881 3676 leader_election.cc:341] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): Illegal state: must be running to vote when last-logged opid is not known
I20260811 18:57:42.041631 5630 raft_consensus.cc:504] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.050799 5630 leader_election.cc:288] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.051064 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80d3f09aada44148b45897e4a7e30bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.053282 4163 ts_tablet_manager.cc:1428] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.032s user 0.003s sys 0.000s
W20260811 18:57:42.057549 3678 leader_election.cc:334] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.057719 3678 leader_election.cc:302] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd8c4bdbd811417ab011a7131f17d283; no voters: 57e8f62fa63a4fee97f268f5b7d3ae24, 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.058495 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.058544 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 0e928aaaf32d4ba1ac24a4df0e05f2f0 (DEFAULT_TABLE table=default.Table_020300b2b8b949278c35159bd43fe19d [id=5726999b7dcf4236b82fc9a7d097cfa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.058768 3741 raft_consensus.cc:2454] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.059039 5734 raft_consensus.cc:2726] T 114397bea09444479df113552c22e154 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:42.059295 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e928aaaf32d4ba1ac24a4df0e05f2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.062686 3851 tablet_service.cc:1467] Processing CreateTablet for tablet dac5803fac024ec8aeaafee8dad0ce7f (DEFAULT_TABLE table=default.Table_c17b389153604aeeb9c985eaf661f44a [id=40aa393b93fd4946b648765f3c864b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.044934 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd80c582f9043d79e6bedf2a500c7ec (DEFAULT_TABLE table=default.Table_5e94a3f1b3a1418d8d5ff4ae641da751 [id=2c12b1a39437497f982e0b90b1b86a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.065439 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 659c2937c672482f9ea7b4e524d9847b (DEFAULT_TABLE table=default.Table_d57e6ac8edbc46a69270b9277778e7b2 [id=a9fd0364a9664280a268e512d946f4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.067013 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a811d4488d0e4541808a5e1aa44d663b" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.067135 3552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: Sending DeleteTablet for 3 replicas of tablet 1f9621bbd0f94223a5d36bc01487f615
I20260811 18:57:42.067258 3741 raft_consensus.cc:3037] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.067654 3552 catalog_manager.cc:6565] ReplaceTablet: tablet 1f9621bbd0f94223a5d36bc01487f615 deleted and replaced by tablet 332d3d9ddb224a4697809b19b622749d
I20260811 18:57:42.070266 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 6f6b2ca385374abeb709fccbdf6fa0ba (table default.Table_6c8484897dd046ab8499d1069356cdcd [id=221d836949c341a09235288dc5802137]). Expected version 3 got 0
I20260811 18:57:42.070567 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:42.070832 3741 raft_consensus.cc:2454] T a811d4488d0e4541808a5e1aa44d663b P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.070950 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet d73a98bd35bd444195fd32fe6c67a04f (table default.Table_e496287ddd104a6db81655bea6a19802 [id=356820f47d4942d0a4cce16978925421]). Expected version 1 got 0
I20260811 18:57:42.071117 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet eb82243777a9494b96959ced0c24de5a (table default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1]). Expected version 2 got 0
I20260811 18:57:42.071400 4163 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.073709 4163 tablet_bootstrap.cc:654] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.078104 3824 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.078639 5768 raft_consensus.cc:2781] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.078806 5768 raft_consensus.cc:482] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.078936 5768 raft_consensus.cc:3037] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.081851 3361 master-stress-test.cc:580] Killed test thread 24/25
I20260811 18:57:42.081970 3361 master-stress-test.cc:578] Killing test thread 25/25
I20260811 18:57:42.082070 3361 master-stress-test.cc:580] Killed test thread 25/25
I20260811 18:57:42.081974 5768 raft_consensus.cc:504] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.082311 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3659
I20260811 18:57:42.083123 5768 leader_election.cc:288] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.086959 5768 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.087255 5768 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.088202 5768 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.083201 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9621bbd0f94223a5d36bc01487f615 with delete_type TABLET_DATA_DELETED (replaced by tablet 332d3d9ddb224a4697809b19b622749d) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.091307 3824 leader_election.cc:302] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.097259 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dac5803fac024ec8aeaafee8dad0ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.097354 5630 raft_consensus.cc:2781] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.097558 5630 raft_consensus.cc:686] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.097978 5630 consensus_queue.cc:237] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.098394 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd80c582f9043d79e6bedf2a500c7ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.099023 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 659c2937c672482f9ea7b4e524d9847b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.103523 3860 tablet_service.cc:1467] Processing CreateTablet for tablet cc188485180e4fbe995f1b089f9ee8ae (DEFAULT_TABLE table=default.Table_193dfc5ac3914a1f90e082521d0f84d0 [id=4c67d86ae83848199da5466ca7d56b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.104426 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc188485180e4fbe995f1b089f9ee8ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.105520 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 4b65d92574764d508077e9e3c47d56a5 (DEFAULT_TABLE table=default.Table_8cc94cebd1834be58b21d1512bb24571 [id=f397dfcc6d4f409ab0ed55231ef67674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.106014 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b65d92574764d508077e9e3c47d56a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.116591 3824 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.117599 3824 leader_election.cc:334] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.124920 4163 tablet_bootstrap.cc:492] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.125136 4163 ts_tablet_manager.cc:1397] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.004s
I20260811 18:57:42.126008 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ed9c1542c24b2ea3c2de8ec0f589d6" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.126269 3741 raft_consensus.cc:3037] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.129477 3741 raft_consensus.cc:2454] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.130330 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283" is_pre_election: true
I20260811 18:57:42.130579 3741 raft_consensus.cc:2454] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 0.
I20260811 18:57:42.140987 5630 raft_consensus.cc:482] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.141268 5630 raft_consensus.cc:504] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.143018 5630 leader_election.cc:288] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.148064 4162 tablet_bootstrap.cc:492] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:42.148218 4162 ts_tablet_manager.cc:1397] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.206s user 0.082s sys 0.058s
I20260811 18:57:42.149070 4162 raft_consensus.cc:348] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.149307 4162 raft_consensus.cc:374] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.149355 4162 raft_consensus.cc:729] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.149556 4162 consensus_queue.cc:260] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:42.149853 3824 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.150040 3824 leader_election.cc:334] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.150319 3824 leader_election.cc:302] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.150887 5768 raft_consensus.cc:2781] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.151069 5768 raft_consensus.cc:686] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.151322 3548 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_7fcad849494e46c1bcef25d34ef0cd78 [id=3dc6eeef579941eaaa83be543d1c01d1] alter complete (version 2)
I20260811 18:57:42.151355 5768 consensus_queue.cc:237] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.161259 4162 ts_tablet_manager.cc:1428] T 54fa32f8fa1144c4bd9dcd00a550001a P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260811 18:57:42.161396 4169 ts_tablet_manager.cc:1918] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.161595 4169 log.cc:1196] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/68a7051bc539468aafba30fbaf0669d4
I20260811 18:57:42.162027 4169 ts_tablet_manager.cc:1939] T 68a7051bc539468aafba30fbaf0669d4 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.162871 4163 raft_consensus.cc:348] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.163239 4163 raft_consensus.cc:374] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.164191 3824 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters: 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:42.164911 3824 leader_election.cc:341] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Not found: Tablet not found: ffa8a4a1c0cb4834be501bdd7d468472
I20260811 18:57:42.165160 5630 raft_consensus.cc:2781] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:42.167223 5630 raft_consensus.cc:482] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:42.167438 5630 raft_consensus.cc:3037] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.171573 4162 tablet_bootstrap.cc:492] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:42.176218 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 930b6e8839e24ec9a69854349bc3ecf8 (table default.Table_9db7061f3798449280da2e921e2181bf [id=d7d11af2eee54ab880a15c525b7ad0a9]). Expected version 4 got 0
I20260811 18:57:42.176656 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 114397bea09444479df113552c22e154 (table default.Table_14587bf2389240a1b10454cad65b748a [id=cfba5a388f8f406d8b2a9ae1fc397585]). Expected version 1 got 0
I20260811 18:57:42.177124 3547 catalog_manager.cc:5587] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817) does not have the latest schema for tablet 473040a90e6941c38beb697316d2a27b (table default.Table_94cadddacfc545b6af75e68ed02f34c1 [id=565db9ded0d34382b0b6aae53f17ceb8]). Expected version 1 got 0
I20260811 18:57:42.180516 4162 tablet_bootstrap.cc:654] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.197398 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_f5e5c61d6ae446fdbe466f320a17ed6c [id=bd974444a6c546959eb6442a80543187] alter complete (version 2)
I20260811 18:57:42.210290 3547 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652: default.Table_78bb9ed48dc34a4ea11b23eec0467238 [id=0ea510ab7a724cadb94d2cfbc5fde4ba] alter complete (version 2)
I20260811 18:57:42.210906 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 9600b135935b443daee15cc864ba0032 (DEFAULT_TABLE table=default.Table_40846b4d992644f9ad4942d6e529abb1 [id=36f2ed90d7544a70850e1298a9349d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.212190 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9600b135935b443daee15cc864ba0032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.214226 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1079) took 5281 ms. Trace:
I20260811 18:57:42.214442 4169 rpcz_store.cc:276] 0811 18:57:36.933231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.933289 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.017107 (+5083818us) ts_tablet_manager.cc:1145] Deleting tablet 68a7051bc539468aafba30fbaf0669d4
0811 18:57:42.017121 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.110498 (+ 93377us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.161230 (+ 50732us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.214205 (+ 52975us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22558,"tablet-delete.queue_time_us":5083672}
I20260811 18:57:42.215448 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1080) took 5281 ms. Trace:
I20260811 18:57:42.215619 4169 rpcz_store.cc:276] 0811 18:57:36.933669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:36.933723 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:42.214943 (+5281220us) ts_tablet_manager.cc:1145] Deleting tablet 8f9b279c6c7344f1931b81e10248b1b6
0811 18:57:42.214956 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.215427 (+ 471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5281059}
I20260811 18:57:42.218675 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 5852830ee0544f38aa6ed8c8c2e250a6 (DEFAULT_TABLE table=default.Table_c0ad178862a14f968c9a334a1570eb82 [id=7ce0ced05e7d477fb96f5eda68ffbbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.219705 5630 raft_consensus.cc:504] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.220398 5630 leader_election.cc:288] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Requested vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.221377 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 79539e937f81437ab3b12b8465f12d97 (DEFAULT_TABLE table=default.Table_084495aab60944cda9e4a3f796132fa2 [id=05961569f96948d0bd5765c6976d990e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.221804 4162 tablet_bootstrap.cc:492] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: No bootstrap required, opened a new log
I20260811 18:57:42.221946 4162 ts_tablet_manager.cc:1397] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Time spent bootstrapping tablet: real 0.050s user 0.014s sys 0.005s
I20260811 18:57:42.222703 4162 raft_consensus.cc:348] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.222930 4162 raft_consensus.cc:374] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.222978 4162 raft_consensus.cc:729] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c4bdbd811417ab011a7131f17d283, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.223167 4162 consensus_queue.cc:260] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.225644 3863 tablet_service.cc:1467] Processing CreateTablet for tablet fe48bbab3d7e4f7591883296a9996db7 (DEFAULT_TABLE table=default.Table_e98a0fba26cb4171bf8eb772249c3e47 [id=861ccaeafbd246c0a56b21709e61e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.227525 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 68a7051bc539468aafba30fbaf0669d4 (table default.Table_821a790ed161401bb9758d71a0760916 [id=4a5d04ba5f3942a3942909ebd2bf5e20]) successfully deleted
W20260811 18:57:42.227823 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 8f9b279c6c7344f1931b81e10248b1b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f9b279c6c7344f1931b81e10248b1b6 already in progress: creating tablet
I20260811 18:57:42.227766 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc68b2b47ca4426a204df0dea3801f0 (DEFAULT_TABLE table=default.Table_f7751679e1394c27bdfb569a73a14165 [id=65adccc0a8a2492abf41483dcc44bba8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.231402 4162 ts_tablet_manager.cc:1428] T 135a8c34b93544d98383fc3103966a54 P bd8c4bdbd811417ab011a7131f17d283: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260811 18:57:42.232446 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c575ba3060a140759484bea1462ffd73 (DEFAULT_TABLE table=default.Table_cef814687d014504b5a7a569342e4bb4 [id=4d3139f9e2ac44c182ca8bbd99ef2871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.232699 3741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73a43558dbb48889ab205108f4ce6be" candidate_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c4bdbd811417ab011a7131f17d283"
I20260811 18:57:42.232969 3741 raft_consensus.cc:3037] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:42.235656 3741 raft_consensus.cc:2454] T d73a43558dbb48889ab205108f4ce6be P bd8c4bdbd811417ab011a7131f17d283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e8f62fa63a4fee97f268f5b7d3ae24 in term 1.
I20260811 18:57:42.237428 4162 tablet_bootstrap.cc:492] T 4e0747ef88574c68ae074732cb7095b9 P bd8c4bdbd811417ab011a7131f17d283: Bootstrap starting.
I20260811 18:57:42.239317 4162 tablet_bootstrap.cc:654] T 4e0747ef88574c68ae074732cb7095b9 P bd8c4bdbd811417ab011a7131f17d283: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.241400 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 735d4fc7df7f4157ae7df534ab4833ce (DEFAULT_TABLE table=default.Table_fb4433f8104245af96d1746998f92ba1 [id=7e347067e6d049bca74fe91eff329a26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.245698 3852 tablet_service.cc:1467] Processing CreateTablet for tablet 69aa5092d0b94654a64f033cf7ae5ee1 (DEFAULT_TABLE table=default.Table_da206548a6a34eb78f9b4bddfc58d921 [id=9af5a612c8df473db99778c812035966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.246838 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 5387f5c3f2af4b009fad01c936639e1a (DEFAULT_TABLE table=default.Table_0b6fac836f9146ab9a5700100e2bda0f [id=693fe88a8ea9492b986b8e1d42bc9c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.248762 3548 catalog_manager.cc:5495] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:42.249311 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet e1af33fd98c6486f975368ec90493db7 (table default.Table_22bd9bbe66b64e209048c8164f116ba7 [id=7736472828a3499c8ad8026e164b9759]). Expected version 3 got 0
I20260811 18:57:42.249837 3788 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:42.250227 3824 leader_election.cc:334] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.250541 3824 leader_election.cc:302] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:42.250973 5630 raft_consensus.cc:2726] T ffa8a4a1c0cb4834be501bdd7d468472 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:42.250836 3548 catalog_manager.cc:5495] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.251888 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 38ed9c1542c24b2ea3c2de8ec0f589d6 (table default.Table_c71b8760db044bad851b4bcd819bf75f [id=7a869d9623bc4e07938b2d26688723a5]). Expected version 2 got 0
I20260811 18:57:42.252735 3548 catalog_manager.cc:5495] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.253535 3824 leader_election.cc:302] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24, bd8c4bdbd811417ab011a7131f17d283; no voters:
I20260811 18:57:42.253786 5768 raft_consensus.cc:2781] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:42.253930 5768 raft_consensus.cc:686] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Becoming Leader. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Running, Role: LEADER
I20260811 18:57:42.253564 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 8dc07c28ccfd48ee9c33e49442c6047b (table default.Table_710bf71d5383421ca80e2caac75ec12f [id=cabc6cc20754488eb152c0d4f686dce0]). Expected version 3 got 0
I20260811 18:57:42.254235 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_9ec52027b1c944a69e731fed8ad2bd30 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 3 got 0
I20260811 18:57:42.254145 5768 consensus_queue.cc:237] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.254590 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet fb3a9fdc3d594c1a84a5d8efa8490dfa (table default.Table_9194817120074797a6f60bf8e24c0636 [id=1323bd99594a4382ba121cc11e79de31]). Expected version 1 got 0
I20260811 18:57:42.254866 3548 catalog_manager.cc:5495] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.255254 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet c0b355a0d3984012be4e9406998fd4a0 (table default.Table_0f2efce46eed4e0b9626c257cbfe711c [id=aa5ff7c0d063424992f9dc22c89aa620]). Expected version 2 got 0
I20260811 18:57:42.255451 3548 catalog_manager.cc:5495] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.255843 3548 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet a811d4488d0e4541808a5e1aa44d663b (table default.Table_fb8dd203bf3941fbacfd7dad1418e566 [id=a71b71af19a44182aea7e742062558ca]). Expected version 1 got 0
I20260811 18:57:42.260427 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 735d4fc7df7f4157ae7df534ab4833ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.269956 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe48bbab3d7e4f7591883296a9996db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.270610 5593 raft_consensus.cc:482] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.272871 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79539e937f81437ab3b12b8465f12d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.272790 5593 raft_consensus.cc:504] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.273403 5593 leader_election.cc:288] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:42.273610 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5387f5c3f2af4b009fad01c936639e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.276957 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69aa5092d0b94654a64f033cf7ae5ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.283500 4169 tablet_replica.cc:330] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.284436 4169 raft_consensus.cc:2229] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.283878 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc68b2b47ca4426a204df0dea3801f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.298121 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5852830ee0544f38aa6ed8c8c2e250a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.300282 3824 leader_election.cc:334] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.300601 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c575ba3060a140759484bea1462ffd73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.302580 4169 raft_consensus.cc:2258] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.306934 4169 ts_tablet_manager.cc:1905] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.332989 3541 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.72.65:32817, user_credentials={real_user=slave}} blocked reactor thread for 403943us
W20260811 18:57:42.333355 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet b50efab3c782447cad8e45b1fa51aa1f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b50efab3c782447cad8e45b1fa51aa1f already in progress: creating tablet
I20260811 18:57:42.333583 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet a0116997a1c64e658045c319f1c45861 (table default.Table_704e71bf00da49609103fb3fde6afb2c [id=d053378264aa420c9ac90a976818638c]) successfully deleted
I20260811 18:57:42.333770 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 8d1e244addf5462499fcadaa3332cba1 (table default.Table_ef613c8dc0524f318790523ba7652317 [id=4c7bb5bc50f343808fd588924c0e012e]) successfully deleted
W20260811 18:57:42.333938 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet bf1c037a37504b3fad9612f5a734dd33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf1c037a37504b3fad9612f5a734dd33 already in progress: creating tablet
W20260811 18:57:42.315110 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.316095 5630 raft_consensus.cc:482] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.339572 5630 raft_consensus.cc:504] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.340413 5630 leader_election.cc:288] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.342175 3824 consensus_peers.cc:487] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.346025 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 0bc2a90633b840c5be8d8dab46ad6768 (DEFAULT_TABLE table=default.Table_a4a5dc90bb3449ee94760cc397fd49f3 [id=bb87391ac9a3420daff021b5d24e3c54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.346065 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 586b96fbf0344a85a884263e7316a147 (DEFAULT_TABLE table=default.Table_2e44f4b4162648d6911b8a00b8fd7ba6 [id=358ae5b14b774ccf993af80443f75b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.346555 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bc2a90633b840c5be8d8dab46ad6768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.346781 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586b96fbf0344a85a884263e7316a147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.350378 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 531bb240b3af460e832429cc8397a8e2 (DEFAULT_TABLE table=default.Table_343b8634c5a946feacafd49b9aba3c62 [id=dfbfcbc7dfe64bd997058149f61eeb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.350792 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 531bb240b3af460e832429cc8397a8e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.360031 4169 ts_tablet_manager.cc:1918] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.360268 4169 log.cc:1196] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/059b325df1db4fde9d4b26b559121316
I20260811 18:57:42.360715 4169 ts_tablet_manager.cc:1939] T 059b325df1db4fde9d4b26b559121316 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.361078 3547 catalog_manager.cc:5495] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66). New cstate: current_term: 1 leader_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:42.361639 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1081) took 5129 ms. Trace:
I20260811 18:57:42.361701 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet d73a43558dbb48889ab205108f4ce6be (table default.Table_9ec52027b1c944a69e731fed8ad2bd30 [id=d818dd73be204b458c789ff1d10f7b97]). Expected version 3 got 0
I20260811 18:57:42.361757 4169 rpcz_store.cc:276] 0811 18:57:37.232437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.259911 (+ 27474us) service_pool.cc:227] Handling call
0811 18:57:42.216037 (+4956126us) ts_tablet_manager.cc:1145] Deleting tablet 059b325df1db4fde9d4b26b559121316
0811 18:57:42.216049 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.336481 (+120432us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.344806 (+ 8325us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.361617 (+ 16811us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4955906}
I20260811 18:57:42.362254 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1098) took 4929 ms. Trace:
I20260811 18:57:42.362370 4169 rpcz_store.cc:276] 0811 18:57:37.433068 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.433495 (+ 427us) service_pool.cc:227] Handling call
0811 18:57:42.362082 (+4928587us) ts_tablet_manager.cc:1145] Deleting tablet 7f4e1fc5686944dfa3eb57702c810eb8
0811 18:57:42.362098 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.362234 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4928349}
I20260811 18:57:42.362743 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1099) took 4929 ms. Trace:
I20260811 18:57:42.362828 4169 rpcz_store.cc:276] 0811 18:57:37.433142 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.436742 (+ 3600us) service_pool.cc:227] Handling call
0811 18:57:42.362613 (+4925871us) ts_tablet_manager.cc:1145] Deleting tablet ea579c33f8b14e9fa7fa389ce7f45931
0811 18:57:42.362624 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.362727 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4925680}
I20260811 18:57:42.363101 4169 tablet_replica.cc:330] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.363328 4169 raft_consensus.cc:2229] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.363497 4169 raft_consensus.cc:2258] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.364070 4169 ts_tablet_manager.cc:1905] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.373888 4169 ts_tablet_manager.cc:1918] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.374074 4169 log.cc:1196] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/2c0f6df7a291456d80538e2b76417873
I20260811 18:57:42.374487 4169 ts_tablet_manager.cc:1939] T 2c0f6df7a291456d80538e2b76417873 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.375367 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1100) took 4938 ms. Trace:
I20260811 18:57:42.375476 4169 rpcz_store.cc:276] 0811 18:57:37.437115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.443580 (+ 6465us) service_pool.cc:227] Handling call
0811 18:57:42.363055 (+4919475us) ts_tablet_manager.cc:1145] Deleting tablet 2c0f6df7a291456d80538e2b76417873
0811 18:57:42.363066 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.370581 (+ 7515us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.373766 (+ 3185us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.375340 (+ 1574us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4919240}
I20260811 18:57:42.375934 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1112) took 4890 ms. Trace:
I20260811 18:57:42.376030 4169 rpcz_store.cc:276] 0811 18:57:37.485733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.485810 (+ 77us) service_pool.cc:227] Handling call
0811 18:57:42.375782 (+4889972us) ts_tablet_manager.cc:1145] Deleting tablet 2f2f5ce2b1d14a9a988cff644e196088
0811 18:57:42.375795 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.375914 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4889709}
I20260811 18:57:42.376411 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1111) took 4890 ms. Trace:
W20260811 18:57:42.378671 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet a7946503bec9446d8deb02448eb231d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7946503bec9446d8deb02448eb231d5 already in progress: creating tablet
I20260811 18:57:42.379032 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 4059e5907b1e488ab9e1d04e563f96cd (table default.Table_e5769978bc1148a7a53c0df338266d1e [id=07bc4a05b4fb49a6883635270832afe9]) successfully deleted
I20260811 18:57:42.379062 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 24c27902a7ba4289b794f6f136897272 (DEFAULT_TABLE table=default.Table_8a306b155dd94cfaa6649e14665e9418 [id=063614ea03ba403890d5ab712daf00ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.379302 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 54ee342f3b2b40a7a194b16b07f7d525 (DEFAULT_TABLE table=default.Table_93d80fcaa1a646119a16dcb3e546b426 [id=156f7d5e97f2469d8b286865f6812105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.379281 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet 0540e4e43ea34fbfbc66b6035c60684d (table default.Table_a80fc1f336804bea9495ec0bce4bd47a [id=340eb5c96f65473dbde9d2b38fd34f54]) successfully deleted
I20260811 18:57:42.379534 3541 catalog_manager.cc:4841] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): tablet b203df4124364737839bf4c1ae072988 (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
I20260811 18:57:42.379645 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24c27902a7ba4289b794f6f136897272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.379897 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ee342f3b2b40a7a194b16b07f7d525. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.379972 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 76e71af5ee8b463eb158ac9ae76ff252 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76e71af5ee8b463eb158ac9ae76ff252 already in progress: creating tablet
W20260811 18:57:42.380159 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet ae96e10a2ed4407c9d42a359d82d389e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae96e10a2ed4407c9d42a359d82d389e already in progress: creating tablet
W20260811 18:57:42.380337 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet dfb6ac49f1d4403dad38a3125264088a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dfb6ac49f1d4403dad38a3125264088a already in progress: creating tablet
W20260811 18:57:42.380568 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet eea325ca357146fcaa408d87803da349 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eea325ca357146fcaa408d87803da349 already in progress: creating tablet
W20260811 18:57:42.380719 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet 58bc967770de4ed486b638fb6b174db8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 58bc967770de4ed486b638fb6b174db8 already in progress: creating tablet
W20260811 18:57:42.380851 3541 catalog_manager.cc:4821] TS bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): delete failed for tablet c44eee516d634d9a986107b55410dfc3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c44eee516d634d9a986107b55410dfc3 already in progress: creating tablet
I20260811 18:57:42.379063 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6bb0873b4c44cca31c1b1942501358 (DEFAULT_TABLE table=default.Table_388bf7f2862e42ebb5e3a779b1ade554 [id=0598c1530ed24c9fa1f29ceb8b3f46ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.388510 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 059b325df1db4fde9d4b26b559121316 (table default.Table_de899ce498b24cd5b270adf6fbb81b92 [id=ea3b710ab4bc48dda6cbef6e7f6b2bed]) successfully deleted
W20260811 18:57:42.388821 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 7f4e1fc5686944dfa3eb57702c810eb8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f4e1fc5686944dfa3eb57702c810eb8 already in progress: creating tablet
W20260811 18:57:42.389026 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ea579c33f8b14e9fa7fa389ce7f45931 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea579c33f8b14e9fa7fa389ce7f45931 already in progress: creating tablet
I20260811 18:57:42.387162 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 1012e838b09043bba433f742a3084d61 (DEFAULT_TABLE table=default.Table_b5f718e0dd6347099abbd1de1d43ca73 [id=8808c040289440708d0175c9ac3ebe16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.386301 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6bb0873b4c44cca31c1b1942501358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.389346 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 2c0f6df7a291456d80538e2b76417873 (table default.Table_12663a37e9cf40cf8d0c9950d6a729c1 [id=4edd5515cdb74cc0aa17eba93289499a]) successfully deleted
W20260811 18:57:42.389532 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 2f2f5ce2b1d14a9a988cff644e196088 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2f2f5ce2b1d14a9a988cff644e196088 already in progress: creating tablet
I20260811 18:57:42.389653 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1012e838b09043bba433f742a3084d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.388396 3863 tablet_service.cc:1467] Processing CreateTablet for tablet 163e9cc026d0435a835c55cc760ec5f8 (DEFAULT_TABLE table=default.Table_f20cec778f3248419eb734a5100ae8d1 [id=dbafd5502cb944dba9b18f4e5d863044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.394877 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 163e9cc026d0435a835c55cc760ec5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.388991 3851 tablet_service.cc:1467] Processing CreateTablet for tablet e506527756a344498bc63b31815a2d2d (DEFAULT_TABLE table=default.Table_a49eeecb6ae34fe28d40847ab1738944 [id=6e5a954d5dcb4ad1b5564959496a3f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.396992 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e506527756a344498bc63b31815a2d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.376502 4169 rpcz_store.cc:276] 0811 18:57:37.485617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486402 (+ 785us) service_pool.cc:227] Handling call
0811 18:57:42.376304 (+4889902us) ts_tablet_manager.cc:1145] Deleting tablet d21973bc8e514a089c2beb24a686060f
0811 18:57:42.376314 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.376394 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4889742}
I20260811 18:57:42.401420 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1113) took 4915 ms. Trace:
I20260811 18:57:42.401561 4169 rpcz_store.cc:276] 0811 18:57:37.486415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486469 (+ 54us) service_pool.cc:227] Handling call
0811 18:57:42.401281 (+4914812us) ts_tablet_manager.cc:1145] Deleting tablet f2b66196716a4ba3893b6b8bfec2b8c0
0811 18:57:42.401294 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.401399 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4914598}
I20260811 18:57:42.401998 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1114) took 4915 ms. Trace:
I20260811 18:57:42.402137 4169 rpcz_store.cc:276] 0811 18:57:37.486832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.486875 (+ 43us) service_pool.cc:227] Handling call
0811 18:57:42.401891 (+4915016us) ts_tablet_manager.cc:1145] Deleting tablet a3973dfdaac746a6af7a402277d2711b
0811 18:57:42.401902 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.401982 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4914831}
I20260811 18:57:42.402559 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1117) took 4913 ms. Trace:
I20260811 18:57:42.402688 4169 rpcz_store.cc:276] 0811 18:57:37.488700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.488796 (+ 96us) service_pool.cc:227] Handling call
0811 18:57:42.402458 (+4913662us) ts_tablet_manager.cc:1145] Deleting tablet 15dd14e2b4bd4a08b5b08f532a55291b
0811 18:57:42.402467 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.402542 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4913471}
I20260811 18:57:42.403088 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1131) took 4911 ms. Trace:
I20260811 18:57:42.403215 4169 rpcz_store.cc:276] 0811 18:57:37.491695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.780636 (+288941us) service_pool.cc:227] Handling call
0811 18:57:42.402956 (+4622320us) ts_tablet_manager.cc:1145] Deleting tablet 0495c5b770b34d54afb1379a6e36204c
0811 18:57:42.402966 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.403054 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4608038}
I20260811 18:57:42.403607 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1132) took 4911 ms. Trace:
I20260811 18:57:42.403728 4169 rpcz_store.cc:276] 0811 18:57:37.491768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.784392 (+292624us) service_pool.cc:227] Handling call
0811 18:57:42.403498 (+4619106us) ts_tablet_manager.cc:1145] Deleting tablet 75ceba7bfc8240268b16c789f07813d1
0811 18:57:42.403508 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.403589 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4585944}
I20260811 18:57:42.404168 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1133) took 4912 ms. Trace:
I20260811 18:57:42.404301 4169 rpcz_store.cc:276] 0811 18:57:37.491840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.785280 (+293440us) service_pool.cc:227] Handling call
0811 18:57:42.404003 (+4618723us) ts_tablet_manager.cc:1145] Deleting tablet 5a75a480f38047d880b0c1fa51403786
0811 18:57:42.404024 (+ 21us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.404146 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":405,"tablet-delete.queue_time_us":4586447}
I20260811 18:57:42.404727 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1134) took 4912 ms. Trace:
I20260811 18:57:42.404852 4169 rpcz_store.cc:276] 0811 18:57:37.491905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.787870 (+295965us) service_pool.cc:227] Handling call
0811 18:57:42.404603 (+4616733us) ts_tablet_manager.cc:1145] Deleting tablet d32212c817c94fad8379462b027e7d3d
0811 18:57:42.404616 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.404703 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4572213}
I20260811 18:57:42.405264 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1147) took 4523 ms. Trace:
I20260811 18:57:42.405386 4169 rpcz_store.cc:276] 0811 18:57:37.881847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882287 (+ 440us) service_pool.cc:227] Handling call
0811 18:57:42.405132 (+4522845us) ts_tablet_manager.cc:1145] Deleting tablet 3195082643264e0d90f70e1007ab9b24
0811 18:57:42.405143 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.405239 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4483795}
I20260811 18:57:42.405694 4169 tablet_replica.cc:330] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.405890 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet 42405aec21e3405486b6c658438b5fb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.405962 4169 raft_consensus.cc:2229] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.406193 4169 raft_consensus.cc:2258] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.406847 4169 ts_tablet_manager.cc:1905] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.411499 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d21973bc8e514a089c2beb24a686060f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d21973bc8e514a089c2beb24a686060f already in progress: creating tablet
W20260811 18:57:42.411775 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f2b66196716a4ba3893b6b8bfec2b8c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2b66196716a4ba3893b6b8bfec2b8c0 already in progress: creating tablet
I20260811 18:57:42.409686 3853 tablet_service.cc:1467] Processing CreateTablet for tablet c400808b5240455d8bfd8980b943ad01 (DEFAULT_TABLE table=default.Table_194b336657844c34ac93e94c90db9444 [id=62d6b6510702407b92cfe27215761113]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:42.411963 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a3973dfdaac746a6af7a402277d2711b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3973dfdaac746a6af7a402277d2711b already in progress: creating tablet
W20260811 18:57:42.412132 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 15dd14e2b4bd4a08b5b08f532a55291b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15dd14e2b4bd4a08b5b08f532a55291b already in progress: creating tablet
I20260811 18:57:42.412293 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c400808b5240455d8bfd8980b943ad01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 18:57:42.412309 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0495c5b770b34d54afb1379a6e36204c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0495c5b770b34d54afb1379a6e36204c already in progress: creating tablet
W20260811 18:57:42.412457 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 75ceba7bfc8240268b16c789f07813d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75ceba7bfc8240268b16c789f07813d1 already in progress: creating tablet
W20260811 18:57:42.412621 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 5a75a480f38047d880b0c1fa51403786 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a75a480f38047d880b0c1fa51403786 already in progress: creating tablet
W20260811 18:57:42.412798 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d32212c817c94fad8379462b027e7d3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d32212c817c94fad8379462b027e7d3d already in progress: creating tablet
W20260811 18:57:42.412941 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 3195082643264e0d90f70e1007ab9b24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3195082643264e0d90f70e1007ab9b24 already in progress: creating tablet
I20260811 18:57:42.417354 4169 ts_tablet_manager.cc:1918] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260811 18:57:42.417930 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
W20260811 18:57:42.430086 3824 consensus_peers.cc:487] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.430169 4169 log.cc:1196] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/5d495354c0994fb4a74c376078e41a26
I20260811 18:57:42.430819 4169 ts_tablet_manager.cc:1939] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.431466 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1148) took 4548 ms. Trace:
I20260811 18:57:42.431705 4169 rpcz_store.cc:276] 0811 18:57:37.882582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882630 (+ 48us) service_pool.cc:227] Handling call
0811 18:57:42.405649 (+4523019us) ts_tablet_manager.cc:1145] Deleting tablet 5d495354c0994fb4a74c376078e41a26
0811 18:57:42.405658 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.414140 (+ 8482us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.417263 (+ 3123us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.431444 (+ 14181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4484232}
I20260811 18:57:42.432246 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 5d495354c0994fb4a74c376078e41a26 (table default.Table_fdfd955ae68f4bf09dea8635a00bc3b1 [id=e370f90746264263837d433bb35f2c21]) successfully deleted
I20260811 18:57:42.432313 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1149) took 4549 ms. Trace:
I20260811 18:57:42.432406 4169 rpcz_store.cc:276] 0811 18:57:37.882860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.882904 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:42.432163 (+4549259us) ts_tablet_manager.cc:1145] Deleting tablet b7b38d3b9570467a9128bd7fe73acf2f
0811 18:57:42.432179 (+ 16us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.432293 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4510691}
I20260811 18:57:42.432998 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1150) took 4549 ms. Trace:
W20260811 18:57:42.433050 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b7b38d3b9570467a9128bd7fe73acf2f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b38d3b9570467a9128bd7fe73acf2f already in progress: creating tablet
I20260811 18:57:42.433156 4169 rpcz_store.cc:276] 0811 18:57:37.883098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883139 (+ 41us) service_pool.cc:227] Handling call
0811 18:57:42.432882 (+4549743us) ts_tablet_manager.cc:1145] Deleting tablet 429eb5824f3f4b2b8da6e17ba7c69b92
0811 18:57:42.432894 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.432981 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4511362}
W20260811 18:57:42.433568 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 429eb5824f3f4b2b8da6e17ba7c69b92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 429eb5824f3f4b2b8da6e17ba7c69b92 already in progress: creating tablet
I20260811 18:57:42.433597 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1151) took 4550 ms. Trace:
I20260811 18:57:42.433681 4169 rpcz_store.cc:276] 0811 18:57:37.883320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883387 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:42.433487 (+4550100us) ts_tablet_manager.cc:1145] Deleting tablet f89eebbc40234794a081734f834294ab
0811 18:57:42.433500 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.433579 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4511913}
W20260811 18:57:42.434123 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f89eebbc40234794a081734f834294ab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f89eebbc40234794a081734f834294ab already in progress: creating tablet
I20260811 18:57:42.434137 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1152) took 4550 ms. Trace:
I20260811 18:57:42.434218 4169 rpcz_store.cc:276] 0811 18:57:37.883429 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:37.883568 (+ 139us) service_pool.cc:227] Handling call
0811 18:57:42.434020 (+4550452us) ts_tablet_manager.cc:1145] Deleting tablet a5bbd3274980470b9fe71457b49bdd62
0811 18:57:42.434031 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.434120 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4512371}
I20260811 18:57:42.434834 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1159) took 4259 ms. Trace:
W20260811 18:57:42.434845 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet a5bbd3274980470b9fe71457b49bdd62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a5bbd3274980470b9fe71457b49bdd62 already in progress: creating tablet
I20260811 18:57:42.438097 4169 rpcz_store.cc:276] 0811 18:57:38.175718 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.176437 (+ 719us) service_pool.cc:227] Handling call
0811 18:57:42.434696 (+4258259us) ts_tablet_manager.cc:1145] Deleting tablet bfea0cd619024204b9bff1e2dd62b7b0
0811 18:57:42.434707 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.434817 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4257989}
W20260811 18:57:42.439379 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bfea0cd619024204b9bff1e2dd62b7b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfea0cd619024204b9bff1e2dd62b7b0 already in progress: creating tablet
I20260811 18:57:42.439507 4169 tablet_replica.cc:330] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.439751 4169 raft_consensus.cc:2229] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:42.439913 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet c6852aaef1d5486b87e5462eae532719 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.439944 4169 raft_consensus.cc:2258] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.440572 4169 ts_tablet_manager.cc:1905] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.429311 4163 raft_consensus.cc:729] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.450635 4163 consensus_queue.cc:260] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.454237 4163 ts_tablet_manager.cc:1428] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.329s user 0.022s sys 0.000s
I20260811 18:57:42.454581 4163 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.456033 4163 tablet_bootstrap.cc:654] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.457226 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8f5b87062349c4b19ec99fa0175f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.472922 4169 ts_tablet_manager.cc:1918] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.478875 4169 log.cc:1196] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/ce94d37efd2245bf8cc71fdd90bd9735
I20260811 18:57:42.481096 4169 ts_tablet_manager.cc:1939] T ce94d37efd2245bf8cc71fdd90bd9735 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.483975 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1156) took 4311 ms. Trace:
I20260811 18:57:42.486773 4169 rpcz_store.cc:276] 0811 18:57:38.172816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.173076 (+ 260us) service_pool.cc:227] Handling call
0811 18:57:42.439454 (+4266378us) ts_tablet_manager.cc:1145] Deleting tablet ce94d37efd2245bf8cc71fdd90bd9735
0811 18:57:42.439466 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.460835 (+ 21369us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.472806 (+ 11971us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.483946 (+ 11140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4253002}
I20260811 18:57:42.488235 4163 tablet_bootstrap.cc:492] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.488386 4163 ts_tablet_manager.cc:1397] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260811 18:57:42.488818 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet ce94d37efd2245bf8cc71fdd90bd9735 (table default.Table_b48fafe85a504185911a1e7edb4fcbfd [id=647a9faebd8d4471bf36eaa35f9f34ae]) successfully deleted
I20260811 18:57:42.489097 4163 raft_consensus.cc:348] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.489432 4163 raft_consensus.cc:374] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.489519 4163 raft_consensus.cc:729] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.489806 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1157) took 4316 ms. Trace:
I20260811 18:57:42.491706 4169 rpcz_store.cc:276] 0811 18:57:38.173320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.175314 (+ 1994us) service_pool.cc:227] Handling call
0811 18:57:42.489363 (+4314049us) ts_tablet_manager.cc:1145] Deleting tablet 6710084f21734dea9c7ce1f43d1bf587
0811 18:57:42.489381 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.489768 (+ 387us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4302808}
W20260811 18:57:42.492507 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 6710084f21734dea9c7ce1f43d1bf587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6710084f21734dea9c7ce1f43d1bf587 already in progress: creating tablet
I20260811 18:57:42.493196 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1158) took 4317 ms. Trace:
I20260811 18:57:42.489856 4163 consensus_queue.cc:260] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.495868 4169 rpcz_store.cc:276] 0811 18:57:38.175603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.175667 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:42.492987 (+4317320us) ts_tablet_manager.cc:1145] Deleting tablet e9f0e443ab84424dac307b745b496986
0811 18:57:42.493005 (+ 18us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.493167 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4306388}
I20260811 18:57:42.496347 4163 ts_tablet_manager.cc:1428] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
W20260811 18:57:42.496583 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet e9f0e443ab84424dac307b745b496986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9f0e443ab84424dac307b745b496986 already in progress: creating tablet
I20260811 18:57:42.497149 4163 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.498606 4163 tablet_bootstrap.cc:654] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.509779 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1160) took 4332 ms. Trace:
I20260811 18:57:42.510025 4169 rpcz_store.cc:276] 0811 18:57:38.176860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.176904 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:42.496984 (+4320080us) ts_tablet_manager.cc:1145] Deleting tablet 69059a0667234a408cc1c09585df63bc
0811 18:57:42.509632 (+ 12648us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.509755 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4308070}
I20260811 18:57:42.510468 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e24f8d4522466d81605ea37fa7064a with delete_type TABLET_DATA_DELETED (replaced by tablet c17847a358e544b4b4e51f65463a9679) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:42.510674 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 69059a0667234a408cc1c09585df63bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69059a0667234a408cc1c09585df63bc already in progress: creating tablet
I20260811 18:57:42.511119 4169 tablet_replica.cc:330] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.511474 4169 raft_consensus.cc:2229] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.511721 4169 raft_consensus.cc:2258] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.512411 4169 ts_tablet_manager.cc:1905] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.522080 4163 tablet_bootstrap.cc:492] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.522253 4163 ts_tablet_manager.cc:1397] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.005s
I20260811 18:57:42.523033 4163 raft_consensus.cc:348] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.523257 4163 raft_consensus.cc:374] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.523303 4163 raft_consensus.cc:729] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.523475 4163 consensus_queue.cc:260] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.524003 4163 ts_tablet_manager.cc:1428] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.524250 4163 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.525916 4163 tablet_bootstrap.cc:654] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.543344 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet 8304ac7709ca4392baec4cfa78ef5b41 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c81850cdc74b4ab727bd2279162477) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.548653 4163 tablet_bootstrap.cc:492] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.548823 4163 ts_tablet_manager.cc:1397] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.000s
I20260811 18:57:42.549618 4163 raft_consensus.cc:348] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.549867 4163 raft_consensus.cc:374] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.549921 4163 raft_consensus.cc:729] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.550186 4163 consensus_queue.cc:260] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.559468 4163 ts_tablet_manager.cc:1428] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:42.565944 4163 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.568301 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet d37e81d376e14907b13746cca831a96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:34 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.571682 4169 ts_tablet_manager.cc:1918] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.571831 4169 log.cc:1196] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/c99ceca57cb245f4a3a6cb4b36f1306d
I20260811 18:57:42.578820 4169 ts_tablet_manager.cc:1939] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.579334 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1161) took 4390 ms. Trace:
I20260811 18:57:42.579443 4169 rpcz_store.cc:276] 0811 18:57:38.189107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.190677 (+ 1570us) service_pool.cc:227] Handling call
0811 18:57:42.511068 (+4320391us) ts_tablet_manager.cc:1145] Deleting tablet c99ceca57cb245f4a3a6cb4b36f1306d
0811 18:57:42.511079 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.515635 (+ 4556us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.571553 (+ 55918us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.579313 (+ 7760us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4320192}
I20260811 18:57:42.580016 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet c99ceca57cb245f4a3a6cb4b36f1306d (table default.Table_8a27cf06565547a39a8eee2420c9a31b [id=0a4e4826c2cf4250a34e7e26eece8e3a]) successfully deleted
I20260811 18:57:42.580374 4169 tablet_replica.cc:330] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.580644 4169 raft_consensus.cc:2229] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.577244 4163 tablet_bootstrap.cc:654] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.580829 4169 raft_consensus.cc:2258] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.582808 4169 ts_tablet_manager.cc:1905] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.591742 4169 ts_tablet_manager.cc:1918] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.591898 4169 log.cc:1196] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7b404e99e9414c67b3cd5aff12203a53
I20260811 18:57:42.592206 4169 ts_tablet_manager.cc:1939] T 7b404e99e9414c67b3cd5aff12203a53 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.593073 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1162) took 4398 ms. Trace:
I20260811 18:57:42.593185 4169 rpcz_store.cc:276] 0811 18:57:38.195030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.195193 (+ 163us) service_pool.cc:227] Handling call
0811 18:57:42.579867 (+4384674us) ts_tablet_manager.cc:1145] Deleting tablet 7b404e99e9414c67b3cd5aff12203a53
0811 18:57:42.579879 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.588407 (+ 8528us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.591620 (+ 3213us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.593045 (+ 1425us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4384474}
I20260811 18:57:42.593729 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7b404e99e9414c67b3cd5aff12203a53 (table default.Table_d8397de033b74ce9accaa4085f59d7e6 [id=bebb3efbbc56416597abdf4a0b3952ed]) successfully deleted
I20260811 18:57:42.597200 4163 tablet_bootstrap.cc:492] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.597352 4163 ts_tablet_manager.cc:1397] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.000s
I20260811 18:57:42.598083 4163 raft_consensus.cc:348] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.598321 4163 raft_consensus.cc:374] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.604180 4163 raft_consensus.cc:729] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.604547 4163 consensus_queue.cc:260] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.605719 4163 ts_tablet_manager.cc:1428] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260811 18:57:42.606237 4163 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.607964 4163 tablet_bootstrap.cc:654] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.612079 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1164) took 4410 ms. Trace:
I20260811 18:57:42.615387 3851 tablet_service.cc:1514] Processing DeleteTablet for tablet b4dc2df4c6464d71bb13fe25e0046758 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:28 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.615346 4169 rpcz_store.cc:276] 0811 18:57:38.201782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.202210 (+ 428us) service_pool.cc:227] Handling call
0811 18:57:42.596378 (+4394168us) ts_tablet_manager.cc:1145] Deleting tablet d11b889e9d044a049d0586382c263951
0811 18:57:42.596387 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.612049 (+ 15662us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4385918}
W20260811 18:57:42.616004 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d11b889e9d044a049d0586382c263951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d11b889e9d044a049d0586382c263951 already in progress: creating tablet
I20260811 18:57:42.616715 4169 tablet_replica.cc:330] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.616995 4169 raft_consensus.cc:2229] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.617293 4169 raft_consensus.cc:2258] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.618155 4169 ts_tablet_manager.cc:1905] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.620146 5630 raft_consensus.cc:482] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.620289 5630 raft_consensus.cc:504] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.620970 5630 leader_election.cc:288] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.623260 3824 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.648000 4169 ts_tablet_manager.cc:1918] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.649758 4169 log.cc:1196] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0cccebfc54484a9b9abffcce4e459fb5
I20260811 18:57:42.650782 4169 ts_tablet_manager.cc:1939] T 0cccebfc54484a9b9abffcce4e459fb5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.651908 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1165) took 4450 ms. Trace:
I20260811 18:57:42.653110 4169 rpcz_store.cc:276] 0811 18:57:38.201889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.214685 (+ 12796us) service_pool.cc:227] Handling call
0811 18:57:42.616395 (+4401710us) ts_tablet_manager.cc:1145] Deleting tablet 0cccebfc54484a9b9abffcce4e459fb5
0811 18:57:42.616662 (+ 267us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.637932 (+ 21270us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.647876 (+ 9944us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.651872 (+ 3996us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4401458}
I20260811 18:57:42.654022 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0cccebfc54484a9b9abffcce4e459fb5 (table default.Table_d88f576cc3ca4c6eadc6e071aacc15b2 [id=5fc5ade3f38c438d9b501127ac866b88]) successfully deleted
I20260811 18:57:42.654731 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1163) took 4453 ms. Trace:
I20260811 18:57:42.655313 4169 rpcz_store.cc:276] 0811 18:57:38.201675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.214985 (+ 13310us) service_pool.cc:227] Handling call
0811 18:57:42.654536 (+4439551us) ts_tablet_manager.cc:1145] Deleting tablet 82b04e37fc264fc08e0ed9517356a17d
0811 18:57:42.654553 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.654705 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4439360}
W20260811 18:57:42.656122 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 82b04e37fc264fc08e0ed9517356a17d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82b04e37fc264fc08e0ed9517356a17d already in progress: creating tablet
I20260811 18:57:42.656631 4169 tablet_replica.cc:330] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.657080 4169 raft_consensus.cc:2229] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.657549 4169 raft_consensus.cc:2258] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.659209 4169 ts_tablet_manager.cc:1905] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.664034 4163 tablet_bootstrap.cc:492] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.664291 4163 ts_tablet_manager.cc:1397] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.058s user 0.014s sys 0.004s
I20260811 18:57:42.665441 4163 raft_consensus.cc:348] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.665882 4163 raft_consensus.cc:374] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.666947 4163 raft_consensus.cc:729] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.667384 4163 consensus_queue.cc:260] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.683480 5630 raft_consensus.cc:482] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.683712 5630 raft_consensus.cc:504] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.684626 5630 leader_election.cc:288] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.687387 3824 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.688678 4163 ts_tablet_manager.cc:1428] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
W20260811 18:57:42.692883 3824 consensus_peers.cc:487] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.701792 4163 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.704968 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:42.707319 3851 tablet_service.cc:1467] Processing CreateTablet for tablet 220ea85575904e0ea8fdf7799c9ddeed (DEFAULT_TABLE table=default.Table_bef031fd83e44076810e019e0038f70f [id=e216b4160f2a4e40988252fe28960cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.707803 3851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220ea85575904e0ea8fdf7799c9ddeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.709299 4163 tablet_bootstrap.cc:654] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.711007 3853 tablet_service.cc:1467] Processing CreateTablet for tablet 88f35da2dc8a43fc86f9a12958e3be1b (DEFAULT_TABLE table=default.Table_b149c08055014e08aad5ad5be387047b [id=a2113616a0b445549c700f7e1c707191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711114 3863 tablet_service.cc:1467] Processing CreateTablet for tablet ac974915a88b41c59d7613d4a69f62c2 (DEFAULT_TABLE table=default.Table_65f6fe5195b443fcbd5f077f1a147107 [id=f93e0ff8e13f48739183e1dc3fc3a66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711277 3856 tablet_service.cc:1467] Processing CreateTablet for tablet 73880e575a4845658fe1766aa9715c9f (DEFAULT_TABLE table=default.Table_beb35f174ef644f5994dd0282f0d4ee3 [id=82c9ae34f67544de859f6c7ffa7e7dd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.711385 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f35da2dc8a43fc86f9a12958e3be1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.711540 3863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac974915a88b41c59d7613d4a69f62c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.711758 3852 tablet_service.cc:1467] Processing CreateTablet for tablet f0b07f91bb794ad88326db69d154c173 (DEFAULT_TABLE table=default.Table_525108ad39d242bc818b5bb86a5c0103 [id=1b6f82af31cb46bfb9a295a323479be9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.712071 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b07f91bb794ad88326db69d154c173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.713598 3856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73880e575a4845658fe1766aa9715c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.714630 3862 tablet_service.cc:1467] Processing CreateTablet for tablet 8300d38bad3244eb9bc4cc20bf4eb4d2 (DEFAULT_TABLE table=default.Table_6f3b91b31a884708a27160ebfca75cab [id=c83e220d78e54a8e8291795ba64f94f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.714967 3862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8300d38bad3244eb9bc4cc20bf4eb4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.715736 3867 tablet_service.cc:1467] Processing CreateTablet for tablet 0efae9cb481a435c8a4511fd6d1fe415 (DEFAULT_TABLE table=default.Table_88ee0abdcd9943568a07d73017ff598a [id=06b70587cd6d417a8ef0b3ec4ecc7c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.716068 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0efae9cb481a435c8a4511fd6d1fe415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.718983 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 730a2aed5a1344b387398114f6105d3f (DEFAULT_TABLE table=default.Table_3aa1c735a06c46e9833fdb958cac274d [id=0916d009f52243bba02b715768a244f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.719501 3859 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6d41c0dd71408e89231ff56fde6e29 (DEFAULT_TABLE table=default.Table_22ffcd40375b45aab1616fa1fb6f030d [id=73fc4e78208d4494b0c0e9a1150c0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.719944 3865 tablet_service.cc:1467] Processing CreateTablet for tablet 332d3d9ddb224a4697809b19b622749d (DEFAULT_TABLE table=default.Table_d86b05dec10e4f3da6b9c48860fe6d88 [id=c6864a249c234170a0e6183e4668703c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.720898 3864 tablet_service.cc:1467] Processing CreateTablet for tablet f336fdd55965432f83b2d1c436fdf1fa (DEFAULT_TABLE table=default.Table_a72e17b47b774a0fae13895039a39086 [id=a896a340966c44b89d5b32da4c6b1b68]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 18:57:42.727080 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet 7fdacc021a974bd981dc30e4915378a3: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.730588 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6d41c0dd71408e89231ff56fde6e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.731827 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 730a2aed5a1344b387398114f6105d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.709857 3871 tablet_service.cc:1467] Processing CreateTablet for tablet 523be25072014425b867e97d429334e8 (DEFAULT_TABLE table=default.Table_9c66e99a00ba4953b1faed44821e14ba [id=edb2d64fea5e43a094186bea93ee7ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.734205 3870 tablet_service.cc:1467] Processing CreateTablet for tablet a4e0b84bdb134a459d2d9afd046326c2 (DEFAULT_TABLE table=default.Table_97a83e019035401aa2d468bf20c5ec11 [id=e952001370ce4c0a934cb8d05903b71f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.734386 3872 tablet_service.cc:1467] Processing CreateTablet for tablet 5faabb319fc84f24ab548a707be284f1 (DEFAULT_TABLE table=default.Table_6531b518884b4b9b8ecfdc09c2a0e15f [id=2b3c971b60a24e698fc8941e29298af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:42.745803 4163 tablet_bootstrap.cc:492] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.745987 4163 ts_tablet_manager.cc:1397] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.000s
W20260811 18:57:42.750468 3824 consensus_peers.cc:487] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.753917 3865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332d3d9ddb224a4697809b19b622749d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754148 3871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 523be25072014425b867e97d429334e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754312 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e0b84bdb134a459d2d9afd046326c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.754679 3864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f336fdd55965432f83b2d1c436fdf1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.758476 3872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5faabb319fc84f24ab548a707be284f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:42.768261 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet edd393d9c4d24aab8a170e1329400656 (table default.Table_2ce145eabbad4a958c6799476314823c [id=2699d23349ac49f0b9a0b92dbab4e3ef]). Expected version 2 got 0
I20260811 18:57:42.779369 4169 ts_tablet_manager.cc:1918] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.779520 4169 log.cc:1196] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/8cce97bfdb1d41e685e091c8c5cc6946
I20260811 18:57:42.779861 4169 ts_tablet_manager.cc:1939] T 8cce97bfdb1d41e685e091c8c5cc6946 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.780691 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1166) took 4461 ms. Trace:
I20260811 18:57:42.780788 4169 rpcz_store.cc:276] 0811 18:57:38.319222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.319294 (+ 72us) service_pool.cc:227] Handling call
0811 18:57:42.656572 (+4337278us) ts_tablet_manager.cc:1145] Deleting tablet 8cce97bfdb1d41e685e091c8c5cc6946
0811 18:57:42.656586 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.704236 (+ 47650us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.779254 (+ 75018us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.780666 (+ 1412us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4337010}
I20260811 18:57:42.781215 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 8cce97bfdb1d41e685e091c8c5cc6946 (table default.Table_a108c0f554824b94acc46ae9330d1115 [id=a2253ec340614e15ba71a1999fd13856]) successfully deleted
W20260811 18:57:42.785063 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260811 18:57:42.790506 4169 tablet_replica.cc:330] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.791412 4169 raft_consensus.cc:2229] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.791795 4169 raft_consensus.cc:2258] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.792847 4169 ts_tablet_manager.cc:1905] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:42.801352 3824 consensus_peers.cc:487] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:42.803807 5593 raft_consensus.cc:482] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.803963 5593 raft_consensus.cc:504] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.804564 5593 leader_election.cc:288] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.806273 3824 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.807173 4169 ts_tablet_manager.cc:1918] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.807461 4169 log.cc:1196] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/7057b8a7bae249caaed304b5c66e6871
I20260811 18:57:42.808166 4169 ts_tablet_manager.cc:1939] T 7057b8a7bae249caaed304b5c66e6871 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.809547 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1167) took 4481 ms. Trace:
I20260811 18:57:42.809767 4169 rpcz_store.cc:276] 0811 18:57:38.328355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.334770 (+ 6415us) service_pool.cc:227] Handling call
0811 18:57:42.781082 (+4446312us) ts_tablet_manager.cc:1145] Deleting tablet 7057b8a7bae249caaed304b5c66e6871
0811 18:57:42.790400 (+ 9318us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.799654 (+ 9254us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.807020 (+ 7366us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.809524 (+ 2504us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4446052}
I20260811 18:57:42.812484 4169 tablet_replica.cc:330] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.812805 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 7057b8a7bae249caaed304b5c66e6871 (table default.Table_6317dbacc890428caf5adac65ad7895d [id=05a56fb414f74ec18f9abb327b54e867]) successfully deleted
I20260811 18:57:42.812969 4169 raft_consensus.cc:2229] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:42.813534 4169 raft_consensus.cc:2258] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.814790 4169 ts_tablet_manager.cc:1905] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.818451 4163 raft_consensus.cc:348] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.818869 4163 raft_consensus.cc:374] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.818976 4163 raft_consensus.cc:729] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.819267 4163 consensus_queue.cc:260] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.819928 4163 ts_tablet_manager.cc:1428] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.824244 4169 ts_tablet_manager.cc:1918] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:42.824560 4169 log.cc:1196] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/b4c7f500a5444bae9a015a5e903c0622
I20260811 18:57:42.825245 4169 ts_tablet_manager.cc:1939] T b4c7f500a5444bae9a015a5e903c0622 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.828117 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet e14f8717bcd94d6d88295fd8be130ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet 649cb98f553a418b911e307fdaf49917) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.833169 4163 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.837649 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1169) took 4482 ms. Trace:
I20260811 18:57:42.839075 4169 rpcz_store.cc:276] 0811 18:57:38.355194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.355377 (+ 183us) service_pool.cc:227] Handling call
0811 18:57:42.812402 (+4457025us) ts_tablet_manager.cc:1145] Deleting tablet b4c7f500a5444bae9a015a5e903c0622
0811 18:57:42.812416 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.819622 (+ 7206us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.824081 (+ 4459us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.837404 (+ 13323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4456781}
I20260811 18:57:42.841537 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet b4c7f500a5444bae9a015a5e903c0622 (table default.Table_34d2346219e14bd58aa02228ad109763 [id=7c0d12bf15b945c68177bfb057015870]) successfully deleted
I20260811 18:57:42.842799 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1173) took 4443 ms. Trace:
I20260811 18:57:42.843394 4169 rpcz_store.cc:276] 0811 18:57:38.399428 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.399489 (+ 61us) service_pool.cc:227] Handling call
0811 18:57:42.841969 (+4442480us) ts_tablet_manager.cc:1145] Deleting tablet 76506ad179a9408c9e4cf8e4275ebddd
0811 18:57:42.841983 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.842772 (+ 789us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4424795}
I20260811 18:57:42.844561 4163 tablet_bootstrap.cc:654] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:42.845448 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 76506ad179a9408c9e4cf8e4275ebddd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76506ad179a9408c9e4cf8e4275ebddd already in progress: creating tablet
I20260811 18:57:42.848579 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1172) took 4449 ms. Trace:
I20260811 18:57:42.849215 4169 rpcz_store.cc:276] 0811 18:57:38.399307 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.401725 (+ 2418us) service_pool.cc:227] Handling call
0811 18:57:42.847377 (+4445652us) ts_tablet_manager.cc:1145] Deleting tablet 31bb11617f1548f88f534289706edb06
0811 18:57:42.847559 (+ 182us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.848386 (+ 827us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4422771}
W20260811 18:57:42.850811 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 31bb11617f1548f88f534289706edb06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31bb11617f1548f88f534289706edb06 already in progress: creating tablet
I20260811 18:57:42.851562 4163 tablet_bootstrap.cc:492] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.851715 4163 ts_tablet_manager.cc:1397] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.005s
I20260811 18:57:42.852439 4163 raft_consensus.cc:348] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.852738 4163 raft_consensus.cc:374] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.852809 4163 raft_consensus.cc:729] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.853048 4163 consensus_queue.cc:260] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.853652 4163 ts_tablet_manager.cc:1428] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:42.860051 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1185) took 4193 ms. Trace:
I20260811 18:57:42.860337 4169 rpcz_store.cc:276] 0811 18:57:38.666184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.666250 (+ 66us) service_pool.cc:227] Handling call
0811 18:57:42.855270 (+4189020us) ts_tablet_manager.cc:1145] Deleting tablet ee5b7d1e32514ad48177371582e85241
0811 18:57:42.859870 (+ 4600us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.860012 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4188713}
I20260811 18:57:42.861363 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet b20872d80c9545e5822bfc82d496ce97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.861413 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1186) took 4194 ms. Trace:
I20260811 18:57:42.861634 4169 rpcz_store.cc:276] 0811 18:57:38.666714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.666766 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:42.860840 (+4194074us) ts_tablet_manager.cc:1145] Deleting tablet 4451cc4d7c5940cdb41c021ae486a5a9
0811 18:57:42.861272 (+ 432us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.861386 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4193837}
W20260811 18:57:42.861719 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ee5b7d1e32514ad48177371582e85241 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5b7d1e32514ad48177371582e85241 already in progress: creating tablet
I20260811 18:57:42.862159 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1187) took 4195 ms. Trace:
W20260811 18:57:42.862215 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 4451cc4d7c5940cdb41c021ae486a5a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4451cc4d7c5940cdb41c021ae486a5a9 already in progress: creating tablet
I20260811 18:57:42.862290 4169 rpcz_store.cc:276] 0811 18:57:38.667145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.667194 (+ 49us) service_pool.cc:227] Handling call
0811 18:57:42.862030 (+4194836us) ts_tablet_manager.cc:1145] Deleting tablet c590c98e39f84ea09b26bbe8640ca93d
0811 18:57:42.862041 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.862141 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4194649}
I20260811 18:57:42.862696 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1188) took 4195 ms. Trace:
W20260811 18:57:42.862754 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet c590c98e39f84ea09b26bbe8640ca93d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c590c98e39f84ea09b26bbe8640ca93d already in progress: creating tablet
I20260811 18:57:42.862802 4169 rpcz_store.cc:276] 0811 18:57:38.667507 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.667553 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:42.862590 (+4195037us) ts_tablet_manager.cc:1145] Deleting tablet cb54beb34a654fbdb9c7632901bb0b9e
0811 18:57:42.862600 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.862681 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4194867}
I20260811 18:57:42.863137 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1184) took 4197 ms. Trace:
W20260811 18:57:42.863214 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet cb54beb34a654fbdb9c7632901bb0b9e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb54beb34a654fbdb9c7632901bb0b9e already in progress: creating tablet
I20260811 18:57:42.863219 4169 rpcz_store.cc:276] 0811 18:57:38.666101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.697528 (+ 31427us) service_pool.cc:227] Handling call
0811 18:57:42.863020 (+4165492us) ts_tablet_manager.cc:1145] Deleting tablet eb43127dda814a718b338906fc2a3c47
0811 18:57:42.863033 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863120 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4164924}
I20260811 18:57:42.863545 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1193) took 3934 ms. Trace:
W20260811 18:57:42.863648 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet eb43127dda814a718b338906fc2a3c47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eb43127dda814a718b338906fc2a3c47 already in progress: creating tablet
I20260811 18:57:42.863654 4169 rpcz_store.cc:276] 0811 18:57:38.929261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:38.929319 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.863441 (+3934122us) ts_tablet_manager.cc:1145] Deleting tablet 09366eccf38a4876840cc5ddddbbb103
0811 18:57:42.863452 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863530 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3933941}
I20260811 18:57:42.863960 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1206) took 3643 ms. Trace:
W20260811 18:57:42.864085 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 09366eccf38a4876840cc5ddddbbb103 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09366eccf38a4876840cc5ddddbbb103 already in progress: creating tablet
I20260811 18:57:42.864039 4169 rpcz_store.cc:276] 0811 18:57:39.220336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.220391 (+ 55us) service_pool.cc:227] Handling call
0811 18:57:42.863861 (+3643470us) ts_tablet_manager.cc:1145] Deleting tablet 1ccd56a8b7cc454ea3f2f08b59374521
0811 18:57:42.863871 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.863944 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3643278}
I20260811 18:57:42.864377 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1207) took 3643 ms. Trace:
I20260811 18:57:42.864455 4169 rpcz_store.cc:276] 0811 18:57:39.221024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.221076 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:42.864267 (+3643191us) ts_tablet_manager.cc:1145] Deleting tablet 78868ff1f8024163ae1d782151100cd4
0811 18:57:42.864276 (+ 9us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.864361 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3642789}
W20260811 18:57:42.864512 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 1ccd56a8b7cc454ea3f2f08b59374521 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ccd56a8b7cc454ea3f2f08b59374521 already in progress: creating tablet
I20260811 18:57:42.864707 4169 tablet_replica.cc:330] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.864943 4169 raft_consensus.cc:2229] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260811 18:57:42.864974 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 78868ff1f8024163ae1d782151100cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78868ff1f8024163ae1d782151100cd4 already in progress: creating tablet
I20260811 18:57:42.865180 4169 raft_consensus.cc:2258] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.865797 4169 ts_tablet_manager.cc:1905] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.870751 4163 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.875451 4163 tablet_bootstrap.cc:654] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.875726 4169 ts_tablet_manager.cc:1918] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.884541 4169 log.cc:1196] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cf6c8d542ae2489ba466ffadd518db12
I20260811 18:57:42.884967 4169 ts_tablet_manager.cc:1939] T cf6c8d542ae2489ba466ffadd518db12 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.885789 4163 tablet_bootstrap.cc:492] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.885814 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1208) took 3664 ms. Trace:
I20260811 18:57:42.886402 4163 ts_tablet_manager.cc:1397] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.006s
I20260811 18:57:42.886302 4169 rpcz_store.cc:276] 0811 18:57:39.221671 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.221717 (+ 46us) service_pool.cc:227] Handling call
0811 18:57:42.864660 (+3642943us) ts_tablet_manager.cc:1145] Deleting tablet cf6c8d542ae2489ba466ffadd518db12
0811 18:57:42.864671 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.872550 (+ 7879us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.875625 (+ 3075us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.885789 (+ 10164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3642779}
I20260811 18:57:42.887570 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cf6c8d542ae2489ba466ffadd518db12 (table default.Table_70493900fc9346389c1d4b71c611b962 [id=97d17499bd684f0abf98e0582fa06f5d]) successfully deleted
I20260811 18:57:42.888541 4163 raft_consensus.cc:348] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:42.888778 4163 raft_consensus.cc:374] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.888837 4163 raft_consensus.cc:729] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.889050 4163 consensus_queue.cc:260] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:42.889820 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1209) took 3517 ms. Trace:
I20260811 18:57:42.889905 4169 rpcz_store.cc:276] 0811 18:57:39.372248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.372327 (+ 79us) service_pool.cc:227] Handling call
0811 18:57:42.889676 (+3517349us) ts_tablet_manager.cc:1145] Deleting tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8
0811 18:57:42.889689 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.889801 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3517076}
I20260811 18:57:42.890331 4163 ts_tablet_manager.cc:1428] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
W20260811 18:57:42.890780 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ac5e1db6944fdfa9b2d0e6d0ee02d8 already in progress: creating tablet
I20260811 18:57:42.891009 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1210) took 3400 ms. Trace:
I20260811 18:57:42.891079 4169 rpcz_store.cc:276] 0811 18:57:39.490527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.490594 (+ 67us) service_pool.cc:227] Handling call
0811 18:57:42.890115 (+3399521us) ts_tablet_manager.cc:1145] Deleting tablet fa95131ce50943e3b0c6b5bdd5a30939
0811 18:57:42.890484 (+ 369us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.890992 (+ 508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3397060}
I20260811 18:57:42.894704 5593 raft_consensus.cc:482] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.894858 5593 raft_consensus.cc:504] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.895455 5593 leader_election.cc:288] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.898972 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fa95131ce50943e3b0c6b5bdd5a30939 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa95131ce50943e3b0c6b5bdd5a30939 already in progress: creating tablet
W20260811 18:57:42.899210 3824 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.900020 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a64df7c295c4d9baded1b63a9980581 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:42.902073 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1219) took 3215 ms. Trace:
I20260811 18:57:42.902284 4169 rpcz_store.cc:276] 0811 18:57:39.687067 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687118 (+ 51us) service_pool.cc:227] Handling call
0811 18:57:42.891275 (+3204157us) ts_tablet_manager.cc:1145] Deleting tablet ab03996c5c624e5eb17267b124e27f45
0811 18:57:42.900531 (+ 9256us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.902032 (+ 1501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3197104}
I20260811 18:57:42.903328 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1218) took 3216 ms. Trace:
I20260811 18:57:42.903944 4163 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.903638 4169 rpcz_store.cc:276] 0811 18:57:39.686933 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.686991 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:42.902922 (+3215931us) ts_tablet_manager.cc:1145] Deleting tablet bc38e6760754449fbd6209d765cce70c
0811 18:57:42.902956 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.903310 (+ 354us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3205919}
I20260811 18:57:42.904718 4169 tablet_replica.cc:330] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:42.905208 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet ab03996c5c624e5eb17267b124e27f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab03996c5c624e5eb17267b124e27f45 already in progress: creating tablet
W20260811 18:57:42.905501 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet bc38e6760754449fbd6209d765cce70c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc38e6760754449fbd6209d765cce70c already in progress: creating tablet
I20260811 18:57:42.905821 4169 raft_consensus.cc:2229] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.906488 4169 raft_consensus.cc:2258] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.908474 4169 ts_tablet_manager.cc:1905] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.909473 4163 tablet_bootstrap.cc:654] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:42.923552 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 6f3060822021431584a494e6941be756 on TS 59811bdf82234ef7ac5865dbecfcc388
I20260811 18:57:42.924012 5593 raft_consensus.cc:482] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.924166 5593 raft_consensus.cc:504] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.924728 5593 leader_election.cc:288] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.927804 3824 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.931154 4169 ts_tablet_manager.cc:1918] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:42.931386 4169 log.cc:1196] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/e544b4b1592c4d51a2ed0d6461edc75e
I20260811 18:57:42.932178 4163 tablet_bootstrap.cc:492] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.932394 4163 ts_tablet_manager.cc:1397] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.000s
I20260811 18:57:42.933205 4163 raft_consensus.cc:348] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.933504 4163 raft_consensus.cc:374] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.933624 4163 raft_consensus.cc:729] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.933892 4163 consensus_queue.cc:260] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.934546 4163 ts_tablet_manager.cc:1428] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:42.934862 4163 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.936463 4163 tablet_bootstrap.cc:654] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.937786 4169 ts_tablet_manager.cc:1939] T e544b4b1592c4d51a2ed0d6461edc75e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.943583 5593 raft_consensus.cc:482] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:42.943756 5593 raft_consensus.cc:504] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.944429 5593 leader_election.cc:288] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:42.946077 3824 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:42.946743 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1216) took 3260 ms. Trace:
I20260811 18:57:42.946883 4169 rpcz_store.cc:276] 0811 18:57:39.686714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687161 (+ 447us) service_pool.cc:227] Handling call
0811 18:57:42.904585 (+3217424us) ts_tablet_manager.cc:1145] Deleting tablet e544b4b1592c4d51a2ed0d6461edc75e
0811 18:57:42.904595 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.922960 (+ 18365us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.931029 (+ 8069us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.946718 (+ 15689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3207455}
I20260811 18:57:42.947392 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet e544b4b1592c4d51a2ed0d6461edc75e (table default.Table_2e89aca0564646a5bcc2c6ce3f851910 [id=462a49e1d6d441eb8510ff1def7a54d2]) successfully deleted
I20260811 18:57:42.947619 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1217) took 3260 ms. Trace:
I20260811 18:57:42.947744 4169 rpcz_store.cc:276] 0811 18:57:39.686834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.687305 (+ 471us) service_pool.cc:227] Handling call
0811 18:57:42.947403 (+3260098us) ts_tablet_manager.cc:1145] Deleting tablet f9cef27540ff46aebc9b6f4760ae0db4
0811 18:57:42.947417 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.947599 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3250142}
W20260811 18:57:42.948213 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f9cef27540ff46aebc9b6f4760ae0db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9cef27540ff46aebc9b6f4760ae0db4 already in progress: creating tablet
I20260811 18:57:42.948372 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1226) took 3148 ms. Trace:
I20260811 18:57:42.948493 4169 rpcz_store.cc:276] 0811 18:57:39.799988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:39.802413 (+ 2425us) service_pool.cc:227] Handling call
0811 18:57:42.948192 (+3145779us) ts_tablet_manager.cc:1145] Deleting tablet f389d2b7ff354ac4bab5d68b9ff201c6
0811 18:57:42.948202 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.948290 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3145567}
W20260811 18:57:42.948896 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet f389d2b7ff354ac4bab5d68b9ff201c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f389d2b7ff354ac4bab5d68b9ff201c6 already in progress: creating tablet
I20260811 18:57:42.948949 4169 tablet_replica.cc:330] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:42.949268 4169 raft_consensus.cc:2229] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:42.949478 4169 raft_consensus.cc:2258] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:42.950413 4169 ts_tablet_manager.cc:1905] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:42.967990 4163 tablet_bootstrap.cc:492] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:42.969422 4163 ts_tablet_manager.cc:1397] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.000s
I20260811 18:57:42.970286 4163 raft_consensus.cc:348] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.974908 4163 raft_consensus.cc:374] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:42.975052 4163 raft_consensus.cc:729] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:42.979511 4163 consensus_queue.cc:260] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:42.980389 4163 ts_tablet_manager.cc:1428] T ebf7720e31d84ae2ab592da1e7a3132a P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 18:57:42.981034 4163 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:42.983562 4163 tablet_bootstrap.cc:654] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:42.984753 4169 ts_tablet_manager.cc:1918] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:42.986092 4169 log.cc:1196] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/cd1fc828f73642c88171d6ac827de90e
I20260811 18:57:42.986570 4169 ts_tablet_manager.cc:1939] T cd1fc828f73642c88171d6ac827de90e P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:42.990451 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf7720e31d84ae2ab592da1e7a3132a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:17 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.000068 4163 tablet_bootstrap.cc:492] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.000396 4163 ts_tablet_manager.cc:1397] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.000s
I20260811 18:57:43.001586 4163 raft_consensus.cc:348] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.002015 4163 raft_consensus.cc:374] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.002153 4163 raft_consensus.cc:729] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.002660 4163 consensus_queue.cc:260] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.003930 4163 ts_tablet_manager.cc:1428] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:57:43.005800 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1228) took 2909 ms. Trace:
I20260811 18:57:43.005905 4169 rpcz_store.cc:276] 0811 18:57:40.096155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.096219 (+ 64us) service_pool.cc:227] Handling call
0811 18:57:42.948901 (+2852682us) ts_tablet_manager.cc:1145] Deleting tablet cd1fc828f73642c88171d6ac827de90e
0811 18:57:42.948913 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:42.972182 (+ 23269us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:42.984642 (+ 12460us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.005779 (+ 21137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":384,"tablet-delete.queue_time_us":2852464}
I20260811 18:57:43.006543 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet cd1fc828f73642c88171d6ac827de90e (table default.Table_6533d74b53c54546bdd082863b38257c [id=05015e596cdd481f95a70d3477757a32]) successfully deleted
I20260811 18:57:43.005860 4163 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.007359 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1229) took 2910 ms. Trace:
I20260811 18:57:43.007448 4169 rpcz_store.cc:276] 0811 18:57:40.096647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.096694 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:43.006302 (+2909608us) ts_tablet_manager.cc:1145] Deleting tablet b9fbac57c9c44a028f5b258593aff86d
0811 18:57:43.007209 (+ 907us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.007340 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2909449}
I20260811 18:57:43.007710 4169 tablet_replica.cc:330] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.007893 4169 raft_consensus.cc:2229] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.008033 4169 raft_consensus.cc:2258] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.008636 4169 ts_tablet_manager.cc:1905] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:43.010542 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet b9fbac57c9c44a028f5b258593aff86d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9fbac57c9c44a028f5b258593aff86d already in progress: creating tablet
I20260811 18:57:43.027263 4169 ts_tablet_manager.cc:1918] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.027853 4169 log.cc:1196] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/55f97f37fa7c483cacaeed54f93e7a24
I20260811 18:57:43.028923 4163 tablet_bootstrap.cc:654] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.029531 4169 ts_tablet_manager.cc:1939] T 55f97f37fa7c483cacaeed54f93e7a24 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.031494 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1230) took 2934 ms. Trace:
W20260811 18:57:43.033903 3824 consensus_peers.cc:487] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:43.031594 4169 rpcz_store.cc:276] 0811 18:57:40.097032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097079 (+ 47us) service_pool.cc:227] Handling call
0811 18:57:43.007672 (+2910593us) ts_tablet_manager.cc:1145] Deleting tablet 55f97f37fa7c483cacaeed54f93e7a24
0811 18:57:43.007682 (+ 10us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.016671 (+ 8989us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.026774 (+ 10103us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.031471 (+ 4697us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2910434}
I20260811 18:57:43.034417 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 55f97f37fa7c483cacaeed54f93e7a24 (table default.Table_6fa45c96f1dc479cb4c1cabf87f18f1c [id=7428a6780e8e4435b06867b50c863df7]) successfully deleted
I20260811 18:57:43.034509 4169 tablet_replica.cc:330] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.034823 4169 raft_consensus.cc:2229] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.035024 4169 raft_consensus.cc:2258] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.035640 4169 ts_tablet_manager.cc:1905] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.046277 4163 tablet_bootstrap.cc:492] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.046677 5593 raft_consensus.cc:482] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.046759 4163 ts_tablet_manager.cc:1397] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.005s
I20260811 18:57:43.046856 5593 raft_consensus.cc:504] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.047561 5593 leader_election.cc:288] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:43.049000 3824 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.049409 4163 raft_consensus.cc:348] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.049733 4163 raft_consensus.cc:374] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.049858 4163 raft_consensus.cc:729] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.050163 4163 consensus_queue.cc:260] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
W20260811 18:57:43.062868 3539 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9af9fb059984c9680e78b68120b3726 (table default.Table_8e407f239f804e5cbd2c9ecbc2c170c5 [id=f5e0de91886b4d318b39cb19e45e9c14], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9af9fb059984c9680e78b68120b3726 consensus state has no leader
I20260811 18:57:43.068943 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet a0116997a1c64e658045c319f1c45861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='slave'} at 127.0.0.1:37348
W20260811 18:57:43.070551 3824 consensus_peers.cc:487] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Couldn't send request to peer 59811bdf82234ef7ac5865dbecfcc388. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:43.072276 4163 ts_tablet_manager.cc:1428] T a0116997a1c64e658045c319f1c45861 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260811 18:57:43.072638 4163 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
W20260811 18:57:43.084622 3361 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3659: Timed out: CheckLeaks RPC to 127.3.72.65:32817 timed out after 1.000s (SENT)
I20260811 18:57:43.084980 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3659
I20260811 18:57:43.091089 4163 tablet_bootstrap.cc:654] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.091555 4169 ts_tablet_manager.cc:1918] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.091818 4169 log.cc:1196] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/a847581412494e34b2c76e605ff11772
I20260811 18:57:43.092288 4169 ts_tablet_manager.cc:1939] T a847581412494e34b2c76e605ff11772 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.093263 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1231) took 2995 ms. Trace:
I20260811 18:57:43.093441 4169 rpcz_store.cc:276] 0811 18:57:40.097414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097455 (+ 41us) service_pool.cc:227] Handling call
0811 18:57:43.034453 (+2936998us) ts_tablet_manager.cc:1145] Deleting tablet a847581412494e34b2c76e605ff11772
0811 18:57:43.034467 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.038954 (+ 4487us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.091423 (+ 52469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.093239 (+ 1816us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2936851}
I20260811 18:57:43.094101 4169 tablet_replica.cc:330] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.094434 4169 raft_consensus.cc:2229] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.094470 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet a847581412494e34b2c76e605ff11772 (table default.Table_a303b89a5ef54a7ea92f2bef6298d619 [id=02e48593d37e4fb8b8411f134fdf8839]) successfully deleted
I20260811 18:57:43.094694 4169 raft_consensus.cc:2258] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.095714 4169 ts_tablet_manager.cc:1905] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.105684 4169 ts_tablet_manager.cc:1918] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.105989 4169 log.cc:1196] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/0c6e0d1eeb63493bbea4cd78dd085a3c
I20260811 18:57:43.106462 4169 ts_tablet_manager.cc:1939] T 0c6e0d1eeb63493bbea4cd78dd085a3c P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.107517 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1232) took 3009 ms. Trace:
I20260811 18:57:43.107720 4169 rpcz_store.cc:276] 0811 18:57:40.097800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.097853 (+ 53us) service_pool.cc:227] Handling call
0811 18:57:43.094039 (+2996186us) ts_tablet_manager.cc:1145] Deleting tablet 0c6e0d1eeb63493bbea4cd78dd085a3c
0811 18:57:43.094054 (+ 15us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.101056 (+ 7002us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.105221 (+ 4165us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.107489 (+ 2268us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2996020}
I20260811 18:57:43.108196 5593 raft_consensus.cc:482] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.108498 4169 tablet_replica.cc:330] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.108361 5593 raft_consensus.cc:504] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.108722 4169 raft_consensus.cc:2229] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.108912 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 0c6e0d1eeb63493bbea4cd78dd085a3c (table default.Table_495c9c518a6d4462b9983c0f74a02098 [id=f7fa32f5e00247d789399dcebea75999]) successfully deleted
I20260811 18:57:43.109133 5593 leader_election.cc:288] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:43.109623 4163 tablet_bootstrap.cc:492] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.109776 4163 ts_tablet_manager.cc:1397] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.007s
I20260811 18:57:43.109889 4169 raft_consensus.cc:2258] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260811 18:57:43.110769 3824 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.110720 4163 raft_consensus.cc:348] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.110975 4163 raft_consensus.cc:374] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.111043 4163 raft_consensus.cc:729] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.111279 4163 consensus_queue.cc:260] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.111938 4163 ts_tablet_manager.cc:1428] T 330801031cc74eb2a5bc2caf53a18ead P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:43.112012 4169 ts_tablet_manager.cc:1905] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.112263 4163 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.114208 4163 tablet_bootstrap.cc:654] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.120330 3547 catalog_manager.cc:5587] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837) does not have the latest schema for tablet 330801031cc74eb2a5bc2caf53a18ead (table default.Table_04ac8209b46549e3bb3f12971ab01b73 [id=54b88a6b23fe45e8b0f235baf8f7234f]). Expected version 1 got 0
I20260811 18:57:43.126463 4169 ts_tablet_manager.cc:1918] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.127363 4169 log.cc:1196] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/6b4de110bd3348f0b0dae21c9f2b6a19
I20260811 18:57:43.128176 4169 ts_tablet_manager.cc:1939] T 6b4de110bd3348f0b0dae21c9f2b6a19 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.129506 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1233) took 3031 ms. Trace:
I20260811 18:57:43.129696 4169 rpcz_store.cc:276] 0811 18:57:40.098470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.098514 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:43.108439 (+3009925us) ts_tablet_manager.cc:1145] Deleting tablet 6b4de110bd3348f0b0dae21c9f2b6a19
0811 18:57:43.108453 (+ 14us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.115483 (+ 7030us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.126136 (+ 10653us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.129482 (+ 3346us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3009773}
I20260811 18:57:43.130249 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 6b4de110bd3348f0b0dae21c9f2b6a19 (table default.Table_08e3d1b9d0af4aeea3a9b46ad0b7fb7f [id=5ce66576efa14a358fdea22ba2f781b3]) successfully deleted
I20260811 18:57:43.130463 4169 tablet_replica.cc:330] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.130833 4169 raft_consensus.cc:2229] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.131129 4169 raft_consensus.cc:2258] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.132099 4169 ts_tablet_manager.cc:1905] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.132956 5593 raft_consensus.cc:482] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.133085 5593 raft_consensus.cc:504] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.133713 5593 leader_election.cc:288] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
W20260811 18:57:43.135356 3824 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.141155 4163 tablet_bootstrap.cc:492] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.141400 4163 ts_tablet_manager.cc:1397] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.008s
I20260811 18:57:43.142392 4163 raft_consensus.cc:348] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.142674 4163 raft_consensus.cc:374] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.142781 4163 raft_consensus.cc:729] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.143028 4163 consensus_queue.cc:260] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.146592 4163 ts_tablet_manager.cc:1428] T bfea0cd619024204b9bff1e2dd62b7b0 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:43.147070 4163 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.159472 4163 tablet_bootstrap.cc:654] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.162932 4169 ts_tablet_manager.cc:1918] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.163528 4169 log.cc:1196] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/62c365f79d4a4201ac323f16bddf3e62
I20260811 18:57:43.163542 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet bfea0cd619024204b9bff1e2dd62b7b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:38 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.164104 4169 ts_tablet_manager.cc:1939] T 62c365f79d4a4201ac323f16bddf3e62 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.165089 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1234) took 3066 ms. Trace:
I20260811 18:57:43.165422 4169 rpcz_store.cc:276] 0811 18:57:40.099007 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.099051 (+ 44us) service_pool.cc:227] Handling call
0811 18:57:43.130390 (+3031339us) ts_tablet_manager.cc:1145] Deleting tablet 62c365f79d4a4201ac323f16bddf3e62
0811 18:57:43.130402 (+ 12us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.139744 (+ 9342us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.162573 (+ 22829us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.165064 (+ 2491us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3031076}
I20260811 18:57:43.165982 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 62c365f79d4a4201ac323f16bddf3e62 (table default.Table_d1d051735e24450cb04f8f19ea9ec4d8 [id=c1f04f46c67e40468d7aaaebc8e6835e]) successfully deleted
I20260811 18:57:43.166253 4169 tablet_replica.cc:330] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.166576 4169 raft_consensus.cc:2229] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.167277 4169 raft_consensus.cc:2258] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.171756 4169 ts_tablet_manager.cc:1905] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.179728 4163 tablet_bootstrap.cc:492] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.180006 4163 ts_tablet_manager.cc:1397] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.000s
I20260811 18:57:43.181025 4163 raft_consensus.cc:348] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.181455 4163 raft_consensus.cc:374] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.181586 4163 raft_consensus.cc:729] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.181903 4163 consensus_queue.cc:260] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.182639 4163 ts_tablet_manager.cc:1428] T 9ec4bba841dd446aa3db751358ce7f32 P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:43.187690 4163 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.190026 4163 tablet_bootstrap.cc:654] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.191359 4169 ts_tablet_manager.cc:1918] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.191579 4169 log.cc:1196] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/27cda53b7ca54346a7a22bf26f5db990
I20260811 18:57:43.192031 4169 ts_tablet_manager.cc:1939] T 27cda53b7ca54346a7a22bf26f5db990 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
I20260811 18:57:43.198904 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1235) took 3092 ms. Trace:
I20260811 18:57:43.199122 4169 rpcz_store.cc:276] 0811 18:57:40.106724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.106781 (+ 57us) service_pool.cc:227] Handling call
0811 18:57:43.166173 (+3059392us) ts_tablet_manager.cc:1145] Deleting tablet 27cda53b7ca54346a7a22bf26f5db990
0811 18:57:43.166190 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.175659 (+ 9469us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.191238 (+ 15579us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.198874 (+ 7636us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3036181}
I20260811 18:57:43.199688 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 27cda53b7ca54346a7a22bf26f5db990 (table default.Table_4c14312f53aa4eb892770f9f6ef4bdb9 [id=6d03fdd79eec47df96db0a2fa206ab1b]) successfully deleted
I20260811 18:57:43.199995 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1236) took 3092 ms. Trace:
I20260811 18:57:43.200137 4169 rpcz_store.cc:276] 0811 18:57:40.107049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.107101 (+ 52us) service_pool.cc:227] Handling call
0811 18:57:43.199770 (+3092669us) ts_tablet_manager.cc:1145] Deleting tablet d65cb8d901c04d8ba2b7a40efc7fa3af
0811 18:57:43.199787 (+ 17us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.199976 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3069819}
I20260811 18:57:43.200645 4169 tablet_replica.cc:330] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
W20260811 18:57:43.200767 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet d65cb8d901c04d8ba2b7a40efc7fa3af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65cb8d901c04d8ba2b7a40efc7fa3af already in progress: creating tablet
I20260811 18:57:43.201010 4169 raft_consensus.cc:2229] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.201275 4169 raft_consensus.cc:2258] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.201946 4169 ts_tablet_manager.cc:1905] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.207346 3859 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ec4bba841dd446aa3db751358ce7f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:22 UTC) from {username='slave'} at 127.0.0.1:37348
I20260811 18:57:43.211628 4163 tablet_bootstrap.cc:492] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.212024 4163 ts_tablet_manager.cc:1397] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.006s
I20260811 18:57:43.212898 4163 raft_consensus.cc:348] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.213218 4163 raft_consensus.cc:374] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.213335 4163 raft_consensus.cc:729] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.213631 4163 consensus_queue.cc:260] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.214422 4163 ts_tablet_manager.cc:1428] T 0540e4e43ea34fbfbc66b6035c60684d P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:43.210983 4169 ts_tablet_manager.cc:1918] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.214744 4169 log.cc:1196] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting WAL directory at /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1786474617098566-3361-0/minicluster-data/ts-1/wal/wals/452ee450778e4d33adfa7e2745b5d970
I20260811 18:57:43.215229 4169 ts_tablet_manager.cc:1939] T 452ee450778e4d33adfa7e2745b5d970 P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting consensus metadata
W20260811 18:57:43.221938 3824 connection.cc:535] client connection to 127.3.72.65:32817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:43.222505 3542 connection.cc:535] server connection from 127.3.72.65:60889 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:43.222934 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3804
I20260811 18:57:43.223541 4163 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.224720 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1237) took 3099 ms. Trace:
I20260811 18:57:43.224905 4169 rpcz_store.cc:276] 0811 18:57:40.125332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.125391 (+ 59us) service_pool.cc:227] Handling call
0811 18:57:43.200598 (+3075207us) ts_tablet_manager.cc:1145] Deleting tablet 452ee450778e4d33adfa7e2745b5d970
0811 18:57:43.200609 (+ 11us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.205825 (+ 5216us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.210870 (+ 5045us) tablet_metadata.cc:641] Metadata flushed
0811 18:57:43.224680 (+ 13810us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3070407}
I20260811 18:57:43.225520 4163 tablet_bootstrap.cc:654] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.225628 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1239) took 2949 ms. Trace:
I20260811 18:57:43.225713 4169 rpcz_store.cc:276] 0811 18:57:40.275867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.278408 (+ 2541us) service_pool.cc:227] Handling call
0811 18:57:43.225491 (+2947083us) ts_tablet_manager.cc:1145] Deleting tablet fdab1c3c431c4dceb08b00eabcfa1ceb
0811 18:57:43.225504 (+ 13us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.225611 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2946801}
I20260811 18:57:43.228143 3933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:43.228480 3539 catalog_manager.cc:4841] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): tablet 452ee450778e4d33adfa7e2745b5d970 (table default.Table_4d6c4d770dc949e78c1707ddbab71740 [id=48f45fcc06e0418185841260a06cb98c]) successfully deleted
W20260811 18:57:43.228825 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet fdab1c3c431c4dceb08b00eabcfa1ceb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdab1c3c431c4dceb08b00eabcfa1ceb already in progress: creating tablet
W20260811 18:57:43.235298 3541 connection.cc:535] client connection to 127.3.72.65:32817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:43.724730 5593 raft_consensus.cc:482] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.724920 5593 raft_consensus.cc:504] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } } peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } }
I20260811 18:57:43.725584 5593 leader_election.cc:288] T 4a64df7c295c4d9baded1b63a9980581 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817), 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027)
I20260811 18:57:43.726123 4169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37348 (request call id 1244) took 3288 ms. Trace:
I20260811 18:57:43.726257 4169 rpcz_store.cc:276] 0811 18:57:40.437347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0811 18:57:40.437405 (+ 58us) service_pool.cc:227] Handling call
0811 18:57:43.226063 (+2788658us) ts_tablet_manager.cc:1145] Deleting tablet 0e556811ad1643f1baa3bbf4edebd157
0811 18:57:43.725933 (+499870us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0811 18:57:43.726097 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2788425}
I20260811 18:57:43.726758 4169 tablet_replica.cc:330] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: stopping tablet replica
I20260811 18:57:43.727061 4169 raft_consensus.cc:2229] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.727263 4169 raft_consensus.cc:2258] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.728065 4169 ts_tablet_manager.cc:1905] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:43.730141 3824 thread.cc:636] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.001s sys 0.009s
W20260811 18:57:43.730281 3824 thread.cc:602] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.503s user 0.001s sys 0.009s
W20260811 18:57:43.730567 3824 consensus_peers.cc:487] T 61cea7b0b85448f5854e8fd99ea5e792 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.730692 3824 leader_election.cc:334] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.730793 3824 leader_election.cc:302] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.731070 5593 raft_consensus.cc:2726] T fb3a9fdc3d594c1a84a5d8efa8490dfa P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.731115 3824 consensus_peers.cc:487] T 2c29af3615e74722a809d3e5ede9b9a1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731206 3824 consensus_peers.cc:487] T 3a9efd2f19d74a68a8536fd057377738 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731276 3824 consensus_peers.cc:487] T 23f41a2638e74ba19c1c527b9fc796df P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731343 3824 leader_election.cc:334] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.731413 3824 leader_election.cc:302] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.731595 5593 raft_consensus.cc:2726] T eb8f5b87062349c4b19ec99fa0175f24 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.731643 3824 consensus_peers.cc:487] T faf0594d55844ae49c99247ecd8b810f P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731721 3824 consensus_peers.cc:487] T 9ebe3fed5aaf45918dbadfde06b2c24e P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731787 3824 consensus_peers.cc:487] T e902efbb774244d18e3ba165bfb7c695 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731886 3824 consensus_peers.cc:487] T 71df3738a5cf4b12820dda131ced62e1 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.731966 3824 consensus_peers.cc:487] T c0b355a0d3984012be4e9406998fd4a0 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.732036 3824 consensus_peers.cc:487] T 0bd39dcc0bed4623a06f00a56b740dba P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.732106 3824 leader_election.cc:334] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.732183 3824 leader_election.cc:302] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.732366 5593 raft_consensus.cc:2726] T 8304ac7709ca4392baec4cfa78ef5b41 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.732414 3824 leader_election.cc:334] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.732501 3824 leader_election.cc:302] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.732636 3824 raft_consensus.cc:2670] T c99ceca57cb245f4a3a6cb4b36f1306d P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:43.733222 3824 consensus_peers.cc:487] T 3c96a264e017471ea114ca0c6a9500ef P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.733237 4163 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.020s sys 0.007s
W20260811 18:57:43.733376 4163 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.507s user 0.020s sys 0.007s
I20260811 18:57:43.734635 4163 tablet_bootstrap.cc:492] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.734838 4163 ts_tablet_manager.cc:1397] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.511s user 0.023s sys 0.007s
I20260811 18:57:43.735942 4163 raft_consensus.cc:348] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.736275 4163 raft_consensus.cc:374] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.736338 4163 raft_consensus.cc:729] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.736579 4163 consensus_queue.cc:260] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.737351 4163 ts_tablet_manager.cc:1428] T 3a994b3f4d7d4c92946ad8d595e6ad2b P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260811 18:57:43.733379 3824 consensus_peers.cc:487] T 40311cd1f327461e94eb443d83a16480 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.742627 3824 leader_election.cc:334] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.742766 3824 leader_election.cc:302] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.742944 3824 raft_consensus.cc:2670] T 5d495354c0994fb4a74c376078e41a26 P 57e8f62fa63a4fee97f268f5b7d3ae24: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260811 18:57:43.743705 3824 consensus_peers.cc:487] T eb82243777a9494b96959ced0c24de5a P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.743848 3824 consensus_peers.cc:487] T d73a43558dbb48889ab205108f4ce6be P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744099 3824 consensus_peers.cc:487] T c6852aaef1d5486b87e5462eae532719 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744216 3824 consensus_peers.cc:487] T e1af33fd98c6486f975368ec90493db7 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744303 3824 consensus_peers.cc:487] T 8dc07c28ccfd48ee9c33e49442c6047b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744388 3824 consensus_peers.cc:487] T 4814a3562ef14fb0bec79be8c3cff4d2 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744448 3824 consensus_peers.cc:487] T 73f05e1d5cb947fe9dbcc2e876a23788 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.744510 3824 leader_election.cc:334] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.744587 3824 leader_election.cc:302] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.744923 3824 consensus_peers.cc:487] T 283c3fe4f41f4185ab4488d774e46299 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.745044 3824 consensus_peers.cc:487] T bb56d23ccabc4b02961249312257d9c4 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.745137 3824 leader_election.cc:334] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.745231 3824 leader_election.cc:302] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
W20260811 18:57:43.744357 3541 catalog_manager.cc:4584] TS 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027): CreateTablet RPC failed for tablet c575ba3060a140759484bea1462ffd73: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111)
I20260811 18:57:43.745469 5593 raft_consensus.cc:2726] T edd393d9c4d24aab8a170e1329400656 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:43.745643 5593 raft_consensus.cc:2726] T b20872d80c9545e5822bfc82d496ce97 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.746042 3824 consensus_peers.cc:487] T 38ed9c1542c24b2ea3c2de8ec0f589d6 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.746193 3824 leader_election.cc:334] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.746328 3824 leader_election.cc:302] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.746621 5593 raft_consensus.cc:2726] T d37e81d376e14907b13746cca831a96b P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.746941 3824 leader_election.cc:334] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.747088 3824 leader_election.cc:302] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.747385 5593 raft_consensus.cc:2726] T e14f8717bcd94d6d88295fd8be130ca5 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.747711 3824 consensus_peers.cc:487] T a811d4488d0e4541808a5e1aa44d663b P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.747870 3824 consensus_peers.cc:487] T 4f3c4536fc7440db83851fc8d7b3de10 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.748006 3824 leader_election.cc:334] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.748140 3824 leader_election.cc:302] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.748417 5593 raft_consensus.cc:2726] T 38e24f8d4522466d81605ea37fa7064a P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:57:43.748852 3824 consensus_peers.cc:487] T 04eb81bf6f454fbfadd27393af15ab17 P 57e8f62fa63a4fee97f268f5b7d3ae24 -> Peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Couldn't send request to peer bd8c4bdbd811417ab011a7131f17d283. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:43.749027 3824 leader_election.cc:334] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817): Network error: Client connection negotiation failed: client connection to 127.3.72.65:32817: connect: Connection refused (error 111)
I20260811 18:57:43.749162 3824 leader_election.cc:302] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e8f62fa63a4fee97f268f5b7d3ae24; no voters: 59811bdf82234ef7ac5865dbecfcc388, bd8c4bdbd811417ab011a7131f17d283
I20260811 18:57:43.749434 5593 raft_consensus.cc:2726] T b4dc2df4c6464d71bb13fe25e0046758 P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:57:43.750715 4163 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.752833 4163 tablet_bootstrap.cc:654] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Neither blocks nor log segments found. Creating new log.
W20260811 18:57:43.759809 3539 catalog_manager.cc:4821] TS 57e8f62fa63a4fee97f268f5b7d3ae24 (127.3.72.66:40837): delete failed for tablet 0e556811ad1643f1baa3bbf4edebd157 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e556811ad1643f1baa3bbf4edebd157 already in progress: creating tablet
I20260811 18:57:43.760123 4163 tablet_bootstrap.cc:492] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: No bootstrap required, opened a new log
I20260811 18:57:43.760391 4163 ts_tablet_manager.cc:1397] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260811 18:57:43.760501 5593 raft_consensus.cc:482] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.760635 5593 raft_consensus.cc:504] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.761289 5593 leader_election.cc:288] T 8fe24bb6edb944008ff9cc73faa7374c P 57e8f62fa63a4fee97f268f5b7d3ae24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59811bdf82234ef7ac5865dbecfcc388 (127.3.72.67:36027), bd8c4bdbd811417ab011a7131f17d283 (127.3.72.65:32817)
I20260811 18:57:43.761168 4163 raft_consensus.cc:348] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.761375 4163 raft_consensus.cc:374] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.761533 4163 raft_consensus.cc:729] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e8f62fa63a4fee97f268f5b7d3ae24, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.761756 4163 consensus_queue.cc:260] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59811bdf82234ef7ac5865dbecfcc388" member_type: VOTER last_known_addr { host: "127.3.72.67" port: 36027 } } peers { permanent_uuid: "bd8c4bdbd811417ab011a7131f17d283" member_type: VOTER last_known_addr { host: "127.3.72.65" port: 32817 } } peers { permanent_uuid: "57e8f62fa63a4fee97f268f5b7d3ae24" member_type: VOTER last_known_addr { host: "127.3.72.66" port: 40837 } }
I20260811 18:57:43.762480 4163 ts_tablet_manager.cc:1428] T 4059e5907b1e488ab9e1d04e563f96cd P 57e8f62fa63a4fee97f268f5b7d3ae24: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:43.762794 4163 tablet_bootstrap.cc:492] T f287040b879642448367abe2b1c73cd9 P 57e8f62fa63a4fee97f268f5b7d3ae24: Bootstrap starting.
I20260811 18:57:43.763890 4169 ts_tablet_manager.cc:1918] T 19243ab3d6744aaa87a133f1462cd2fa P 57e8f62fa63a4fee97f268f5b7d3ae24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260811 18:57:43.765357 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3804
I20260811 18:57:43.846863 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 4863
I20260811 18:57:43.861989 5321 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:43.867790 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet b3f0cdcee681460084662e01152ad9dd on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:43.869256 3541 catalog_manager.cc:4616] Request timed out: CreateTablet RPC for tablet 9ebe3fed5aaf45918dbadfde06b2c24e on TS 59811bdf82234ef7ac5865dbecfcc388
W20260811 18:57:44.073136 3542 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 838a9f3025a1466296844c218cedf4fa (table default.Table_9049c4cdc3654b26a7b3d0cad713ec01 [id=0052bf55538f46c5b17831eaa1cfad91], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 838a9f3025a1466296844c218cedf4fa consensus state has no leader
I20260811 18:57:44.082556 5852 raft_consensus.cc:482] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a82a7352eb644b4a6baf09d8084b652)
I20260811 18:57:44.082778 5852 raft_consensus.cc:504] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb005c681cf42d4ac278ec73a28e4a9" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 34821 } } peers { permanent_uuid: "d85eb97df7824bc9bad9dca349b0c180" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 38889 } } peers { permanent_uuid: "0a82a7352eb644b4a6baf09d8084b652" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 44143 } }
I20260811 18:57:44.083618 5852 leader_election.cc:288] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821), 0a82a7352eb644b4a6baf09d8084b652 (127.3.72.124:44143)
I20260811 18:57:44.086061 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d85eb97df7824bc9bad9dca349b0c180" candidate_term: 2 candidate_status { last_received { term: 1 index: 1658 } } ignore_live_leader: false dest_uuid: "0a82a7352eb644b4a6baf09d8084b652" is_pre_election: true
I20260811 18:57:44.092118 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 4863
W20260811 18:57:44.131071 5853 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:44.084008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.126:34821 (local address 127.0.0.1:36432)
0811 18:57:44.085122 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.085143 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:44.085301 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:44.091709 (+ 6408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:44.091717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:44.091762 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:44.092137 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:44.092155 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.130757 (+ 38602us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":305,"threads_started":1}
I20260811 18:57:44.131776 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 4230
W20260811 18:57:44.132460 4381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:57:44.139266 4381 leader_election.cc:334] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111)
I20260811 18:57:44.139472 4381 leader_election.cc:302] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d85eb97df7824bc9bad9dca349b0c180; no voters: 0a82a7352eb644b4a6baf09d8084b652, 7cb005c681cf42d4ac278ec73a28e4a9
I20260811 18:57:44.139828 5852 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d85eb97df7824bc9bad9dca349b0c180 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:57:44.141705 4466 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:57:44.168576 3540 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a82a7352eb644b4a6baf09d8084b652 -> Peer 7cb005c681cf42d4ac278ec73a28e4a9 (127.3.72.126:34821): Couldn't send request to peer 7cb005c681cf42d4ac278ec73a28e4a9. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.126:34821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260811 18:57:44.358069 3541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.67:36027: connect: Connection refused (error 111) [suppressed 3458 similar messages]
I20260811 18:57:44.361200 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 4230
I20260811 18:57:44.403297 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 3522
I20260811 18:57:44.404374 3588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:44.694159 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 3522
2026-08-11T18:57:44Z chronyd exiting
I20260811 18:57:44.754619 3361 master-stress-test.cc:586] Tables created: 739
I20260811 18:57:44.754757 3361 master-stress-test.cc:587] Tables altered: 374
I20260811 18:57:44.754853 3361 master-stress-test.cc:588] Tables deleted: 314
I20260811 18:57:44.754935 3361 master-stress-test.cc:589] Tablets replaced: 93
I20260811 18:57:44.754994 3361 master-stress-test.cc:590] Masters restarted: 2
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (47645 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260811 18:57:44.910611 3361 test_util.cc:268] Using random seed: 198008552
2026-08-11T18:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:44Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
2026-08-11 18:57:52: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task2wDjrE/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260811 18:58:01.568192 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--webserver_interface=127.3.72.126
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:01.938983 6106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:01.939337 6106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:01.939471 6106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:01.947355 6106 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:01.947487 6106 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:01.947562 6106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:01.947654 6106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:01.947737 6106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:01.947861 6106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:01.957542 6106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:01.959156 6106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:01.961040 6106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:01.974146 6112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.974092 6113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.974187 6115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:01.974994 6106 server_base.cc:1034] running on GCE node
I20260811 18:58:01.975903 6106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:01.977639 6106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:01.978889 6106 hybrid_clock.cc:648] HybridClock initialized: now 1786474681978822 us; error 40 us; skew 500 ppm
I20260811 18:58:01.979322 6106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:01.982731 6106 webserver.cc:460] Webserver started at http://127.3.72.126:35183/ using document root <none> and password file <none>
I20260811 18:58:01.983623 6106 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:01.983834 6106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:01.984153 6106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:01.987525 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/instance:
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:01.988421 6106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal/instance:
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:01.995203 6106 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 18:58:01.999697 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:02.001502 6106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:58:02.001768 6106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:02.002023 6106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:02.031759 6106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:02.032811 6106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:02.033208 6106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:02.055088 6106 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:37881
I20260811 18:58:02.055253 6175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:37881 every 8 connection(s)
I20260811 18:58:02.056700 6106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:02.060891 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:02.065835 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6106
I20260811 18:58:02.066092 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal/instance
I20260811 18:58:02.067395 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:02.068097 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.090065 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.091616 6124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:02.093585 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.143042 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 1
W20260811 18:58:02.145465 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.213821 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 2
W20260811 18:58:02.216202 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.307535 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 3
W20260811 18:58:02.309980 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:02.443028 6179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:02.443362 6179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:02.443445 6179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:02.451092 6179 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:02.451210 6179 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:02.451259 6179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:02.451314 6179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:02.451354 6179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:02.451423 6179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:02.453330 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 4
W20260811 18:58:02.455569 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.125:34867: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:02.461638 6179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:02.463327 6179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:02.465353 6179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:02.478598 6185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.478693 6189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.478745 6187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:02.479851 6179 server_base.cc:1034] running on GCE node
I20260811 18:58:02.480773 6179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:02.482501 6179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:02.483752 6179 hybrid_clock.cc:648] HybridClock initialized: now 1786474682483659 us; error 65 us; skew 500 ppm
I20260811 18:58:02.484192 6179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:02.486918 6179 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:02.487686 6179 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:02.487848 6179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:02.488181 6179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:02.491107 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/instance:
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.491935 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal/instance:
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.498582 6179 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:58:02.502723 6195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:02.504954 6179 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:02.505235 6179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:02.505491 6179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:02.538228 6179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:02.539300 6179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:02.539727 6179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:02.562268 6179 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:02.562305 6247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:02.564153 6179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:02.568539 6248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:02.568562 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6179
I20260811 18:58:02.568749 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal/instance
I20260811 18:58:02.570181 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=0
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:02.578858 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.599882 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:02.612174 6248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.613720 6199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:02.615769 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.665220 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 1
W20260811 18:58:02.667695 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.735991 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 2
W20260811 18:58:02.738379 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.754913 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } attempt: 5
I20260811 18:58:02.762907 6176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
W20260811 18:58:02.765645 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.816972 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 1
W20260811 18:58:02.819237 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.829668 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 3
W20260811 18:58:02.832217 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.887493 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 2
W20260811 18:58:02.890017 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:02.958689 6250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:02.959009 6250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:02.959162 6250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:02.967096 6250 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:02.967227 6250 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:02.967272 6250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:02.967430 6250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:02.967509 6250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:02.967624 6250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:02.975507 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 4
W20260811 18:58:02.977712 6248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.978235 6250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:02.979725 6250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:02.981736 6250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:02.995368 6257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.995453 6258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:02.995489 6260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:02.996253 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 3
I20260811 18:58:02.996848 6250 server_base.cc:1034] running on GCE node
I20260811 18:58:02.997927 6250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260811 18:58:02.998405 6176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:02.999922 6250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:03.001207 6250 hybrid_clock.cc:648] HybridClock initialized: now 1786474683001124 us; error 59 us; skew 500 ppm
I20260811 18:58:03.001639 6250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:03.004542 6250 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:03.005622 6250 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:03.005828 6250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:03.006227 6250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:03.009902 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/instance:
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.011132 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal/instance:
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.017896 6250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:58:03.022256 6266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:03.024258 6250 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:03.024523 6250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:03.024765 6250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:03.041419 6250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:03.042455 6250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:03.042788 6250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:03.064268 6250 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:03.064324 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:03.065874 6250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:03.069981 6320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:03.074209 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6250
I20260811 18:58:03.074453 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal/instance
I20260811 18:58:03.077189 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.094305 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.103158 6320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:58:03.115254 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:03.118644 6320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.119705 6320 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:03.122584 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: No bootstrap required, opened a new log
I20260811 18:58:03.130726 6320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.131301 6320 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.131434 6320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:03.132228 6320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.133389 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.133791 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.134450 6320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.155464 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.155821 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.148320 6250 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: OK, instance_id { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" instance_seqno: 1786474682048794 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:58:03.161396 6250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:03.167558 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.167714 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.175688 6176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 4
W20260811 18:58:03.179178 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.179359 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.190821 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.190971 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.191766 6176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap starting.
I20260811 18:58:03.197028 6176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.198691 6176 log.cc:824] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Log is configured to *not* fsync() on all Append() calls
W20260811 18:58:03.202418 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.202561 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.212575 6176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: No bootstrap required, opened a new log
W20260811 18:58:03.214017 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.214164 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.224887 6176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.225368 6176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.225517 6176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Initialized, Role: FOLLOWER
W20260811 18:58:03.225803 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.225946 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.226498 6176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.230253 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.230830 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.231765 6176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.239887 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.240063 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.252086 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.252314 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.264050 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.264216 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.275744 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.275913 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.286407 6248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } attempt: 5
W20260811 18:58:03.287503 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.287664 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.295308 6272 tablet.cc:2369] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:58:03.298394 6127 tablet.cc:2369] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:03.276685 6106 master_runner.cc:186] Error getting master registration for 127.3.72.125:34867: OK, instance_id { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" instance_seqno: 1786474682555705 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:58:03.302487 6106 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:03.306535 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.306689 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.308250 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
W20260811 18:58:03.308290 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.308550 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.312841 6248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:03.314385 6248 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:03.317879 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: No bootstrap required, opened a new log
W20260811 18:58:03.320434 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.320561 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.318280 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.324573 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.329460 6248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.329944 6248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:03.330077 6248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:03.330844 6248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:03.332108 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.332254 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.336143 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.336292 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.337997 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.338666 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.340508 6248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:58:03.343784 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.343955 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.346262 6365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:03.346625 6365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.348320 6365 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:03.348632 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.348757 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.349501 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:03.350178 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 0.
I20260811 18:58:03.350666 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:03.350960 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.350994 6198 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
I20260811 18:58:03.351621 6365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:03.351780 6365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:03.351879 6365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.352609 6326 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:03.353727 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:03.353991 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 0.
I20260811 18:58:03.354127 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:03.354620 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:03.355113 6326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:03.355266 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:03.355356 6326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.355209 6365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:03.356580 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:58:03.356611 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:03.356850 6149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:03.357548 6222 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 1: Already voted for candidate 6bb6ba710ee24c348d653d524bb73f3d in this term.
I20260811 18:58:03.358727 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:03.359990 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.360244 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 1.
I20260811 18:58:03.360545 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d in current term 1: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:03.361023 6198 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
W20260811 18:58:03.356719 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.364490 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:58:03.364470 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
W20260811 18:58:03.364629 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.364755 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d in current term 1: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:03.365535 6365 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 1 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:03.365514 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:03.365899 6149 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 1: Already voted for candidate 6bb6ba710ee24c348d653d524bb73f3d in this term.
I20260811 18:58:03.366180 6365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:03.366554 6365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:03.366891 6326 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:03.367218 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:03.367569 6365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:03.367758 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:03.368147 6326 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260811 18:58:03.372969 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.373145 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.374848 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.375176 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
W20260811 18:58:03.376204 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.376346 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.381788 6374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 18:58:03.384543 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.384688 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.388159 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.388345 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.396361 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.396529 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.399775 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.399932 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.400394 6374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 18:58:03.408108 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.408280 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.411337 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.411481 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.419849 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.420013 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.423084 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.423236 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.426453 6149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:03.426468 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:03.427423 6365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:03.427778 6370 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:58:03.433526 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.433693 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.461417 6379 mvcc.cc:205] Tried to move back new op lower bound from 7317400303302197248 to 7317400303079038976. Current Snapshot: MvccSnapshot[applied={T|T < 7317400303302197248}]
W20260811 18:58:03.461699 6342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.461846 6342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.462613 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.463198 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.481297 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.481676 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.481647 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.482038 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:03.483206 6374 catalog_manager.cc:1347] Generated new cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.483326 6374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 18:58:03.483551 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.483695 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.493520 6370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.493944 6370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
W20260811 18:58:03.500339 6362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:58:03.500505 6362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.500838 6347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.501130 6347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.503328 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:03.503616 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:03.504433 6342 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.504516 6342 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
W20260811 18:58:03.505684 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.505812 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.512491 6362 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:03.512616 6362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: success
W20260811 18:58:03.514058 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.514245 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.517382 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.517516 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.526024 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.526213 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.529563 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.529743 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.538471 6362 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.538616 6362 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.541319 6342 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:58:03.541466 6342 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.543642 6374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:03.545537 6374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:03.553316 6362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: success
W20260811 18:58:03.554896 6362 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.555775 6342 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
W20260811 18:58:03.557116 6342 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.566864 6362 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260811 18:58:03.568836 6342 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 18:58:03.580370 6362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:03.580958 6374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Generated new TSK 0
I20260811 18:58:03.581712 6342 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:03.581945 6374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:03.616145 6395 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:58:03.624933 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.65:0
--local_ip_for_outbound_sockets=127.3.72.65
--webserver_interface=127.3.72.65
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:04.075670 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.075896 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.076956 6397 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:04.076817 6398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.077275 6398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.077646 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:04.077837 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.077983 6149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 1.
I20260811 18:58:04.078555 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.078876 6398 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.079351 6397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:58:04.079520 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6bb6ba710ee24c348d653d524bb73f3d)
I20260811 18:58:04.079689 6397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.079835 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.080456 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:04.082720 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.083144 6294 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a1e448ebc1343319544bc81ea89fed8 in current term 2: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
I20260811 18:58:04.083436 6397 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:04.084148 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8; no voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:04.084568 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:04.085177 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:04.085345 6398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.085810 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:04.086284 6397 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:58:04.088147 6398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260811 18:58:04.152276 6396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:04.152598 6396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:04.152763 6396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:04.160766 6396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:04.161006 6396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.65
I20260811 18:58:04.170941 6396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.72.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.65
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:04.172415 6396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:04.174319 6396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:04.189867 6403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.190303 6404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.190333 6406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.190863 6396 server_base.cc:1034] running on GCE node
I20260811 18:58:04.191651 6396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:04.192638 6396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:04.193976 6396 hybrid_clock.cc:648] HybridClock initialized: now 1786474684193801 us; error 120 us; skew 500 ppm
I20260811 18:58:04.194388 6396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:04.195449 6294 raft_consensus.cc:1226] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Rejecting Update request from peer 6bb6ba710ee24c348d653d524bb73f3d for earlier term 1. Current term is 2. Ops: []
I20260811 18:58:04.196235 6365 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:04.196986 6409 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260811 18:58:04.197088 6409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:04.197415 6409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.197858 6409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:04.203440 6395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:04.204276 6396 webserver.cc:460] Webserver started at http://127.3.72.65:40485/ using document root <none> and password file <none>
I20260811 18:58:04.205212 6396 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:04.205386 6396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:04.205739 6396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:04.208770 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/instance:
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.209607 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance:
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.216593 6396 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 18:58:04.221040 6414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.222846 6396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 18:58:04.223097 6396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
uuid: "b811be76a6a34060af786d6580079a01"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-zpfg"
I20260811 18:58:04.223357 6396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:04.251508 6396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:04.252494 6396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:04.252940 6396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:04.254104 6396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:04.255908 6396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:04.255986 6396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.256112 6396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:04.256160 6396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.297106 6396 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.65:38675
I20260811 18:58:04.297222 6530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.65:38675 every 8 connection(s)
I20260811 18:58:04.298816 6396 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:04.302698 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6396
I20260811 18:58:04.302912 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:04.323276 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:04.323662 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.324679 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:04.324738 6531 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:04.324859 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.325306 6533 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:04.325724 6532 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:04.325878 6532 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.326442 6532 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:04.327411 6284 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.327387 6212 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.328362 6138 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:04.629313 6541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:04.629539 6541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.630412 6541 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.631090 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:04.631100 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:04.631390 6222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 2.
I20260811 18:58:04.631394 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 2.
I20260811 18:58:04.632018 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.632376 6541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 18:58:04.632581 6541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:04.632707 6541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.635377 6541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.636231 6541 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:04.636873 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:04.637182 6222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.637126 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:04.637377 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:04.640111 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 3.
I20260811 18:58:04.640110 6222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 3.
I20260811 18:58:04.640807 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:04.641419 6541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:58:04.642122 6541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:04.642637 6541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:04.644490 6542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:04.644927 6542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:04.645609 6544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:04.648196 6544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:04.649066 6544 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:04.649156 6544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:04.650579 6544 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:04.651449 6544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:04.652603 6544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:04.653432 6544 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:04.654218 6544 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:04.672048 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.66:0
--local_ip_for_outbound_sockets=127.3.72.66
--webserver_interface=127.3.72.66
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:04.832711 6138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.65:37527
W20260811 18:58:05.070482 6545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:05.070899 6545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:05.071051 6545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:05.082237 6545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:05.082494 6545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.66
I20260811 18:58:05.093065 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 18:58:05.093334 6545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.72.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.66
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:05.093919 6542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:05.095343 6545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:05.096531 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.096853 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.097856 6545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:05.099109 6541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.099530 6541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.101063 6222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260811 18:58:05.101996 6553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260811 18:58:05.110949 6549 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.111195 6549 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.112640 6559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.112999 6559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.113138 6554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.113559 6554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:58:05.121903 6558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.122136 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.122162 6562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.123528 6545 server_base.cc:1034] running on GCE node
I20260811 18:58:05.124293 6545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:05.125324 6545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:05.126600 6545 hybrid_clock.cc:648] HybridClock initialized: now 1786474685126517 us; error 58 us; skew 500 ppm
I20260811 18:58:05.126960 6545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:05.130064 6545 webserver.cc:460] Webserver started at http://127.3.72.66:37167/ using document root <none> and password file <none>
I20260811 18:58:05.130960 6545 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:05.131127 6545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:05.131477 6545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:05.134922 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/instance:
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.135954 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance:
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.142940 6545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:58:05.147426 6568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.149206 6545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:05.149436 6545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
uuid: "974ea924500f4de6a519cb4061746395"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.149682 6545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:05.164919 6545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:05.165896 6545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:05.166220 6545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:05.167380 6545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:05.169142 6545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:05.169252 6545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.169420 6545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:05.169493 6545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.208971 6545 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.66:44849
I20260811 18:58:05.209022 6683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.66:44849 every 8 connection(s)
I20260811 18:58:05.210526 6545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:05.213084 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6545
I20260811 18:58:05.213276 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:05.217947 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.72.67:0
--local_ip_for_outbound_sockets=127.3.72.67
--webserver_interface=127.3.72.67
--webserver_port=0
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260811 18:58:05.236925 6685 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:05.237025 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:05.237285 6685 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.237280 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.238565 6684 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:05.239849 6685 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:05.240993 6138 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.241071 6284 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.242309 6138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.66:52437
I20260811 18:58:05.246439 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:05.246670 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.247215 6686 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:05.248554 6212 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:05.336058 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:05.599345 6549 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.599577 6549 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.600513 6549 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:05.601207 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:05.601255 6222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:05.601982 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:05.602392 6549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260811 18:58:05.605434 6554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.605645 6554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.606510 6554 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:05.607038 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:05.607093 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:05.607353 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 3.
I20260811 18:58:05.607874 6199 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:05.608189 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260811 18:58:05.608357 6554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:05.608490 6554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 3 FOLLOWER]: Advancing to term 4
I20260811 18:58:05.610916 6554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.611814 6554 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:05.612473 6149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8"
I20260811 18:58:05.612941 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:05.613283 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 3 FOLLOWER]: Advancing to term 4
I20260811 18:58:05.616600 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 4.
I20260811 18:58:05.617239 6199 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:05.617758 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Leader election won for term 4
I20260811 18:58:05.618238 6554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:05.618589 6554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.619884 6698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 4 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.620316 6698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.621012 6699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:05.623153 6699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:05.624259 6699 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:05.624351 6699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 18:58:05.625135 6694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:05.625599 6694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:05.625780 6694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:58:05.626104 6699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:05.627096 6699 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:05.627674 6700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:05.628319 6700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.628494 6222 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 LEADER]: Rejecting Update request from peer 1a1e448ebc1343319544bc81ea89fed8 for earlier term 3. Current term is 4. Ops: []
I20260811 18:58:05.628851 6699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:05.629923 6699 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:05.631086 6699 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:05.631660 6700 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:58:05.632031 6700 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer 6bb6ba710ee24c348d653d524bb73f3d log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260811 18:58:05.632267 6703 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Stepping down as leader of term 3
I20260811 18:58:05.632383 6703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:05.632742 6703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:05.633384 6703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 3 FOLLOWER]: Advancing to term 4
W20260811 18:58:05.637724 6694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:05.637918 6694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.72.67
I20260811 18:58:05.648341 6694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.72.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.72.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.72.67
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:05.649945 6694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:05.652045 6694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:05.667347 6708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.667470 6706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.667510 6705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.669092 6694 server_base.cc:1034] running on GCE node
I20260811 18:58:05.669891 6694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:05.671204 6694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:05.672538 6694 hybrid_clock.cc:648] HybridClock initialized: now 1786474685672487 us; error 44 us; skew 500 ppm
I20260811 18:58:05.672925 6694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:05.675845 6694 webserver.cc:460] Webserver started at http://127.3.72.67:38631/ using document root <none> and password file <none>
I20260811 18:58:05.676644 6694 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:05.676826 6694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:05.677179 6694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:05.680289 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/instance:
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.681278 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance:
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.687852 6694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 18:58:05.692037 6714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.693789 6694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:05.694020 6694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
uuid: "ef2832ff2b3b4de3b724b81817332809"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-zpfg"
I20260811 18:58:05.694263 6694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:05.718532 6694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:05.719705 6694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:05.720062 6694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:05.721220 6694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:05.723193 6694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:05.723282 6694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.723416 6694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:05.723486 6694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.745836 6685 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:05.764787 6694 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.67:33471
I20260811 18:58:05.764824 6842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.67:33471 every 8 connection(s)
I20260811 18:58:05.766248 6694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:05.767588 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6694
I20260811 18:58:05.767831 3361 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:05.791746 6844 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:05.792080 6844 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.793246 6844 heartbeater.cc:511] Master 127.3.72.126:37881 requested a full tablet report, sending...
I20260811 18:58:05.795297 6139 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.797154 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:05.797186 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:05.797386 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.797390 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.797981 6843 heartbeater.cc:511] Master 127.3.72.124:35445 requested a full tablet report, sending...
I20260811 18:58:05.798122 6846 heartbeater.cc:511] Master 127.3.72.125:34867 requested a full tablet report, sending...
I20260811 18:58:05.799306 6212 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.799443 6284 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:05.800721 6212 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.72.67:39425
I20260811 18:58:05.813747 3361 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:58:05.847095 6212 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.842221 6211 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.846885 6209 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.843432 6208 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.842561 6210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.850971 6205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.852422 6203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.844342 6207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.851495 6204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_1743b9268cc34cdd95330f6add35aa52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:05.849637 6206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:05.870471 6212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_91ae855fc2ac45b7bca9c6f666775f82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:05.890920 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 6179
I20260811 18:58:05.892259 6242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:06.083236 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 6179
I20260811 18:58:06.083429 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260811 18:58:06.083953 6149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Refusing update from remote peer 6bb6ba710ee24c348d653d524bb73f3d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260811 18:58:06.116391 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:06.120529 6270 connection.cc:535] server connection from 127.0.0.1:42342 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.121598 6123 connection.cc:535] server connection from 127.0.0.1:58904 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.124042 6861 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:06.118178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.72.125:34867 (local address 127.0.0.1:50718)
0811 18:58:06.119162 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:06.119186 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:06.119366 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:06.121880 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":580}
I20260811 18:58:06.181262 6899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:06.181821 6899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:06.183612 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:06.183899 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 4.
I20260811 18:58:06.184605 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:06.188113 6899 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:06.189275 6899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260811 18:58:06.189590 6899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:06.189869 6899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 4 FOLLOWER]: Advancing to term 5
I20260811 18:58:06.193252 6899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:06.196925 6899 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:06.201397 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:06.201630 6294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 4 FOLLOWER]: Advancing to term 5
I20260811 18:58:06.204596 6294 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 5.
I20260811 18:58:06.205305 6125 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f; no voters:
I20260811 18:58:06.205773 6901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Leader election won for term 5
I20260811 18:58:06.206151 6901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:06.206676 6901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
W20260811 18:58:06.223069 6124 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:58:06.224440 6124 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.227881 6899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.228170 6899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:06.240398 6904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:06.243229 6904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:06.244462 6904 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:06.244576 6904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:06.246245 6904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:06.247517 6904 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:06.248157 6904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:06.249228 6904 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:06.250317 6904 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:06.252504 6686 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.254528 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.265173 6138 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_1743b9268cc34cdd95330f6add35aa52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.266037 6137 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.265826 6139 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.272629 6135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.274897 6134 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275521 6130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275686 6131 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.273015 6132 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.275774 6136 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:06.273391 6129 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58932:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:06.278467 6138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1743b9268cc34cdd95330f6add35aa52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 18:58:06.285275 6137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fb574299b4df4b758235b87988424ca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:06.303521 6846 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.305369 6846 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.335906 6533 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
W20260811 18:58:06.348979 6533 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:06.445300 6138 catalog_manager.cc:2148] created HMS catalog entry for table [id=d5fd615c99fb4b41b85da755d34202ed]
F20260811 18:58:06.452348 6360 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7efe8762d9ad google::LogMessage::Fail() at ??:0
I20260811 18:58:06.457188 6294 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260811 18:58:06.458030 6899 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
@ 0x7efe8763088a google::LogMessage::SendToLog() at ??:0
@ 0x7efe8762d4de google::LogMessage::Flush() at ??:0
@ 0x7efe8762d6d1 google::LogMessage::~LogMessage() at ??:0
W20260811 18:58:06.465929 6124 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 -> Peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Couldn't send request to peer 6bb6ba710ee24c348d653d524bb73f3d. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7efe9d1cbba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7efe9d1ca901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7efe9d1d166c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260811 18:58:06.499500 6910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.499835 6910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:06.502682 6910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:06.502983 6910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7efe9d1d14e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efe8a257eaf kudu::Thread::SuperviseThread() at ??:0
I20260811 18:58:06.524391 6138 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58932:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:06.526258 6899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
@ 0x7efe8d5316db start_thread at ??:0
I20260811 18:58:06.526611 6899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:06.532058 6137 catalog_manager.cc:2148] created HMS catalog entry for table [id=c920f925a4074d738b2e2720429bf7e0]
@ 0x7efe864c761f clone at ??:0
W20260811 18:58:06.767418 6857 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.767719 6571 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:06.768486 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.126:37881)
I20260811 18:58:06.991760 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260811 18:58:07.025858 6844 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:07.025919 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:07.032001 6933 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.032264 6933 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.033022 6933 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:07.034410 6269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:07.036360 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:07.036664 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:07.036819 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:07.037159 6933 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260811 18:58:07.436733 6932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:07.437152 6932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:07.437330 6932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:07.445037 6932 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:07.445195 6932 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:07.445305 6932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:07.445415 6932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:07.445498 6932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:07.445595 6932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:07.455763 6932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:07.457404 6932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:07.459378 6932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:07.474108 6944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:07.474210 6946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:07.475467 6932 server_base.cc:1034] running on GCE node
W20260811 18:58:07.482851 6943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:07.484447 6932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:07.486219 6932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:07.487516 6932 hybrid_clock.cc:648] HybridClock initialized: now 1786474687487423 us; error 68 us; skew 500 ppm
I20260811 18:58:07.487929 6932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:07.491015 6932 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:07.491772 6932 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:07.491930 6932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:07.498662 6932 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:07.502532 6952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:07.504266 6932 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:07.504491 6932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:07.505637 6932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:07.558557 6932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:07.559609 6932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:07.559949 6932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:07.581235 6932 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:07.581254 7007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:07.583498 6932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:07.589978 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 6932
I20260811 18:58:07.591367 7009 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:07.596053 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:07.605298 7009 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:07.616009 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:07.616842 7009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:07.625736 7009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.627393 7009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:07.628142 7009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.629316 7013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:07.629828 7013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:07.630640 7009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:07.645895 7028 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:07.646130 7028 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:07.650799 7028 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:07.643827 6932 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:07.655301 6932 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:07.655519 7028 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:07.731040 7032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.731303 7032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.731987 7032 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:07.736616 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:07.743002 6980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:07.743654 6980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 4.
I20260811 18:58:07.744334 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:07.744707 7032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260811 18:58:07.744866 7032 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:07.744978 7032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 5 FOLLOWER]: Advancing to term 6
I20260811 18:58:07.748433 7032 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.749297 7032 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
I20260811 18:58:07.750452 6980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:07.750739 6980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 4 FOLLOWER]: Advancing to term 6
W20260811 18:58:07.751484 6269 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:07.758396 6980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19621f5b4944a18b25b0ed2e383027f in term 6.
I20260811 18:58:07.759606 6269 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:07.759996 7032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Leader election won for term 6
I20260811 18:58:07.761157 7032 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 LEADER]: Becoming Leader. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Running, Role: LEADER
I20260811 18:58:07.761770 7032 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:07.764878 7036 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19621f5b4944a18b25b0ed2e383027f. Latest consensus state: current_term: 6 leader_uuid: "e19621f5b4944a18b25b0ed2e383027f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:07.765311 7036 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:07.770195 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.770494 6686 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
I20260811 18:58:07.774204 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:37365; Asking this server to re-register.
I20260811 18:58:07.775478 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:07.777623 6970 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:07.779181 7039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:07.782810 7039 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:07.785185 7039 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:07.785574 7039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:07.786830 7039 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:07.786931 7039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:07.788800 7039 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:07.790285 7039 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Loaded TSK: 0
I20260811 18:58:07.791868 7039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:07.793068 7039 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:07.794191 7039 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:07.812455 6284 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42346:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:07.812943 6281 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.813508 6280 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.814002 6279 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816380 6277 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816943 6274 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.816982 6275 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.818698 6276 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.819223 6278 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:07.819912 6282 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42346:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:07.823487 6341 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
I20260811 18:58:07.825162 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.825490 6846 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
@ 0x7fd4fdd059ad google::LogMessage::Fail() at ??:0
I20260811 18:58:07.826812 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:43675; Asking this server to re-register.
I20260811 18:58:07.827562 6846 heartbeater.cc:464] Registering TS with master...
@ 0x7fd4fdd0888a google::LogMessage::SendToLog() at ??:0
I20260811 18:58:07.829409 6970 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
@ 0x7fd4fdd054de google::LogMessage::Flush() at ??:0
@ 0x7fd4fdd056d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fd5138a3ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fd5138a2901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fd5138a966c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fd5138a94e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd50092feaf kudu::Thread::SuperviseThread() at ??:0
I20260811 18:58:07.868878 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:07.869145 6533 heartbeater.cc:503] Master 127.3.72.125:34867 was elected leader, sending a full tablet report...
@ 0x7fd503c096db start_thread at ??:0
I20260811 18:58:07.870368 6970 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:56579; Asking this server to re-register.
I20260811 18:58:07.871397 6533 heartbeater.cc:464] Registering TS with master...
@ 0x7fd4fcb9f61f clone at ??:0
I20260811 18:58:07.872831 6970 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
W20260811 18:58:08.079995 6858 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.124:35445)
W20260811 18:58:08.080804 6858 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.124:35445)
I20260811 18:58:08.311600 6843 heartbeater.cc:503] Master 127.3.72.124:35445 was elected leader, sending a full tablet report...
I20260811 18:58:08.384094 7052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:08.384392 7052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:08.386996 7052 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:08.388090 6956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:08.389807 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:08.390092 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:08.390305 6956 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:08.390813 7052 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:09.002782 7057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:09.003034 7057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:09.004098 7057 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:09.006596 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:09.006817 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:09.006976 6957 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:09.007301 7057 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:09.627568 7061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:09.627797 7061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:09.629622 7061 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:09.631485 6957 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:09.631703 6956 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:09.631840 6956 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:09.632117 7061 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:10.122273 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 6932
I20260811 18:58:10.137323 7003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:10.257462 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 6932
I20260811 18:58:11.151294 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:11.534826 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:11.535144 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:11.535224 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:11.544495 7069 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:11.544624 7069 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:11.544667 7069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:11.544729 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:11.544770 7069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:11.544829 7069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:11.554808 7069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:11.556344 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:11.558717 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:11.572418 7075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:11.572438 7074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:11.572556 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:11.573261 7069 server_base.cc:1034] running on GCE node
I20260811 18:58:11.574139 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:11.575948 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:11.577287 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786474691577158 us; error 80 us; skew 500 ppm
I20260811 18:58:11.577783 7069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:11.580549 7069 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:11.581336 7069 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:11.581496 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:11.587618 7069 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:11.591821 7083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:11.593386 7069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:58:11.593585 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:11.594764 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:11.623047 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:11.624017 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:11.624361 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:11.647418 7069 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:11.647437 7137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:11.649135 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:11.650437 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7069
I20260811 18:58:11.658238 7138 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:11.663384 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:11.672245 7138 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:11.682250 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:11.683076 7138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:11.692831 7138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:11.694249 7138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.695019 7138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:11.695942 7142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:11.696321 7142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:11.697256 7138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:11.715840 7156 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:11.716126 7156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:11.722121 7156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:11.725273 7156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:11.717518 7069 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:11.728484 7069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:11.801949 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.805008 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:52809; Asking this server to re-register.
I20260811 18:58:11.805908 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.807832 7100 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:11.851076 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.852293 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:40289; Asking this server to re-register.
I20260811 18:58:11.853003 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.854477 7100 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:11.895342 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:11.896765 7100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:34299; Asking this server to re-register.
I20260811 18:58:11.897547 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:11.899837 7100 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:12.183928 7142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:12.184216 7142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:12.186159 7142 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:12.186980 7086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:12.188494 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:12.188897 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:12.189069 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:12.189527 7142 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:12.925866 7163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:12.926120 7163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:12.927129 7163 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:12.930639 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:12.930934 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:12.931180 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:12.931533 7163 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:13.451040 7167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:13.451292 7167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:13.451980 7167 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:13.454582 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:13.454792 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:13.454941 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:13.455235 7167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:14.076944 7171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:14.077155 7171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:14.077926 7171 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:14.080896 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
W20260811 18:58:14.081172 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:14.081348 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:14.081650 7171 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:14.755280 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--webserver_interface=127.3.72.126
--webserver_port=35183
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:15.155936 7175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:15.156347 7175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:15.156512 7175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:15.164979 7175 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:15.165210 7175 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:15.165341 7175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:15.165477 7175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:15.165575 7175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:15.165685 7175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:15.177347 7175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.72.126
--webserver_port=35183
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:15.179021 7175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:15.181062 7175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:15.195993 7180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:15.196062 7181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:15.196236 7183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:15.197364 7175 server_base.cc:1034] running on GCE node
I20260811 18:58:15.198863 7175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:15.200845 7175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:15.202145 7175 hybrid_clock.cc:648] HybridClock initialized: now 1786474695202038 us; error 69 us; skew 500 ppm
I20260811 18:58:15.202607 7175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:15.212163 7175 webserver.cc:460] Webserver started at http://127.3.72.126:35183/ using document root <none> and password file <none>
I20260811 18:58:15.213174 7175 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:15.213351 7175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:15.221449 7175 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:58:15.224818 7189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:15.226504 7175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:15.226701 7175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
uuid: "1a1e448ebc1343319544bc81ea89fed8"
format_stamp: "Formatted at 2026-08-11 18:58:01 on dist-test-slave-zpfg"
I20260811 18:58:15.227849 7175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:15.246827 7175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:15.247871 7175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:15.248216 7175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:15.270087 7175 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.126:37881
I20260811 18:58:15.270211 7241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.126:37881 every 8 connection(s)
I20260811 18:58:15.271904 7175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:15.279806 7242 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:15.281828 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7175
I20260811 18:58:15.284410 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap starting.
I20260811 18:58:15.296660 7242 log.cc:824] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:15.305920 6685 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.310240 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:15.310958 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Bootstrap complete.
I20260811 18:58:15.319761 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.323431 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Initialized, Role: FOLLOWER
I20260811 18:58:15.324278 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.325294 7248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:15.325642 7248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:15.326607 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:15.341362 7262 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:15.341511 7262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: loading cluster ID for follower catalog manager: success
I20260811 18:58:15.346840 7262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: acquiring CA information for follower catalog manager: success
I20260811 18:58:15.350137 7262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:15.373560 7175 master_runner.cc:186] Error getting master registration for 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111),
I20260811 18:58:15.383657 7175 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:15.562997 6532 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.563474 6532 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:15.564041 6844 heartbeater.cc:347] Connected to a master server at 127.3.72.126:37881
I20260811 18:58:15.564282 6844 heartbeater.cc:503] Master 127.3.72.126:37881 was elected leader, sending a full tablet report...
I20260811 18:58:15.566920 7206 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:55045; Asking this server to re-register.
I20260811 18:58:15.567026 7207 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:33633; Asking this server to re-register.
I20260811 18:58:15.568856 6844 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.569373 6532 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.571136 7206 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:15.571208 7207 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:15.728530 7248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:15.728991 7248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:15.731257 7248 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:15.732177 7194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:15.732266 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:15.732964 7111 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a1e448ebc1343319544bc81ea89fed8 in current term 6: Already voted for candidate e19621f5b4944a18b25b0ed2e383027f in this term.
W20260811 18:58:15.734061 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:15.734253 7194 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8; no voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:15.734707 7248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 5 FOLLOWER]: Advancing to term 6
I20260811 18:58:15.739266 7248 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260811 18:58:15.809684 7207 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:56075; Asking this server to re-register.
I20260811 18:58:15.810441 6685 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:15.811821 7207 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:16.208565 7269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:16.208861 7269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.209764 7269 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:16.222565 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.226066 7217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1a1e448ebc1343319544bc81ea89fed8" is_pre_election: true
I20260811 18:58:16.226886 7217 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6bb6ba710ee24c348d653d524bb73f3d for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260811 18:58:16.227926 7086 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:16.228317 7269 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:16.449712 7277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:16.449976 7277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.450846 7277 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:16.451584 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:16.451906 7111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 6.
I20260811 18:58:16.452591 7193 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters:
I20260811 18:58:16.452950 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260811 18:58:16.453259 7277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:16.453354 7277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 6 FOLLOWER]: Advancing to term 7
W20260811 18:58:16.454066 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.456180 7277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.456832 7277 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: Requested vote from peers 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:16.457507 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a1e448ebc1343319544bc81ea89fed8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d"
I20260811 18:58:16.457768 7111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 6 FOLLOWER]: Advancing to term 7
W20260811 18:58:16.458436 7194 leader_election.cc:334] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:16.462527 7111 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1e448ebc1343319544bc81ea89fed8 in term 7.
I20260811 18:58:16.463217 7193 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d; no voters: e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:16.463651 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 FOLLOWER]: Leader election won for term 7
I20260811 18:58:16.464211 7277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [term 7 LEADER]: Becoming Leader. State: Replica: 1a1e448ebc1343319544bc81ea89fed8, State: Running, Role: LEADER
I20260811 18:58:16.464787 7277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:16.467255 7280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:16.467768 7280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:16.743526 7069 master_runner.cc:186] Error getting master registration for 127.3.72.124:35445: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111),
I20260811 18:58:16.743829 7069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:58:16.932271 7194 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 -> Peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Couldn't send request to peer e19621f5b4944a18b25b0ed2e383027f. Status: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:58:16.994429 7111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Refusing update from remote peer 1a1e448ebc1343319544bc81ea89fed8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260811 18:58:16.995357 7280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:58:16.995759 7280 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [LEADER]: Peer 6bb6ba710ee24c348d653d524bb73f3d log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260811 18:58:16.997853 7111 pending_rounds.cc:76] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Aborting all ops after (but not including) 5
I20260811 18:58:16.998124 7111 pending_rounds.cc:97] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260811 18:58:16.998345 7111 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260811 18:58:17.006273 7289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1e448ebc1343319544bc81ea89fed8. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.006363 7292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.006831 7289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.006887 7292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.007138 7290 mvcc.cc:205] Tried to move back new op lower bound from 7317400315719868416 to 7317400314706460672. Current Snapshot: MvccSnapshot[applied={T|T < 7317400315719868416 or (T in {7317400315719868416})}]
I20260811 18:58:17.009758 7292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.009742 7294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "1a1e448ebc1343319544bc81ea89fed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:17.010035 7292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:17.010161 7294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:17.114589 7282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:17.117508 7282 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:17.119329 7282 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:17.119660 7282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:17.120545 7282 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:17.120654 7282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:17.121866 7282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:17.122826 7282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a1e448ebc1343319544bc81ea89fed8: Loaded TSK: 0
I20260811 18:58:17.123703 7282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:17.124460 7282 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:17.125149 7282 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:17.243397 7206 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243541 7205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243832 7203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.243397 7207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.244823 7202 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36952:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:17.246322 7198 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245195 7204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245253 7199 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.245806 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:17.246572 7201 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:17.251233 7261 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f03187849ad google::LogMessage::Fail() at ??:0
@ 0x7f031878788a google::LogMessage::SendToLog() at ??:0
@ 0x7f03187844de google::LogMessage::Flush() at ??:0
@ 0x7f03187846d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f032e322ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f032e321901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f032e32866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f032e3284e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f031b3aeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f031e6886db start_thread at ??:0
@ 0x7f031761e61f clone at ??:0
W20260811 18:58:17.481992 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.126:37881)
W20260811 18:58:17.482470 6571 connection.cc:535] client connection to 127.3.72.126:37881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:17.483399 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.126:37881)
I20260811 18:58:17.510010 7292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:17.510231 7292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:17.511168 7292 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:17.512176 7087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260811 18:58:17.513841 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:17.514165 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:17.514331 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:17.514662 7292 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.130678 7316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:18.130929 7316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.131613 7316 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:18.134080 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:18.134441 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:18.134560 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:18.134895 7316 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.160869 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:18.579213 7320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:18.579526 7320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:18.579604 7320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:18.588446 7320 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:18.588569 7320 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:18.588614 7320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:18.588670 7320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:18.588711 7320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:18.588778 7320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:18.599332 7320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:18.601086 7320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:18.603224 7320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:18.617705 7325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.617468 7326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.617535 7328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:18.619343 7320 server_base.cc:1034] running on GCE node
I20260811 18:58:18.620464 7320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:18.622380 7320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:18.623720 7320 hybrid_clock.cc:648] HybridClock initialized: now 1786474698623589 us; error 103 us; skew 500 ppm
I20260811 18:58:18.624238 7320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:18.627234 7320 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:18.628122 7320 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:18.628429 7320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:18.634922 7320 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 18:58:18.638706 7335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.640659 7320 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:58:18.640887 7320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:18.642068 7320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:18.663653 7340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:18.663887 7340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.664752 7340 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:18.667657 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:18.667896 7087 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445): Network error: Client connection negotiation failed: client connection to 127.3.72.124:35445: connect: Connection refused (error 111)
I20260811 18:58:18.668056 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d; no voters: 1a1e448ebc1343319544bc81ea89fed8, e19621f5b4944a18b25b0ed2e383027f
I20260811 18:58:18.668354 7340 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260811 18:58:18.680785 7320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:18.681886 7320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:18.682264 7320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:18.706449 7320 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:18.706557 7393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:18.708173 7320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:18.710819 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7320
I20260811 18:58:18.716590 7394 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:18.720460 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:18.731840 7394 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:18.744320 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:18.745035 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:18.755316 7394 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.757016 7394 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:18.757952 7394 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:18.759254 7398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:18.759797 7398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:18.760798 7394 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:18.784077 7413 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:18.784302 7413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:18.776634 7320 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:18.787835 7320 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:18.789924 7413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:18.794412 7413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:18.797214 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:18.800978 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:58771; Asking this server to re-register.
I20260811 18:58:18.802157 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.804374 7357 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:18.851450 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:18.851722 6843 heartbeater.cc:503] Master 127.3.72.124:35445 was elected leader, sending a full tablet report...
I20260811 18:58:18.852751 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:45467; Asking this server to re-register.
I20260811 18:58:18.853403 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.854700 7357 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:19.120090 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:19.121390 7357 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:35365; Asking this server to re-register.
I20260811 18:58:19.122077 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:19.123684 7357 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:19.212527 7398 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:19.212878 7398 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.215232 7398 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:19.216292 7338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:19.218794 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.224226 7111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e19621f5b4944a18b25b0ed2e383027f" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6bb6ba710ee24c348d653d524bb73f3d" is_pre_election: true
I20260811 18:58:19.224599 7111 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e19621f5b4944a18b25b0ed2e383027f in current term 7: Already voted for candidate 1a1e448ebc1343319544bc81ea89fed8 in this term.
I20260811 18:58:19.225584 7338 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:19.226027 7398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 6 FOLLOWER]: Advancing to term 7
I20260811 18:58:19.230790 7398 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260811 18:58:19.332111 7421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:19.332386 7421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.333145 7421 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:19.335733 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.340662 7367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:19.341185 7367 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 7.
I20260811 18:58:19.341707 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:19.342021 7421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260811 18:58:19.342149 7421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1a1e448ebc1343319544bc81ea89fed8)
I20260811 18:58:19.342262 7421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 7 FOLLOWER]: Advancing to term 8
I20260811 18:58:19.344748 7421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.345309 7421 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:19.345927 7367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:19.346304 7367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 7 FOLLOWER]: Advancing to term 8
W20260811 18:58:19.347087 7086 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:19.348814 7367 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 8.
I20260811 18:58:19.349282 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:19.349591 7421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Leader election won for term 8
I20260811 18:58:19.350114 7421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:19.350643 7421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:19.352610 7425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 8 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:19.353107 7425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:19.353868 7427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:19.357038 7427 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:19.358820 7427 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:19.359169 7427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:19.360124 7427 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:19.360208 7427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:19.361423 7427 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:19.362419 7427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:19.363415 7427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:19.364282 7427 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:19.365123 7427 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:19.379349 7098 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379709 7097 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380532 7096 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.381114 7100 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379395 7099 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380581 7094 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.379801 7095 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_007bd8c95ec94672af614d10b2bb2038"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.380630 7093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.382586 7091 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:19.382556 7092 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58770:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:19.386703 7155 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f3a9f9949ad google::LogMessage::Fail() at ??:0
@ 0x7f3a9f99788a google::LogMessage::SendToLog() at ??:0
@ 0x7f3a9f9944de google::LogMessage::Flush() at ??:0
@ 0x7f3a9f9946d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f3ab5532ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f3ab5531901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f3ab553866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f3ab55384e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3aa25beeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3aa58986db start_thread at ??:0
@ 0x7f3a9e82e61f clone at ??:0
W20260811 18:58:19.613148 6417 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:19.613902 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:19.614244 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
I20260811 18:58:20.247174 7446 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:20.247490 7446 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:20.248386 7446 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:20.251160 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:20.251392 7338 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:20.251477 7338 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:20.251751 7446 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:21.003063 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7320
I20260811 18:58:21.019088 7387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:21.136538 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7320
I20260811 18:58:22.065078 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:22.454006 7454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:22.454487 7454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:22.454610 7454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:22.462903 7454 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:22.463021 7454 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:22.463091 7454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:22.463186 7454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:22.463253 7454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:22.463339 7454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:22.473186 7454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:22.474777 7454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:22.476848 7454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:22.491696 7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:22.491734 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:22.491660 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:22.492818 7454 server_base.cc:1034] running on GCE node
I20260811 18:58:22.493657 7454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:22.495466 7454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:22.496719 7454 hybrid_clock.cc:648] HybridClock initialized: now 1786474702496641 us; error 33 us; skew 500 ppm
I20260811 18:58:22.497102 7454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:22.499993 7454 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:22.500773 7454 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:22.500931 7454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:22.506958 7454 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:22.510306 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:22.512290 7454 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:58:22.512897 7454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:22.514684 7454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:22.546968 7454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:22.547966 7454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:22.548309 7454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:22.571120 7454 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:22.571269 7523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:22.572866 7454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:22.576802 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7454
I20260811 18:58:22.581768 7524 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:22.585700 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:22.596342 7524 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:22.608453 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:22.609170 7524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:22.617673 7524 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:22.619760 7524 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:22.620573 7524 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:22.621553 7529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:22.622006 7529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:22.623062 7524 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:22.637333 7543 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:22.637522 7543 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:22.648631 7543 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:22.656498 7543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:22.664748 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.660135 7454 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:22.671815 7454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:22.672668 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:41971; Asking this server to re-register.
I20260811 18:58:22.673537 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.675261 7488 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:22.828688 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.830065 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:45081; Asking this server to re-register.
I20260811 18:58:22.830904 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.832340 7488 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:22.876919 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:22.878934 7488 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:37451; Asking this server to re-register.
I20260811 18:58:22.879678 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:22.881137 7488 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:23.129132 7549 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:23.129503 7549 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:23.131852 7549 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:23.132725 7471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.134840 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.135144 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:23.135296 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:23.135783 7549 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:23.811040 7552 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:23.811245 7552 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:23.811863 7552 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:23.814656 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:23.815233 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:23.815411 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:23.815737 7552 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:24.431964 7555 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:24.432193 7555 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:24.432960 7555 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:24.435451 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:24.435662 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:24.435817 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:24.436124 7555 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:25.665565 7559 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:25.665766 7559 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:25.666546 7559 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:25.669133 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:25.669296 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:25.669381 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:25.669612 7559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260811 18:58:26.046603 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--webserver_interface=127.3.72.125
--webserver_port=36521
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:26.427449 7563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:26.427870 7563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:26.428045 7563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:26.436300 7563 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:26.436411 7563 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:26.436489 7563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:26.436592 7563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:26.436687 7563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:26.436800 7563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:26.446713 7563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.125:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.72.125
--webserver_port=36521
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:26.448096 7563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:26.449954 7563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:26.462601 7572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.462641 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.462687 7568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:26.468434 7563 server_base.cc:1034] running on GCE node
I20260811 18:58:26.469488 7563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:26.471302 7563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:26.472623 7563 hybrid_clock.cc:648] HybridClock initialized: now 1786474706472542 us; error 50 us; skew 500 ppm
I20260811 18:58:26.473022 7563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:26.480504 7563 webserver.cc:460] Webserver started at http://127.3.72.125:36521/ using document root <none> and password file <none>
I20260811 18:58:26.481271 7563 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:26.481395 7563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:26.487401 7563 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:26.490741 7578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.492349 7563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:58:26.492558 7563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
uuid: "6bb6ba710ee24c348d653d524bb73f3d"
format_stamp: "Formatted at 2026-08-11 18:58:02 on dist-test-slave-zpfg"
I20260811 18:58:26.493659 7563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:26.516739 7563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:26.517654 7563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:26.517967 7563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:26.539494 7563 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.125:34867
I20260811 18:58:26.539499 7632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.125:34867 every 8 connection(s)
I20260811 18:58:26.541378 7563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-1/data/info.pb
I20260811 18:58:26.547780 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7563
I20260811 18:58:26.549981 7633 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:26.554920 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap starting.
I20260811 18:58:26.567046 7633 log.cc:824] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:26.581060 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:26.581697 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Bootstrap complete.
I20260811 18:58:26.590619 7633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.591827 7633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.592459 7633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.593492 7638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:26.593907 7638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:26.594725 7633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:26.607481 7652 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:26.607698 7652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: loading cluster ID for follower catalog manager: success
I20260811 18:58:26.611864 7652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: acquiring CA information for follower catalog manager: success
I20260811 18:58:26.615330 7652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:26.616937 7563 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:26.627295 7563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:26.642761 6846 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.643400 6533 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.645711 7594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:40415; Asking this server to re-register.
I20260811 18:58:26.645744 7595 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:44173; Asking this server to re-register.
I20260811 18:58:26.646559 6846 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.646886 6533 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.648526 7595 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:26.648576 7594 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:26.871507 6686 heartbeater.cc:347] Connected to a master server at 127.3.72.125:34867
I20260811 18:58:26.872848 7594 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:38645; Asking this server to re-register.
I20260811 18:58:26.873525 6686 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.874888 7594 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:26.977432 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:26.977814 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.979866 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
W20260811 18:58:26.981527 7581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:26.983706 7581 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:26.989078 7498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f" is_pre_election: true
I20260811 18:58:26.989797 7498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 8.
I20260811 18:58:26.990619 7582 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:26.991010 7638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260811 18:58:26.991148 7638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:26.991283 7638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 8 FOLLOWER]: Advancing to term 9
I20260811 18:58:26.995889 7638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:26.997504 7638 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: Requested vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), e19621f5b4944a18b25b0ed2e383027f (127.3.72.124:35445)
I20260811 18:58:26.998181 7498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6bb6ba710ee24c348d653d524bb73f3d" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e19621f5b4944a18b25b0ed2e383027f"
I20260811 18:58:26.998490 7498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 8 FOLLOWER]: Advancing to term 9
W20260811 18:58:27.000160 7581 leader_election.cc:334] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:58:27.003755 7498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb6ba710ee24c348d653d524bb73f3d in term 9.
I20260811 18:58:27.004312 7582 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6bb6ba710ee24c348d653d524bb73f3d, e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8
I20260811 18:58:27.004729 7638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 FOLLOWER]: Leader election won for term 9
I20260811 18:58:27.005241 7638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [term 9 LEADER]: Becoming Leader. State: Replica: 6bb6ba710ee24c348d653d524bb73f3d, State: Running, Role: LEADER
I20260811 18:58:27.005789 7638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:27.008268 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb6ba710ee24c348d653d524bb73f3d. Latest consensus state: current_term: 9 leader_uuid: "6bb6ba710ee24c348d653d524bb73f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:27.008605 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:27.009415 7661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:27.012035 7661 catalog_manager.cc:669] Loaded metadata for table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed]
I20260811 18:58:27.013705 7661 tablet_loader.cc:96] loaded metadata for tablet f92a22485d204195bbc455179d4b4a84 (table default.table_1743b9268cc34cdd95330f6add35aa52 [id=d5fd615c99fb4b41b85da755d34202ed])
I20260811 18:58:27.014037 7661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:27.015039 7661 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:27.015164 7661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:27.016628 7661 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:27.017490 7661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bb6ba710ee24c348d653d524bb73f3d: Loaded TSK: 0
I20260811 18:58:27.018474 7661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:27.019387 7661 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260811 18:58:27.020229 7661 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260811 18:58:27.047457 7595 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_1e31e1132ff54476bf241265a9eefc37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047529 7592 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_453238b625fc46ce9cc3ef19d744a739"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047832 7593 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_fb574299b4df4b758235b87988424ca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.047045 7594 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54082:
table { table_name: "default.Table_1743b9268cc34cdd95330f6add35aa52" } new_table_name: "default.Table_617db2be49cd4c97876d07e22d318d0a" modify_external_catalogs: true
I20260811 18:58:27.049403 7590 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_c4a886a971b746d0b84ff09943a2b8d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049678 7587 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_b4c535b1bf15433bb22c93170702c2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.048995 7588 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_112ed990657d4a509c2662cd07311c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049129 7589 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_9bb9cf75a5f74aa49a9912821ea73672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.048820 7591 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_91ae855fc2ac45b7bca9c6f666775f82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:27.049256 7586 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54082:
name: "default.Table_a5ec20fae0f84dbca051bde9a07e975a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260811 18:58:27.053748 7651 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_91ae855fc2ac45b7bca9c6f666775f82
*** Check failure stack trace: ***
@ 0x7f9044c629ad google::LogMessage::Fail() at ??:0
@ 0x7f9044c6588a google::LogMessage::SendToLog() at ??:0
@ 0x7f9044c624de google::LogMessage::Flush() at ??:0
@ 0x7f9044c626d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f905a800ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f905a7ff901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f905a80666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f905a8064e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f904788ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f904ab666db start_thread at ??:0
@ 0x7f9043afc61f clone at ??:0
W20260811 18:58:27.279080 6717 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:27.279361 6417 connection.cc:535] client connection to 127.3.72.125:34867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:58:27.281122 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
W20260811 18:58:27.281917 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
I20260811 18:58:27.673807 7454 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:27.674050 7454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:28.906725 7678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:28.906991 7678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:28.907785 7678 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:28.909026 7471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260811 18:58:28.910578 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:28.910769 7471 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:28.910862 7471 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:28.911103 7678 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:28.982614 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7454
I20260811 18:58:28.990974 7519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:29.112630 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7454
I20260811 18:58:29.824922 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:30.207415 7686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:30.207782 7686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:30.207916 7686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:30.216292 7686 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:30.216450 7686 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:30.216528 7686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:30.216616 7686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:30.216688 7686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:30.216782 7686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:30.227523 7686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:30.229117 7686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:30.231355 7686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:30.244166 7692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:30.245167 7691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:30.245368 7694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:30.246500 7686 server_base.cc:1034] running on GCE node
I20260811 18:58:30.247448 7686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:30.249172 7686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:30.250505 7686 hybrid_clock.cc:648] HybridClock initialized: now 1786474710250418 us; error 37 us; skew 500 ppm
I20260811 18:58:30.250955 7686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:30.253839 7686 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:30.254781 7686 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:30.254931 7686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:30.261178 7686 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:30.264597 7701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:30.266094 7686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:58:30.266315 7686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:30.267464 7686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:30.284762 7686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:30.285857 7686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:30.286191 7686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:30.308552 7686 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:30.308586 7753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:30.310088 7686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:30.316403 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7686
I20260811 18:58:30.317387 7754 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:30.321951 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:30.334904 7754 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:30.346849 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:30.347606 7754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:30.356266 7754 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.357605 7754 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:30.358361 7754 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.359400 7758 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:30.359982 7758 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:30.361254 7754 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:30.370199 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.394381 7774 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:30.394516 7774 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:30.399672 7774 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:30.386549 7686 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:30.401043 7686 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:30.402958 7774 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:30.418064 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.421396 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:39531; Asking this server to re-register.
I20260811 18:58:30.422246 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.424360 7719 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:30.715948 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:30.717590 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:32983; Asking this server to re-register.
I20260811 18:58:30.718437 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.719873 7719 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:30.768415 7758 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:30.768783 7758 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:30.771076 7758 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:30.772430 7705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:30.774730 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:30.775112 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:30.775266 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:30.775743 7758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:30.874241 7719 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:59101; Asking this server to re-register.
I20260811 18:58:30.874997 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:30.876370 7719 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:31.453084 7779 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:31.453375 7779 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:31.454185 7779 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:31.457044 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:31.457297 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:31.457458 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:31.457780 7779 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:31.985026 7783 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:31.985240 7783 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:31.985854 7783 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:31.989204 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:31.989494 7705 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:31.989629 7705 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:31.989935 7783 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:32.793278 3361 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task2wDjrE/build/asan/bin/kudu pid 7686
I20260811 18:58:32.807073 7749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:32.926050 3361 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2wDjrE/build/asan/bin/kudu with pid 7686
W20260811 18:58:32.950824 6718 connection.cc:535] client connection to 127.3.72.124:35445 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:58:33.814267 3361 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2wDjrE/build/asan/bin/kudu
/tmp/dist-test-task2wDjrE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--webserver_interface=127.3.72.124
--webserver_port=40599
--builtin_ntp_servers=127.3.72.84:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--hive_metastore_uris=thrift://127.0.0.1:40359
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260811 18:58:34.204828 7791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:34.205165 7791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:34.205324 7791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:34.213058 7791 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260811 18:58:34.213231 7791 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260811 18:58:34.213325 7791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:34.213428 7791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:34.213503 7791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:34.213603 7791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:34.224558 7791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.72.84:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40359
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.72.124:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.72.124
--webserver_port=40599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:34.226066 7791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:34.228116 7791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:34.241853 7796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:34.242003 7797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:34.242698 7799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:34.243450 7791 server_base.cc:1034] running on GCE node
I20260811 18:58:34.244467 7791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:34.246197 7791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:34.247548 7791 hybrid_clock.cc:648] HybridClock initialized: now 1786474714247476 us; error 38 us; skew 500 ppm
I20260811 18:58:34.247948 7791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:34.250706 7791 webserver.cc:460] Webserver started at http://127.3.72.124:40599/ using document root <none> and password file <none>
I20260811 18:58:34.251474 7791 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:34.251626 7791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:34.257733 7791 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:58:34.261178 7805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:34.263032 7791 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 18:58:34.263264 7791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data,/tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
uuid: "e19621f5b4944a18b25b0ed2e383027f"
format_stamp: "Formatted at 2026-08-11 18:58:03 on dist-test-slave-zpfg"
I20260811 18:58:34.264395 7791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:34.301982 7791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:34.303045 7791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:34.303373 7791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:34.324743 7791 rpc_server.cc:307] RPC server started. Bound to: 127.3.72.124:35445
I20260811 18:58:34.324769 7856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.72.124:35445 every 8 connection(s)
I20260811 18:58:34.326505 7791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2wDjrE/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1786474617098566-3361-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:34.333554 7857 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:34.336295 3361 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2wDjrE/build/asan/bin/kudu as pid 7791
I20260811 18:58:34.337266 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap starting.
I20260811 18:58:34.349362 7857 log.cc:824] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:34.360654 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=1 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260811 18:58:34.361279 7857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: Bootstrap complete.
I20260811 18:58:34.369086 7857 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.370491 7857 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19621f5b4944a18b25b0ed2e383027f, State: Initialized, Role: FOLLOWER
I20260811 18:58:34.371080 7857 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.371953 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } } }
I20260811 18:58:34.372368 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:34.373059 7857 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:34.387579 7876 catalog_manager.cc:1259] Loaded cluster ID: bca9dc401c194d1fa2005e4d8684445f
I20260811 18:58:34.387789 7876 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: loading cluster ID for follower catalog manager: success
I20260811 18:58:34.391584 7876 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: acquiring CA information for follower catalog manager: success
I20260811 18:58:34.394950 7876 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:34.386322 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:34.398895 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:34.438792 6684 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.442173 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "974ea924500f4de6a519cb4061746395" instance_seqno: 1786474685199062) as {username='slave'} at 127.3.72.66:55337; Asking this server to re-register.
I20260811 18:58:34.443033 6684 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.444917 7822 ts_manager.cc:194] Registered new tserver with Master: 974ea924500f4de6a519cb4061746395 (127.3.72.66:44849)
I20260811 18:58:34.461942 6843 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.463243 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ef2832ff2b3b4de3b724b81817332809" instance_seqno: 1786474685756147) as {username='slave'} at 127.3.72.67:50885; Asking this server to re-register.
I20260811 18:58:34.463927 6843 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.465098 7822 ts_manager.cc:194] Registered new tserver with Master: ef2832ff2b3b4de3b724b81817332809 (127.3.72.67:33471)
I20260811 18:58:34.742602 6531 heartbeater.cc:347] Connected to a master server at 127.3.72.124:35445
I20260811 18:58:34.743986 7822 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b811be76a6a34060af786d6580079a01" instance_seqno: 1786474684287773) as {username='slave'} at 127.3.72.65:45027; Asking this server to re-register.
I20260811 18:58:34.744664 6531 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:34.745986 7822 ts_manager.cc:194] Registered new tserver with Master: b811be76a6a34060af786d6580079a01 (127.3.72.65:38675)
I20260811 18:58:34.877432 7880 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:34.877775 7880 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:34.879869 7880 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:34.881170 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:34.882921 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:34.883232 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:34.883370 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:34.883824 7880 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:35.412868 7883 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:35.413168 7883 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:35.414115 7883 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:35.417181 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:35.417441 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:35.417572 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:35.417845 7883 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:36.492873 7888 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:36.493101 7888 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:36.493782 7888 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:36.496563 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:36.496794 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:36.496879 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:36.497097 7888 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:36.866720 3361 master-stress-test.cc:578] Killing test thread 1/25
I20260811 18:58:37.642323 7892 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:37.642788 7892 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:37.643662 7892 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:37.646428 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:37.646656 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:37.646776 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:37.647027 7892 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:38.961045 7896 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:38.961324 7896 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:38.962260 7896 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:38.965154 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:38.965356 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:38.965438 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:38.965667 7896 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:39.400697 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:39.400923 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:39.593753 7900 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:39.593940 7900 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:39.594584 7900 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:39.597101 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:39.597448 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:39.597599 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:39.597905 7900 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:43.734455 7903 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:43.734743 7903 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:43.735538 7903 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:43.737308 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260811 18:58:43.739181 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:43.739440 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:43.739605 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:43.739969 7903 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:44.402817 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:44.403035 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:45.003635 7908 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:45.003854 7908 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:45.004544 7908 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:45.007624 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:45.007898 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:45.008037 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:45.008617 7908 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:49.405068 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:49.405514 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:53.826318 7912 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:53.826766 7912 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:58:53.827770 7912 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:58:53.829479 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 18:58:53.831815 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:58:53.832099 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:58:53.832193 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:58:53.832494 7912 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:58:54.407380 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:54.407598 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:58:59.409605 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:58:59.409950 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:04.412015 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:04.412408 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:06.099373 7922 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:06.099666 7922 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:06.100638 7922 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:06.102135 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:06.103814 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:06.104140 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:06.104275 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:06.104616 7922 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260811 18:59:06.395023 6533 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:06.395159 6846 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:06.504822 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (78 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:09.414198 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:09.414541 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:14.416687 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:14.416918 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:18.623854 7931 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:18.624151 7931 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:18.624969 7931 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:18.627087 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:18.628983 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:18.629379 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:18.629485 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:18.629853 7931 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:19.418907 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:19.419219 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:24.426990 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:24.427345 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:29.429617 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:29.429836 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:33.941705 7941 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:33.941988 7941 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:33.943043 7941 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:33.944380 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 18:59:33.945920 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:33.946153 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:33.946235 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:33.946528 7941 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:34.431495 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:34.431697 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:36.075011 7947 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:36.075330 7947 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:36.076049 7947 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:36.079867 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 18:59:36.080094 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 18:59:36.080232 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:36.080551 7947 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:39.433455 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:39.433650 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:44.435492 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:44.435676 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:49.437656 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:49.437933 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:54.439867 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:54.440062 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:55.736479 7961 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:55.736788 7961 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 18:59:55.737772 7961 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 18:59:55.739459 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 18:59:55.744019 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 18:59:55.744342 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 18:59:55.744493 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 18:59:55.744833 7961 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 18:59:59.441952 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 18:59:59.442210 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:04.444458 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:04.444718 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:00:06.406613 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (217 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:06.406651 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (217 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:06.689261 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (198 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:09.062016 7968 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:09.062394 7968 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:09.063426 7968 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:09.065153 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:09.067040 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:09.067288 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:09.067432 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:09.067826 7968 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:09.446271 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:09.446512 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:14.448335 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:14.448549 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:19.450500 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:19.450686 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:23.433378 7977 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:23.433660 7977 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:23.434664 7977 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:23.435853 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:23.437616 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:23.437847 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:23.437983 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:23.438320 7977 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:24.452404 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:24.452589 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:29.454685 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:29.454983 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:34.456832 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:34.457059 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:34.506237 7811 maintenance_manager.cc:419] P e19621f5b4944a18b25b0ed2e383027f: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033371
I20260811 19:00:34.563489 7810 maintenance_manager.cc:643] P e19621f5b4944a18b25b0ed2e383027f: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.056s user 0.022s sys 0.009s Metrics: {"bytes_written":6567,"cfile_init":1,"dirs.queue_time_us":504,"dirs.run_cpu_time_us":362,"dirs.run_wall_time_us":1015,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":685,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":639,"threads_started":2,"wal-append.queue_time_us":457}
I20260811 19:00:34.564803 7811 maintenance_manager.cc:419] P e19621f5b4944a18b25b0ed2e383027f: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 275 bytes on disk
I20260811 19:00:34.565747 7810 maintenance_manager.cc:643] P e19621f5b4944a18b25b0ed2e383027f: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":93,"lbm_reads_lt_1ms":4}
I20260811 19:00:34.835101 7991 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:34.835477 7991 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:34.836282 7991 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:34.837414 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:34.839085 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:34.839300 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:34.839427 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:34.839787 7991 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:39.458923 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:39.459136 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:44.461148 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:44.461371 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:49.463608 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:49.463788 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:51.506369 8058 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:51.506585 8058 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:00:51.507519 8058 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:00:51.509214 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:00:51.510829 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:00:51.511051 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:00:51.511188 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:00:51.511515 8058 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:00:54.465586 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:54.465862 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:59.467804 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:00:59.468052 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:02.258162 8105 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:02.258565 8105 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:02.259464 8105 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:02.260838 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:02.262176 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:02.262508 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:02.262612 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:02.262861 8105 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:04.470180 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:04.470563 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:01:06.586760 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (337 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:06.586810 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (337 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:06.869063 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (318 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:09.221571 8133 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:09.221890 8133 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:09.222695 8133 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:09.224210 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:09.226147 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:09.226379 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:09.226521 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:09.226827 8133 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:09.472057 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:09.472260 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:14.474462 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:14.474716 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:19.477075 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:19.477377 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:22.256177 8183 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:22.256407 8183 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:22.257110 8183 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:22.258765 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:22.260535 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:22.260811 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:22.260967 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:22.261402 8183 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:24.479135 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:24.479444 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:29.481608 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:29.481927 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:34.486135 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:34.486570 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:39.249642 8246 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:39.250025 8246 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:39.251255 8246 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:39.252861 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:01:39.255031 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:39.255342 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:39.255499 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:39.255826 8246 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:39.488129 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:39.488473 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:42.578307 8261 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:42.578610 8261 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:42.579381 8261 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:42.583354 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 19:01:42.583607 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
I20260811 19:01:42.583731 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:42.584079 8261 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:44.490156 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:44.490409 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:49.492689 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:49.492951 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:54.494848 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:54.495091 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:01:56.777765 8312 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:56.778113 8312 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:01:56.779130 8312 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:01:56.781039 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260811 19:01:56.782910 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:01:56.783221 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:01:56.783366 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:01:56.783702 8312 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:01:59.496927 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:01:59.497186 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:04.498886 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:04.499136 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:02:06.774541 6532 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (457 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:06.774806 6844 heartbeater.cc:644] Failed to heartbeat to 127.3.72.126:37881 (457 consecutive failures): Network error: Failed to ping master at 127.3.72.126:37881: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:07.051815 6686 heartbeater.cc:644] Failed to heartbeat to 127.3.72.125:34867 (438 consecutive failures): Network error: Failed to ping master at 127.3.72.125:34867: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:09.521814 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:09.522025 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:10.254267 8362 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:10.254568 8362 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:10.255378 8362 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:10.257041 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:10.258667 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:10.258915 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:10.259030 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:10.259320 8362 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:14.523857 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:14.524057 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:19.526374 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:19.526613 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:22.908085 8404 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:22.908268 8404 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:22.908975 8404 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:22.910416 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:22.911820 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:22.912019 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:22.912081 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:22.912303 8404 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:24.528486 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:24.528668 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:29.530419 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:29.530627 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:31.317351 8435 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:31.317584 8435 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:31.318262 8435 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:31.319643 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:31.323495 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:31.323706 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:31.323800 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:31.324106 8435 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:34.532498 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:34.532684 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:39.465492 8466 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:39.465740 8466 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:39.466653 8466 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:39.468179 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:39.470016 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:39.470305 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:39.470461 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:39.470801 8466 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:39.534384 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:39.534588 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:44.536438 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:44.536624 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:49.538308 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:49.538552 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:49.864441 8501 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:49.864697 8501 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:49.865558 8501 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:49.866459 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:49.869565 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:49.869715 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:49.869778 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:49.869982 8501 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:54.540390 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:54.540573 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:02:58.082775 8527 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:58.083009 8527 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:02:58.083853 8527 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:02:58.085043 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:02:58.086257 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:02:58.086596 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:02:58.086664 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:02:58.086905 8527 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:02:59.542419 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:02:59.542635 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:03.782367 8546 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:03.782583 8546 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a1e448ebc1343319544bc81ea89fed8" member_type: VOTER last_known_addr { host: "127.3.72.126" port: 37881 } } peers { permanent_uuid: "6bb6ba710ee24c348d653d524bb73f3d" member_type: VOTER last_known_addr { host: "127.3.72.125" port: 34867 } } peers { permanent_uuid: "e19621f5b4944a18b25b0ed2e383027f" member_type: VOTER last_known_addr { host: "127.3.72.124" port: 35445 } }
I20260811 19:03:03.783327 8546 leader_election.cc:288] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881), 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867)
W20260811 19:03:03.784561 7808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260811 19:03:03.785763 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1a1e448ebc1343319544bc81ea89fed8 (127.3.72.126:37881): Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111)
W20260811 19:03:03.785969 7808 leader_election.cc:334] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6bb6ba710ee24c348d653d524bb73f3d (127.3.72.125:34867): Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
I20260811 19:03:03.786058 7808 leader_election.cc:302] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19621f5b4944a18b25b0ed2e383027f; no voters: 1a1e448ebc1343319544bc81ea89fed8, 6bb6ba710ee24c348d653d524bb73f3d
I20260811 19:03:03.786278 8546 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e19621f5b4944a18b25b0ed2e383027f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260811 19:03:04.544224 7791 master_runner.cc:186] Error getting master registration for 127.3.72.126:37881: Network error: Client connection negotiation failed: client connection to 127.3.72.126:37881: connect: Connection refused (error 111),
I20260811 19:03:04.544430 7791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:03:06.216778 6857 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
W20260811 19:03:06.217228 6857 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
F20260811 19:03:06.228590 6873 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1e31e1132ff54476bf241265a9eefc37 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1786474986 (unix time) try "date -d @1786474986" if you are using GNU date ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:0*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
W20260811 19:03:06.234799 6857 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.72.125:34867)
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238019 6869 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b4c535b1bf15433bb22c93170702c2e3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260811 19:03:06.229597 6865 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9bb9cf75a5f74aa49a9912821ea73672 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.229628 6866 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c4a886a971b746d0b84ff09943a2b8d2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.230906 6870 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_453238b625fc46ce9cc3ef19d744a739 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.235379 6872 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fb574299b4df4b758235b87988424ca7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237263 6863 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_91ae855fc2ac45b7bca9c6f666775f82 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.237901 6868 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_112ed990657d4a509c2662cd07311c9a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238019 6869 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b4c535b1bf15433bb22c93170702c2e3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)F20260811 19:03:06.238483 6864 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_007bd8c95ec94672af614d10b2bb2038 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.72.126:37881,127.3.72.125:34867,127.3.72.124:35445, num_attempts: 741) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.72.125:34867: connect: Connection refused (error 111)
*** SIGABRT (@0x3e800000d21) received by PID 3361 (TID 0x7fc09b271700) from PID 3361; stack trace: ***
*** Check failure stack trace: ***
W20260811 19:03:06.238891 6857 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.72.125:34867)
@ 0x7fc0ad93da46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fc0b1a7f980 (unknown) at ??:0
@ 0x7fc0aac6be87 gsignal at ??:0
@ 0x7fc0aac6d7f1 abort at ??:0
@ 0x7fc0ad92f9ad google::LogMessage::Fail() at ??:0
@ 0x7fc0ad93288a google::LogMessage::SendToLog() at ??:0
@ 0x7fc0ad92f4de google::LogMessage::Flush() at ??:0
@ 0x7fc0ad930a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x403d42 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:298
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7fc0b63bba4f execute_native_thread_routine at ??:0
@ 0x7fc0b1a746db start_thread at ??:0
@ 0x7fc0aad4e61f clone at ??:0